Lesson 5: Observability (Prometheus & PromQL)
🧠 The Concept (Explain Like I'm 5)
If you drive a car without a dashboard, you won't know you're out of gas until the engine stops. Prometheus is the engine sensor that records speed and gas levels every 5 seconds. Grafana is the beautiful dashboard that shows you the dials.
🏢 The Enterprise Context
- Pull-based Metrics: Prometheus doesn't wait for apps to send data. It actively reaches out to a
/metricsweb page on every Pod every 15 seconds and downloads the numbers (Scraping). - PromQL: The query language used to ask complex questions, like "Give me the 99th percentile HTTP response time over the last 5 minutes."
🗺️ Visual Architecture: The Prometheus Scraping Flow
flowchart LR
subgraph Apps ["Kubernetes Pods"]
App1["Payment Service<br/>(Exposes /metrics)"]
App2["Auth Service<br/>(Exposes /metrics)"]
end
Prom["Prometheus Server"] -->|HTTP GET /metrics<br/>every 15s| Apps
Prom --> TSDB[("Time Series DB")]
Grafana["Grafana Dashboard"] -->|PromQL Queries| TSDB
Prom -->|Threshold Exceeded| Alert["Alertmanager (PagerDuty)"]