Adds a metrics capabilities page for Pomerium Zero (#1635)

* moves metrics page location

* rewrites front matter

* first draft of zero metrics

* removes enterprise images

* runs prettier

* migrates reports content

* adds reports redirect

* removes extra tabs block

* runs prettier

* fixes typo

* fixes breaking links

* Update content/docs/capabilities/metrics.md

Co-authored-by: Denis Mishin <[email protected]>

* adds requested feedback

* updates header

* runs prettier

* Update content/docs/capabilities/metrics.md

* adds configure metrics link

---------

Co-authored-by: Denis Mishin <[email protected]>

Author: ZPain8464

content/docs/capabilities/img/metrics/console-ext-datasource-monitoring.png

@@ -1,79 +1,235 @@
 ---
 # cSpell:ignore XPOST tsdb
 
-title: Configure Metrics
+title: Metrics in Pomerium
 sidebar_label: Metrics
-description: Use Prometheus as a metrics data store.
+description: Learn how Pomerium collects and displays metrics in Pomerium Zero and Pomerium Enterprise.
 lang: en-US
-keywords: [pomerium, enterprise pomerium, telemetry, metrics, prometheus]
+keywords:
+  [pomerium, pomerium enterprise, telemetry, metrics, prometheus, pomerium zero]
 ---
 
-Pomerium Enterprise uses Prometheus as a metrics collection back-end. You can configure Pomerium and the Console to talk to an existing Prometheus server, or configure the embedded Prometheus backend. This guide assumes you're running both Pomerium and Pomerium Enterprise on localhost `127.0.0.1`.
+import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem';
 
-:::tip
+Metrics in Pomerium provide observability and monitoring data from your Pomerium deployment. Use metrics to review traffic and its effects on your system.
 
-For production deployments, we suggest using a dedicated Prometheus instance.
+Pomerium exposes standard Prometheus metrics related to its operation. Open-source users must [configure](/docs/reference/metrics) metrics collection and visualization manually.
+
+Pomerium Zero and Pomerium Enterprise provide simplified, built-in metrics collection and visualizations.
+
+## Metrics definitions
+
+- **Traffic**: requests proxied by Pomerium to routes defined in a Pomerium deployment.
+- **Route**: the location of an upstream service protected behind Pomerium. At a minimum, a route consists of a [From](/docs/reference/routes/from) URL and a [To](/docs/reference/routes/to) URL.
+
+## How metrics work
+
+Pomerium collects and aggregates traffic data from your Pomerium deployment. This data includes the:
+
+- request size, duration, and rate
+- number of authorized and denied requests
+- response code distribution
+
+You can view and filter this data in your traffic dashboard to gauge how much demand is placed on your server.
+
+### Filter metrics
+
+#### Time range and routes
+
+Pomerium stores traffic metrics up to the last 30 days of usage. You can filter traffic by time range and routes.
+
+The following time ranges are supported:
+
+- Last 30 days
+- Last 2 weeks
+- Last 7 days
+- Last 24 hours
+- Last 12 hours
+- Last 3 hours
+- Last hour
+- Last 15 minutes (Enterprise only)
+
+When filtering by route, select:
+
+- **All Routes** to review aggregated traffic metrics across all routes defined in your Pomerium deployment.
+- an individual route to review aggregated traffic metrics for that route.
+
+<Tabs>
+<TabItem label="Zero" value="zero">
+
+![Filtering metrics by time range routes in Pomerium Zero](./img/metrics/zero-filter-traffic.gif)
+
+</TabItem>
+<TabItem label="Enterprise" value="enterprise">
+
+![Filtering metrics by time range routes in the Pomerium Enterprise console](./img/metrics/enterprise-filter-traffic.gif)
+
+</TabItem>
+</Tabs>
+
+## Traffic dashboard
+
+<Tabs>
+<TabItem value="zero" label="Zero">
+
+Pomerium Zero collects traffic metrics at the [cluster](/docs/concepts/clusters) level, which includes active replicas. To review the traffic dashboard in Pomerium Zero:
+
+1. In the left-hand sidebar, select **Reports**.
+1. Select **Traffic**.
+
+![Find traffic metrics in Pomerium Zero](./img/metrics/zero-find-metrics.gif)
+
+</TabItem>
+<TabItem value="enterprise" label="Enterprise">
+
+When you access the Enterprise Console, you'll land on the traffic dashboard. Pomerium Enterprise organizes traffic metrics in your deployment with [namespaces](/docs/capabilities/namespacing).
+
+Namespaces follow a hierarchical system. You can view traffic metrics for all namespaces, or a specific namespace, using the namespace dropdown menu. You can filter by:
+
+- Global namespace, which encompasses all namespaces in your deployment.
+- Parent namespace, which includes child namespaces (if any).
+- Child namespace, which displays metrics only for that namespace.
+
+![Selecting a namespace to view its traffic in the Enterprise Console](./img/metrics/enterprise-metrics-namespaces.gif)
+
+:::enterprise
+
+In Pomerium Enterprise, you must configure metrics before you can view them. Metrics are not enabled by default, and are not required to run Pomerium Enterprise. See the [Configure Metrics](/docs/enterprise/configure-metrics) guide to enable metrics in your Enterprise deployment.
 
 :::
 
-## Prepare Pomerium
+</TabItem>
+</Tabs>
+
+### Total and Authorized requests
+
+<Tabs>
+<TabItem value="zero" label="Zero">
+
+The **Total requests** chart shows the total number of proxied requests. The **Authorized requests** chart shows the total number of requests Pomerium authorized and forwarded to an upstream service.
+
+Both charts display the difference in requests between the selected and previous time ranges. ![Displaying the total and authorized requests in Pomerium Zero](./img/metrics/zero-total-and-authorized-requests.png)
+
+The **Authorized Requests** pie chart displays the total number of authorized and denied requests. ![A pie chart showing the number of authorized and denied requests in Pomerium Zero](./img/metrics/zero-authorized-and-denied-chart.png)
+
+</TabItem>
+<TabItem value="enterprise" label="Enterprise">
+
+The **Total requests** chart shows the total number of proxied requests. The **Authorized requests** chart shows the total number of requests Pomerium authorized and forwarded to an upstream service.
+
+The **Healthy Endpoints** chart displays the number of healthy upstream endpoints, and roughly correlates with the number of routes defined in your deployment.
+
+For example, if a route's **To** definition includes [multiple upstream resources](/docs/reference/routes/to#target-multiple-upstream-resources), Pomerium includes these resources in the total sum of healthy endpoints. Pomerium excludes unhealthy endpoints from this total. See Load Balancing - [Active Health Checks](/docs/capabilities/load-balancing#active-health-checks) and [Passive Health Checks](/docs/capabilities/load-balancing#passive-health-checks) for more information.
+
+![Viewing the total and authorized request charts in the Enterprise Console](./img/metrics/enterprise-total-requests.png)
+
+The **Authorized Requests** pie chart displays the total number of authorized and denied requests.
+
+![A pie chart showing the number of authorized and denied requests in Pomerium Enterprise](./img/metrics/enterprise-authorized-requests.png)
+
+The **Healthy Upstream Endpoints** graph shows you the number of healthy endpoints over time. A dip in the graph denotes unhealthy endpoints.
+
+![A graph displaying the number of healthy upstream endpoints in Pomerium Enterprise](./img/metrics/enterprise-healthy-upstream-endpoints-graph.png)
+
+</TabItem>
+</Tabs>
+
+### Request durations
+
+<Tabs>
+<TabItem value="zero" label="Zero">
+
+Request duration measures the amount of time it takes Pomerium to proxy a request in milliseconds (ms). Pomerium Zero provides two request duration charts:
+
+The first chart organizes requests by duration ranges defined along the x-axis. Pomerium sums the total value of requests within each range and calculates the amount as a percentage value.
+
+![A chart displaying request duration in Pomerium Zero](./img/metrics/zero-request-duration.png)
+
+The second chart organizes requests by percentile ranges, date, and time.
+
+![A chart displaying request duration in Pomerium Zero](./img/metrics/zero-request-duration-second-chart.png)
+
+</TabItem>
+<TabItem value="enterprise" label="Enterprise">
+
+Request duration measures the amount of time it takes Pomerium to proxy a request in milliseconds (ms). Pomerium Enterprise provides two request duration charts:
+
+The first chart organizes requests by duration ranges defined along the x-axis. Pomerium sums the total value of requests within each range and calculates the amount as a percentage value.
+
+![A chart displaying request duration in Pomerium Enterprise](./img/metrics/enterprise-request-duration.png)
+
+The second chart organizes requests by percentile ranges, date, and time. You can select a specific percentile range to view more granular data in that range:
+
+![A chart displaying request duration with percentile ranges in Enterprise Console](./img/metrics/enterprise-percentile-ranges.gif)
+
+</TabItem>
+</Tabs>
+
+### Requests rate and response codes
+
+<Tabs>
+<TabItem value="zero" label="Zero">
+
+The **Requests per second** chart calculates the average amount of proxied requests per second over the span of an hour. Requests are organized by date and time, and categorized by the following response status codes:
+
+- **200s** (200-299): successful responses
+- **300s** (300-399): redirection messages
+- **400s** (400-499): client error responses
+- **500s** (500-599): server error responses
+
+![A chart displaying requests per second in Pomerium Zero](./img/metrics/zero-requests-per-second.png)
+
+</TabItem>
+<TabItem value="enterprise" label="Enterprise">
+
+The **Request Rate** chart calculates the average amount of proxied requests per second over the span of an hour.
 
-1. In the Pomerium `pomerium-config.yaml`, define the [`metrics_address`](/docs/reference/metrics#metrics-address) key to a network interface and/or port. For example:
+![A chart displaying requests per second in Pomerium Enterprise](./img/metrics/enterprise-request-rate.png)
 
-   ```yaml title="pomerium-config.yaml"
-   metrics_address: 127.0.0.1:9091
-   ```
+The **Response Codes** chart organizes requests by date and time, and categorizes them with the following response status codes:
 
-   The example above has Pomerium providing metrics at port `9999` on an IP address reachable by the Pomerium Console service.
+- **HTTP 2xx** (200-299): successful responses
+- **HTTP 3xx** (300-399): redirection messages
+- **HTTP 4xx** (400-499): client error responses
+- **HTTP 5xx** (500-599): server error responses
 
-   If you're running Pomerium Enterprise in a distributed environment where the IP address is not known at the time of deployment, you can use the resolvable FQDN of the Pomerium host (`pomerium0.internal.example.com`, for example), or override this key with the environment variable `METRICS_ADDRESS`. We do not recommend exposing this endpoint to public traffic as it can contain potentially sensitive information.
+![A response codes chart in Pomerium Enterprise](./img/metrics/enterprise-response-status-codes.gif)
 
-1. In the Pomerium Enterprise `pomerium-enterprise-config.yaml`, define the `metrics_addr` key to a network interface and/or port. For example:
+</TabItem>
+</Tabs>
 
-   ```yaml title="config.yaml"
-   metrics_addr: 127.0.0.1:9092
-   ```
+### Bytes sent and received, request size
 
-## External Prometheus
+<Tabs>
+<TabItem value="zero" label="Zero">
 
-1. Add the listener to your Prometheus configuration, usually via `prometheus.yml`:
+The **Bytes sent** and **Bytes received** charts display the average amount of bytes sent and received over the span of an hour.
 
-   ```yaml
-   - job_name: 'Pomerium'
-      scrape_interval: 30s
-      scrape_timeout: 5s
-      static_configs:
-         - targets: ['127.0.0.1:9901']
-   - job_name: 'Pomerium Enterprise'
-      scrape_interval: 30s
-      scrape_timeout: 5s
-      static_configs:
-         - targets: ['127.0.0.1:9902']
+![A chart displaying bytes sent and received in Pomerium Zero](./img/metrics/zero-bytes-sent-received.png)
 
-   ```
+</TabItem>
+<TabItem value="enterprise" label="Enterprise">
 
-1. [Reload](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#configuration) the Prometheus configuration:
+The **Request Size** chart organizes requests based on their request size measured in kilobytes (KB).
 
-   ```bash
-   curl -i -XPOST path.to.prometheus:port/-/reload
-   ```
+![A chart displaying response size in Pomerium Enterprise](./img/metrics/enterprise-request-size.png)
 
-1. In the Pomerium Enterprise `config.yaml` file, define the `prometheus_url` key to point to your Prometheus instance(s):
+</TabItem>
+</Tabs>
 
-   ```yaml
-   prometheus_url: http://192.168.122.50:9090
-   ```
+## Runtime dashboard
 
-1. Restart the Pomerium and Pomerium Enterprise services. You should now see route traffic and External Data Source monitoring data in the Enterprise Console:
+<Tabs>
+<TabItem value="zero" label="Zero">
 
-   ![Traffic Data in Pomerium Enterprise](./img/metrics/console-route-traffic.png) ![External Data Source in Pomerium Enterprise](./img/metrics/console-ext-datasource-monitoring.png)
+Runtime metrics are not supported in Pomerium Zero.
 
-## Embedded Prometheus
+</TabItem>
+<TabItem value="enterprise" label="Enterprise">
 
-To take advantage of Prometheus embedded in Pomerium Enterprise, edit Pomerium Console's config file:
+In the **Runtime** dashboard, you can monitor how many system resources Pomerium is consuming. Filter by date range, service, and instance.
 
-```yaml title="config.yaml"
-prometheus_data_dir: /var/lib/pomerium-console/tsdb
-```
+![The Runtime Info page in Pomerium Enterprise](./img/metrics/reports-runtime-fullpage.png)
 
-The directory path can be any location that the `pomerium` system user can write to. The example above uses the default location created by the [OS packages](/docs/enterprise/quickstart).
+</TabItem>
+</Tabs>

content/docs/capabilities/img/metrics/console-route-traffic.png

@@ -85,6 +85,6 @@ Pomerium populates users and groups from your IdP. This data is cached to preven
 You may encounter a situation where you may want to add users that are not directly associated with your corporate IdP service. For example, if you have a corporate Google Workspace account and want to add a contractor with a Gmail account, you would have two options:
 
 - Create a group within your IdP directly with the non-domain users in it. This group can be found and added to Namespaces and Policies.
-- Manually add the user's unique ID. Identify the ID from a user's **Session Details** page, or the [Sessions](/docs/capabilities/reports#sessions) page in [Pomerium Enterprise](/docs/enterprise). A user can see their session ID by navigating to the special `/.pomerium` URL endpoint from any Pomerium-managed route. The unique ID is listed as **Sub** under User Details:
+- Manually add the user's unique ID. Identify the ID from a user's **Session Details** page, or the [Sessions](/docs/capabilities/metrics#sessions-enterprise) page in [Pomerium Enterprise](/docs/enterprise). A user can see their session ID by navigating to the special `/.pomerium` URL endpoint from any Pomerium-managed route. The unique ID is listed as **Sub** under User Details:
 
   ![The User Details page, showing the "sub" data](img/access-control/session-details.png)

content/docs/capabilities/img/metrics/enterprise-authorized-requests.png

@@ -81,11 +81,11 @@ In the **External Data** dashboard, you can import, view, and manage [external d
 | TCP Support | ![Pomerium checkmark](./enterprise/img/pomerium-checkmark.svg) | ![Pomerium checkmark](./enterprise/img/pomerium-checkmark.svg) |
 | Enterprise Console | <ClearIcon /> | ![Pomerium checkmark](./enterprise/img/pomerium-checkmark.svg) |
 | [Enterprise API](/docs/capabilities/enterprise-api) | <ClearIcon /> | ![Pomerium checkmark](./enterprise/img/pomerium-checkmark.svg) |
-| [Session Management](/docs/capabilities/reports#sessions) | <ClearIcon /> | ![Pomerium checkmark](./enterprise/img/pomerium-checkmark.svg) |
+| [Session Management](/docs/capabilities/metrics#sessions) | <ClearIcon /> | ![Pomerium checkmark](./enterprise/img/pomerium-checkmark.svg) |
 | [Namespaces](/docs/capabilities/namespacing) | <ClearIcon /> | ![Pomerium checkmark](./enterprise/img/pomerium-checkmark.svg) |
 | [Directory Sync](/docs/capabilities/directory-sync) | <ClearIcon /> | ![Pomerium checkmark](./enterprise/img/pomerium-checkmark.svg) |
 | [User Impersonation](/docs/capabilities/impersonation) | <ClearIcon /> | ![Pomerium checkmark](./enterprise/img/pomerium-checkmark.svg) |
-| [Deployment History](/docs/capabilities/reports#deployments) | <ClearIcon /> | ![Pomerium checkmark](./enterprise/img/pomerium-checkmark.svg) |
+| [Deployment History](/docs/capabilities/metrics#changesets-and-deployments) | <ClearIcon /> | ![Pomerium checkmark](./enterprise/img/pomerium-checkmark.svg) |
 | [Device Identity](/docs/capabilities/device-identity) | <ClearIcon /> | ![Pomerium checkmark](./enterprise/img/pomerium-checkmark.svg) |
 | [Custom Branding](/docs/capabilities/branding) | <ClearIcon /> | ![Pomerium checkmark](./enterprise/img/pomerium-checkmark.svg) |
 | [Service Accounts](/docs/capabilities/service-accounts) | <ClearIcon /> | ![Pomerium checkmark](./enterprise/img/pomerium-checkmark.svg) |

content/docs/capabilities/img/metrics/enterprise-filter-traffic.gif

@@ -334,6 +334,6 @@ Added support for the Rego [**`print()`**](https://www.openpolicyagent.org/docs/
 [`signing key`]: /docs/reference/signing-key
 [google cloud serverless]: /docs/reference/routes/enable-google-cloud-serverless-authentication
 [policy language]: /docs/capabilities/ppl
-[runtime]: /docs/capabilities/reports.md#runtime
+[runtime]: /docs/capabilities/metrics.md#runtime
 [spdy]: /docs/reference/routes/timeouts#spdy
-[telemetry]: /docs/capabilities/reports.md#traffic
+[telemetry]: /docs/capabilities/metrics.md

content/docs/capabilities/img/metrics/enterprise-healthy-upstream-endpoints-graph.png

@@ -100,6 +100,7 @@ const sidebars = {
         },
         'docs/capabilities/hosted-authenticate-service',
         'docs/capabilities/self-hosted-authenticate-service',
+        'docs/capabilities/metrics',
         'docs/capabilities/mtls-clients',
         'docs/capabilities/mtls-services',
         'docs/capabilities/getting-users-identity',
@@ -157,12 +158,6 @@ const sidebars = {
           type: 'doc',
           label: 'High Availability',
         },
-        {
-          id: 'docs/capabilities/metrics',
-          className: 'enterprise',
-          type: 'doc',
-          label: 'Metrics',
-        },
         {
           id: 'docs/capabilities/namespacing',
           className: 'enterprise',
@@ -175,12 +170,6 @@ const sidebars = {
           type: 'doc',
           label: 'Original User Context',
         },
-        {
-          id: 'docs/capabilities/reports',
-          className: 'enterprise',
-          type: 'doc',
-          label: 'Reports',
-        },
         {
           id: 'docs/capabilities/service-accounts',
           className: 'enterprise',

content/docs/capabilities/img/metrics/enterprise-metrics-namespaces.gif

@@ -164,6 +164,7 @@ https://0-20-0.docs.pomerium.com/category/guides https://0-20-0.docs.pomerium.co
 /docs/releases/enterprise /docs/releases/enterprise/about
 /docs/releases/enterprise/about /docs/enterprise
 /enterprise/reference/reports.html /docs/capabilities/reports
+/docs/capabilities/reports /docs/capabilities/metrics
 
 /enterprise/install/helm.html /docs/releases/enterprise/install/helm
 /enterprise/* /docs/enterprise/:splat