Envoy Gateway provides observability for the underlying Envoy Proxy instances. There are currently no metrics available for Envoy Gateway itself.

Envoy Gateway supports exposing metrics for the proxies installed using either:

• Prometheus metrics endpoint
• Sending metrics to an OpenTelemetry sink

TEG doesn’t expose metrics by default, users can enable support for the metrics by following the instructions in this document.

Metrics

Prometheus

TEG metrics can be exposed via Prometheus via the following steps.

Enable Prometheus metrics endpoint

Users can enable the metrics endpoint by applying the following configuration in their TEG installation.

1. Save the below-mentioned configuration in a patch.yaml file.

patch.yaml

spec:
  telemetry:
    metrics:
      prometheus: {}

2. Apply the above configuration in your cluster by running the following:

kubectl patch envoyproxies -n envoy-gateway-system teg-envoy-proxy-config --patch-file patch.yaml --type merge

The Prometheus metrics are exposed on port 19001 and path /stats/prometheus. Users can configure their Prometheus deployment to scrape metrics from the above patch and port.

Configuring Prometheus Pod discovery

If your Prometheus deployment uses Kubernetes pod auto-discovery for scraping metrics, you can configure the pod annotations by:

1. Save the patch manifest in a file named patch.yaml.

patch.yaml

spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyDeployment:
        pod:
          annotations:
            prometheus.io/scrape: 'true'
            prometheus.io/path: '/stats/prometheus'
            prometheus.io/port: '19001'

2. Apply the above patch by running the following:

kubectl patch envoyproxies -n envoy-gateway-system teg-envoy-proxy-config --patch-file patch.yaml --type merge

OpenTelemetry

Envoy supports pushing metrics to an OpenTelemetry sink such as OTEL Collector. Please ensure your OpenTelemetry sink is reachable by the envoy proxy resource.

To configure this:

1. Populate the OpenTelemetry sink configuration in the provided patch manifest and save it in a file, patch.yaml

patch

spec:
  telemetry:
    metrics:
      sinks:
        - type: OpenTelemetry
          openTelemetry:
            host: <HOST> # Host defines the service hostname for your OTEL Sink.
            port: <PORT> # Port defines the port the OTEL sink service is exposed on.

2. Apply the above patch to your envoy proxy configuration by running the following:

kubectl patch envoyproxies -n envoy-gateway-system teg-envoy-proxy-config --patch-file patch.yaml --type merge

Tracing

TEG supports tracing using OpenTelemetry protocol and integrates seamlessly with OpenTelemetry sinks.

You can enable tracing support and configure your OTEL sink by following the below steps:

1. Populate the OpenTelemetry sink configuration in the provided patch manifest and save it in a file, patch.yaml.

patch.yaml

spec:
  telemetry:
    tracing:
       samplingRate: 100 # Default value is 100% sampling.
       provider:
         type: OpenTelemetry
         host: <HOST> # Host defines the service hostname for your OTEL Sink.
         port: <PORT> # Port defines the port the OTEL sink service is exposed on.

Apply the above configuration to your envoy proxy configuration by running the following:

kubectl patch envoyproxies -n envoy-gateway-system teg-envoy-proxy-config --patch-file patch.yaml --type merge

NOTE: EG, by default, uses a 100% sampling rate, which means all requests will be traced. This may cause performance issues. Users can control the sampling rate by configuring spec.telemetry.tracing.samplingRate in the above patch file.

Logs

By default, TEG sends logs to Stdout in default text format. TEG also supports sending logs to an OTEL Sink.

Enable OpenTelemetry Logs sink

1. Populate the OpenTelemetry sink configuration in the provided patch manifest and save it in a file, patch.yaml

patch.yaml

spec:
  telemetry:
    accessLog:
      settings:
      - format:
          type: Text
          text: "" # Blank means default access log format
        sinks:
        - type: File
          file:
            path: /dev/stdout
        - type: OpenTelemetry
          openTelemetry:
            host: <HOST> # Service hostname for your OTEL Sink.
            port: <PORT> # The port where OTEL sink service is exposed.

NOTE: Users can remove the file sink from the above patch if the log sink for stdout is not required.

2. Apply the above configuration to your envoy proxy configuration by running the following:

kubectl patch envoyproxies -n envoy-gateway-system teg-envoy-proxy-config --patch-file patch.yaml --type merge