Prometheus Metrics Collector Configuration

The Promethes Metrics collector configuration launches a preconfigured OpenTelemetry Collector custom resource to scrape prometheus metrics endpoints. By opting into the available pipelines you can control where the collected metrics are sent.

Prerequisites

Amazon Managed Service for Prometheus Workspace

A workspace is required if you are planning on sending metrics to Amazon Managed Service for Prometheus (AMP). See the AMP getting started guide for detailed creation steps.

Setup an IAM role to associate with the service account

An IAM Role with the following policies must be created for the following Kubernetes service account and namespace.

Note: Only attach the minimum set of policies necessary for your advanced configuration.

The IAM Roles for Service Accounts documentation (IRSA) contains instructions for creating the IAM role. The following examples will use eksctl to achieve this.

To create this IAM role, run the following command:

Copy
eksctl create iamserviceaccount \
    --name adot-col-prom-metrics \
    --namespace opentelemetry-operator-system \
    --cluster <your_cluster_name> \
    --attach-policy-arn arn:aws:iam::aws:policy/AmazonPrometheusRemoteWriteAccess \
    --attach-policy-arn arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy \
    --approve \
    --role-only

This IAM role generated by the above command needs to be inserted into the annotations field of the advanced configuration as seen below:

Copy
collector:
  prometheusMetrics:
    serviceAccount:
      annotations:
        eks.amazonaws.com/role-arn: <iam_role_arn>

Prometheus Metrics pipelines

The following pipelines are available for the Prometheus Metrics preconfigured custom resource. Pipelines can be enabled by setting their enabled field to true.

Metrics

• amp

  Metrics scraped by a prometheus receiver are routed to a configured AMP workspace.

• emf

  Metrics scraped by a prometheus receiver are sent to Amazon CloudWatch in Embedded Metric Format (EMF).

Prometheus Metrics Advanced Configuration pipelines

Copy
collector:
  prometheusMetrics:
    serviceAccount:
      annotations:
        eks.amazonaws.com/role-arn: <iam_role_arn>
    pipelines:
      metrics:
        amp:
          enabled: true
        emf:
          enabled: true

Prometheus Metrics exporters

The following exporters can be configured for the Prometheus Metrics preconfigured custom resource.

prometheusremotewrite

• endpoint

  The remote write endpoint associated with the AMP workspace. Required if the amp pipeline is enabled.

Prometheus Metrics Advanced Configuration exporters

Copy
collector:
  prometheusMetrics:
    serviceAccount:
      annotations:
        eks.amazonaws.com/role-arn: <iam_role_arn>
    pipelines:
      metrics:
        amp:
          enabled: true
        emf:
          enabled: true
    exporters:
      prometheusremotewrite:
        endpoint: <remote_write_endpoint>

Prometheus Metrics configurable values

Shown below is the complete list of configurable fields, along with their default values, for the prometheusMetrics resource.

Copy
collector:
  prometheusMetrics:
    resources:
      limits:
        cpu: 1000m
        memory: 750Mi
      requests:
        cpu: 300m
        memory: 512Mi
    serviceAccount:
      annotations:
    pipelines:
      metrics:
        amp:
          enabled: false
        emf:
          enabled: false
    exporters:
      prometheusremotewrite:
        endpoint:

*Note that in Fargate, resource requests and limits must be equal, see this troubleshooting guide for more information.

Previous Topic: Add-on Advanced Configuration: Collector Deployment

Related Topics:

Collector Configuration for OTLP Ingest

Collector Configuration for Container Logs

Next Topic: Updating and Cleanup