AWS Distro for OpenTelemetry

Using CloudWatch Metrics with AWS Distro for OpenTelemetry

Using CloudWatch Metrics with AWS Distro for OpenTelemetry

To support OpenTelemetry Metrics in Amazon CloudWatch. CloudWatch team contributed AWS EMF Exporter for OpenTelemetry Collector as part of AWS OpenTelemetry that converts OpenTelemetry format metrics to CloudWatch Embedded Metric Format(EMF). It enables customer applications integrating OpenTelemetry metrics with CloudWatch and has ability to send high-cardinality application metrics which is well supported by EMF.

In this tutorial, we will walk through how to enable CloudWatch metrics in the ADOT Collector.



Getting Started

The following diagram shows a high-level picture of how the application metrics are generated with AWS Observability toolkits.

Diagram

To get started using ADOT Collector to send application metrics, you must first install ADOT Collector in your computing environment for collecting metrics from various of applications.

For more information about installing ADOT Collector for each of the AWS platforms, see the getting started section for

We will discuss how to configure the ADOT Collector to send different application metrics.

AWS Distro for OpenTelemetry Collector supports CloudWatch metrics with the default configuration ADOT Collector provides a default configuration out of box that enables CloudWatch metrics. The default configuration includes the essential components for metric enabling in ADOT Collector.

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: localhost:4317
      http:
        endpoint: localhost:4318
processors:
  batch:
    timeout: 60s
exporters:
  awsemf:
service:
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [awsemf]

This out-of-box configuration from ADOT Collector has supported CloudWatch metrics, but we provide more options to do the advanced customization for each component on metrics sending. We will walk through some of the available configuration options below.

CloudWatch EMF Exporter (awsemf)

The application metrics are sent to Amazon CloudWatch in form of EMF Logs format. The various configuration options, including EMF Log Group Name, Log Stream Name and CloudWatch Metrics Namespace can be customized in EMF exporter configuration. Note that ADOT Collector utilizes the default log group name and namespace generated from application to support data correlation. The customized values might break the data correlation feature. The log retention of the Cloudwatch log group created by the EMF Exporter can also be customized in the configuration and be set to the following values (in days):

0 (Never Expire), 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 2192, 2557, 2922, 3288, or 3653.

Log Retention can only be set this way for newly created Cloudwatch log groups. This feature will not update log retention for existing Cloudwatch log groups.

The available dimension roll-up options in EMF exporter,

  • "ZeroAndSingleDimensionRollup" - Enable both zero dimension rollup and single dimension rollup
  • "SingleDimensionRollupOnly" - Enable single dimension rollup
  • "NoDimensionRollup" - No dimension rollup (only keep original metrics which contain all dimensions)
awsemf:
    log_group_name: 'ADOTLogGroup'
    log_stream_name: 'ADOTLogStream'
    namespace: 'ADOTMetricNS'
    dimension_rollup_option: 1
    log_retention: 60

The following is are sample metrics with dimension rollup in CloudWatch console.

Diagram

Batch processor (batch) ADOT Collector supports all the components from OpenTelemetry Collector. The batch processor is enabled by default to improve CloudWatch EMF request through-put. With this setting, the received OpenTelemetry metrics data will be batched in ADOT Collector metrics pipeline when either the timeout 30s or batch size (8192) is reached, the batched metrics data will be sent to CloudWatch backend in an efficient fashion.

  • send_batch_size (default = 8192): Number of spans or metrics after which a batch will be sent.
  • timeout (default = 30s): Time duration after which a batch will be sent regardless of size.
processors:
  batch:
    timeout: 30s
    send_batch_size: 10000

For new updates and configuration options on CloudWatch Metrics support, follow us on AWS EMF Repo.

On this page