AWS Open Distro for OpenTelemetry

Using CloudWatch Metrics with AWS Distro for OpenTelemetry

Using CloudWatch Metrics with AWS Distro for OpenTelemetry

To support OpenTelemetry Metrics in Amazon CloudWatch. CloudWatch team contributed AWS EMF Exporter for OpenTelemetry Collector as part of AWS OpenTelemetry that converts OpenTelemetry format metrics to CloudWatch Embedded Metric Format(EMF). It enables customer applications integrating OpenTelemetry metrics with CloudWatch and has ability to send high-cardinality application metrics which is well supported by EMF.

In this tutorial, we will walk through how to enable CloudWatch metrics in AWS OTel Collector.




Getting Started

The following diagram shows a high-level picture of how the application metrics are generated with AWS Observability toolkits.

Diagram

To get started using AWS OTel Collector to send application metrics, you must first install AWS OTel Collector in your computing environment for collecting metrics from various of applications.

For more information about installing AWS OTel Collector for each of the AWS platforms, see the getting started section for

We will discuss how to configure AWS OTel Collector to send different application metrics.

AWS OpenTelemetry Collector supports CloudWatch metrics with the default configuration AWS OTel Collector provides a default configuration out of box that enables CloudWatch metrics. The default configuration includes the essential components for metric enabling in AWS OTel Collector.

receivers:
otlp:
protocols:
grpc:
endpoint: localhost:55680
http:
endpoint: localhost:55681
processors:
batch:
timeout: 60s
exporters:
awsemf:
service:
pipelines:
metrics:
receivers: [otlp]
processors: [batch]
exporters: [awsemf]

This out-of-box configuration from AWS OTel Collector has supported CloudWatch metrics, but we provide more options to do the advanced customization for each component on metrics sending. We will walk through some of the available configuration options below.

CloudWatch EMF Exporter (awsemf)

The application metrics are sent to Amazon CloudWatch in form of EMF Logs format. The various configuration options, including EMF Log Group Name, Log Stream Name and CloudWatch Metrics Namespace can be customized in EMF exporter configuration. Note that AWS OTel Collector utilizes the default log group name and namespace generated from application to support data correlation. The customized values might break the data correlation feature.

The available dimension roll-up options in EMF exporter,

  • "ZeroAndSingleDimensionRollup" - Enable both zero dimension rollup and single dimension rollup
  • "SingleDimensionRollupOnly" - Enable single dimension rollup
  • "NoDimensionRollup" - No dimension rollup (only keep original metrics which contain all dimensions)
awsemf:
log_group_name: 'AOTLogGroup'
log_stream_name: 'AOTLogStream'
namespace: 'AOTMetricNS'
dimension_rollup_option: 1

The following is are sample metrics with dimension rollup in CloudWatch console.

Diagram

Batch processor (batch) AWS Observability Collector supports all the components from OpenTelemetry Collector. The batch processor is enabled by default to improve CloudWatch EMF request through-put. With this setting, the received OpenTelemetry metrics data will be batched in AWS Observability Collector metrics pipeline when either the timeout 30s or batch size (8192) is reached, the batched metrics data will be sent to CloudWatch backend in an efficient fashion.

  • send_batch_size (default = 8192): Number of spans or metrics after which a batch will be sent.
  • timeout (default = 30s): Time duration after which a batch will be sent regardless of size.
processors:
batch:
timeout: 30s
send_batch_size: 10000

For new updates and configuration options on CloudWatch Metrics support, follow us on AWS EMF Repo.

On this page