AWS Distro for OpenTelemetry

Understanding your Configurations

We have 5 different configurations that you can select from, depending on your monitoring service destination. See below for details.

AMP Configurations for metrics

Amazon Managed Services for Prometheus (AMP) allows us to easily establish both monitoring and alerting services for container environments on EKS, ECS, Lambda, and many other AWS services. Specifically, AMP collects metrics in the form of time series data from containers & nodes and stores them. Metrics information is stored with the timestamp at which it was recorded and can contain many forms of data that the user chooses to include such as spans, query duration, etc.

Here is a list of metrics that we can collect using the java-sample-app and AMP configuration.

Metrics	Definition
http_server_active_requests	HTTP requests are messages sent by the client to initiate an action on the server
http_server_duration_count	The average amount of time it takes for a queried metric to deliver
queuesize	Size of HTTP queue requests that provides time for new connections to become available on the server,
runtime_jvm_gc_count	Java virtual machine runtime metrics
runtime_jvm_gc_time	Java virtual machine runtime metrics
runtime_jvm_memory_area	Java virtual machine runtime metrics
runtime_jvm_memory_pool	Java virtual machine runtime metrics
spansExportedByExporter	Number of spans that are exported by the exporter
spansSeenByExporter	Number of spans that are seen by the exporter

Configuring dimensions

While visualizing the Amazon Managed Services for Prometheus (AMP) metrics in Amazon Managed Services for Grafana (AMG), you can query and select a metric using the metric browser. When deploying your configuration, you will have multiple metrics coming from different tasks and clusters. As a result, you can use the following dimensions in your query to identify and analyze the metrics related to specific task or cluster. Below are are the possible dimensions with a description of what each dimension mean.

Metric Dimension	Definition
aws_ecs_cluster_name	ECS Cluster Name
aws_ecs_service_name	ECS Service Name
aws_ecs_task_arn	Task ARN ID
aws_ecs_task_family	Task Family name
aws_ecs_task_id	Task ID
aws_ecs_task_known_status	ECS Task Status
aws_ecs_task_launch_type	ECS Task launch type
aws_ecs_task_version	ECS Task family revision number
cloud_account_id	AWS Account ID
cloud_availability_zone	AWS Availability Zone
cloud_region	AWS region

Below is an example of a configuration which utilizes ECS and AMP to output metrics for you to visualize on a service such as AMG.

ecs-amp.yaml

1extensions:
2  health_check:
3  sigv4auth:
4
5receivers:
6  otlp:
7    protocols:
8      grpc:
9        endpoint: 0.0.0.0:4317
10      http:
11        endpoint: 0.0.0.0:4318
12  awsecscontainermetrics:
13
14processors:
15  batch/metrics:
16    timeout: 60s
17  resourcedetection:
18    detectors:
19      - env
20      - system
21      - ecs
22      - ec2
23  filter:
24    metrics:
25      include:
26        match_type: strict
27        metric_names:
28          - ecs.task.memory.reserved
29          - ecs.task.memory.utilized
30          - ecs.task.cpu.reserved
31          - ecs.task.cpu.utilized
32          - ecs.task.network.rate.rx
33          - ecs.task.network.rate.tx
34          - ecs.task.storage.read_bytes
35          - ecs.task.storage.write_bytes
36          - container.duration
37
38exporters:
39  prometheusremotewrite:
40    endpoint: $AWS_PROMETHEUS_ENDPOINT
41    auth:
42      authenticator: sigv4auth
43    resource_to_telemetry_conversion:
44      enabled: true
45
46service:
47  pipelines:
48    metrics/application:
49      receivers: [otlp]
50      processors: [resourcedetection, batch/metrics]
51      exporters: [prometheusremotewrite]
52    metrics:
53      receivers: [awsecscontainermetrics]
54      processors: [filter]
55      exporters: [prometheusremotewrite]
56
57  extensions: [health_check, sigv4auth]

Amazon CloudWatch Configuration for Application metrics

Amazon CloudWatch collects and processes raw data from Amazon ECS into readable, near real-time metrics. It receives time-series metrics and trace data in real time. Our receiver generates data points and Amazon CloudWatch presents this data within the Amazon CloudWatch console. With Amazon CloudWatch, we able to display more data points for you to observe, one sample count in Amazon CloudWatch is the aggregate of multiple data points in one minute.

Here is a list of metrics that you can receive from your exporter by using the java-sample-app and Amazon CloudWatch Configuration.

Metrics	Definition
http_server_active_requests	HTTP requests are messages sent by the client to initiate an action on the server
http_server_duration_count	The average amount of time it takes for a queried metric to deliver
queuesize	Size of HTTP queue requests that provides time for new connections to become available on the server,
runtime_jvm_gc_count	Java virtual machine runtime metrics
runtime_jvm_gc_time	Java virtual machine runtime metrics
runtime_jvm_memory_area	Java virtual machine runtime metrics
runtime_jvm_memory_pool	Java virtual machine runtime metrics
spansExportedByExporter	Number of spans that are exported by the exporter
spansSeenByExporter	Number of spans that are seen by the exporter

Below is an example of a configuration which utilizes ECS and Amazon CloudWatch.

ecs-cloudwatch.yaml

1extensions:
2  health_check:
3
4receivers:
5  otlp:
6    protocols:
7      grpc:
8        endpoint: 0.0.0.0:4317
9      http:
10        endpoint: 0.0.0.0:4318
11
12processors:
13  batch/metrics:
14    timeout: 60s
15  resourcedetection:
16    detectors:
17      - env
18      - ecs
19      - ec2
20  resource:
21    attributes:
22      - key: TaskDefinitionFamily
23        from_attribute: aws.ecs.task.family
24        action: insert
25      - key: aws.ecs.task.family
26        action: delete
27      - key: InstanceId
28        from_attribute: host.id
29        action: insert
30      - key: host.id
31        action: delete
32      - key: TaskARN
33        from_attribute: aws.ecs.task.arn
34        action: insert
35      - key: aws.ecs.task.arn
36        action: delete
37      - key: TaskDefinitionRevision
38        from_attribute: aws.ecs.task.revision
39        action: insert
40      - key: aws.ecs.task.revision
41        action: delete
42      - key: LaunchType
43        from_attribute: aws.ecs.launchtype
44        action: insert
45      - key: aws.ecs.launchtype
46        action: delete
47      - key: ClusterARN
48        from_attribute: aws.ecs.cluster.arn
49        action: insert
50      - key: aws.ecs.cluster.arn
51        action: delete
52      - key: cloud.provider
53        action: delete
54      - key: cloud.platform
55        action: delete
56      - key: cloud.account.id
57        action: delete
58      - key: cloud.region
59        action: delete
60      - key: cloud.availability_zone
61        action: delete
62      - key: aws.log.group.names
63        action: delete
64      - key: aws.log.group.arns
65        action: delete
66      - key: aws.log.stream.names
67        action: delete
68      - key: host.image.id
69        action: delete
70      - key: host.name
71        action: delete
72      - key: host.type
73        action: delete
74
75exporters:
76  awsemf/application:
77    namespace: ECS/AWSOTel/Application
78    log_group_name: '/aws/ecs/application/metrics'
79    dimension_rollup_option: NoDimensionRollup
80    resource_to_telemetry_conversion:
81      enabled: true
82
83service:
84  pipelines:
85    metrics/application:
86      receivers: [otlp]
87      processors: [resourcedetection, resource, batch/metrics]
88      exporters: [awsemf/application]
89
90  extensions: [health_check]

X-Ray Configuration for traces

AWS X-Ray helps developers analyze and debug production, distributed applications, such as those built using a microservices architecture. With X-Ray, you can understand how your application and its underlying services are performing to identify and troubleshoot the root cause of performance issues and errors.

Using the X-Ray configuration, you can monitor application traces including the performance of API calls to other components or services. For any trace request to your application, you can see detailed information from not only the request and response, but also calls that your application makes to downstream AWS resources, micro-services, databases and HTTP web APIs.

ecs-xray.yaml

1extensions:
2  health_check:
3
4receivers:
5  otlp:
6    protocols:
7      grpc:
8        endpoint: 0.0.0.0:4317
9      http:
10        endpoint: 0.0.0.0:4318
11
12processors:
13  batch/traces:
14    timeout: 1s
15    send_batch_size: 50
16  resourcedetection:
17    detectors:
18      - env
19      - system
20      - ecs
21      - ec2
22
23exporters:
24  awsxray:
25
26service:
27  pipelines:
28    traces:
29      receivers: [otlp]
30      processors: [resourcedetection, batch/traces]
31      exporters: [awsxray]
32
33  extensions: [health_check]

Amazon CloudWatch and X-Ray Configuration for Metrics and Traces

By utilizing this configuration you will be able to see datapoints for both metrics and traces in Amazon CloudWatch and X-Ray.

ecs-cloudwatch-xray.yaml

1extensions:
2  health_check:
3
4receivers:
5  otlp:
6    protocols:
7      grpc:
8        endpoint: 0.0.0.0:4317
9      http:
10        endpoint: 0.0.0.0:4318
11
12processors:
13  batch/traces:
14    timeout: 1s
15    send_batch_size: 50
16  batch/metrics:
17    timeout: 60s
18  resourcedetection:
19    detectors:
20      - env
21      - ecs
22      - ec2
23  resource:
24    attributes:
25      - key: TaskDefinitionFamily
26        from_attribute: aws.ecs.task.family
27        action: insert
28      - key: aws.ecs.task.family
29        action: delete
30      - key: InstanceId
31        from_attribute: host.id
32        action: insert
33      - key: host.id
34        action: delete
35      - key: TaskARN
36        from_attribute: aws.ecs.task.arn
37        action: insert
38      - key: aws.ecs.task.arn
39        action: delete
40      - key: TaskDefinitionRevision
41        from_attribute: aws.ecs.task.revision
42        action: insert
43      - key: aws.ecs.task.revision
44        action: delete
45      - key: LaunchType
46        from_attribute: aws.ecs.launchtype
47        action: insert
48      - key: aws.ecs.launchtype
49        action: delete
50      - key: ClusterARN
51        from_attribute: aws.ecs.cluster.arn
52        action: insert
53      - key: aws.ecs.cluster.arn
54        action: delete
55      - key: cloud.provider
56        action: delete
57      - key: cloud.platform
58        action: delete
59      - key: cloud.account.id
60        action: delete
61      - key: cloud.region
62        action: delete
63      - key: cloud.availability_zone
64        action: delete
65      - key: aws.log.group.names
66        action: delete
67      - key: aws.log.group.arns
68        action: delete
69      - key: aws.log.stream.names
70        action: delete
71      - key: host.image.id
72        action: delete
73      - key: host.name
74        action: delete
75      - key: host.type
76        action: delete
77
78exporters:
79  awsxray:
80  awsemf/application:
81    namespace: ECS/AWSOTel/Application
82    log_group_name: '/aws/ecs/application/metrics'
83    dimension_rollup_option: NoDimensionRollup
84    resource_to_telemetry_conversion:
85      enabled: true
86
87service:
88  pipelines:
89    traces:
90      receivers: [otlp]
91      processors: [resourcedetection, batch/traces]
92      exporters: [awsxray]
93    metrics/application:
94      receivers: [otlp]
95      processors: [resourcedetection, resource, batch/metrics]
96      exporters: [awsemf/application]
97
98  extensions: [health_check]

AMP and X-Ray Configuration for Metrics and Traces

By utilizing this configuration you will be able to see datapoints from both metrics and traces in Amazon Managed Services for Prometheus (AMP) and X-Ray.

ecs-amp-xray.yaml

1extensions:
2  health_check:
3  sigv4auth:
4
5receivers:
6  otlp:
7    protocols:
8      grpc:
9        endpoint: 0.0.0.0:4317
10      http:
11        endpoint: 0.0.0.0:4318
12  awsecscontainermetrics:
13
14processors:
15  batch/traces:
16    timeout: 1s
17    send_batch_size: 50
18  batch/metrics:
19    timeout: 60s
20  resourcedetection:
21    detectors:
22      - env
23      - system
24      - ecs
25      - ec2
26  filter:
27    metrics:
28      include:
29        match_type: strict
30        metric_names:
31          - ecs.task.memory.reserved
32          - ecs.task.memory.utilized
33          - ecs.task.cpu.reserved
34          - ecs.task.cpu.utilized
35          - ecs.task.network.rate.rx
36          - ecs.task.network.rate.tx
37          - ecs.task.storage.read_bytes
38          - ecs.task.storage.write_bytes
39          - container.duration
40
41exporters:
42  awsxray:
43  prometheusremotewrite:
44    endpoint: $AWS_PROMETHEUS_ENDPOINT
45    auth:
46      authenticator: sigv4auth
47    resource_to_telemetry_conversion:
48      enabled: true
49
50service:
51  pipelines:
52    traces:
53      receivers: [otlp]
54      processors: [resourcedetection, batch/traces]
55      exporters: [awsxray]
56    metrics/application:
57      receivers: [otlp]
58      processors: [resourcedetection, batch/metrics]
59      exporters: [prometheusremotewrite]
60    metrics:
61      receivers: [awsecscontainermetrics]
62      processors: [filter]
63      exporters: [prometheusremotewrite]
64
65  extensions: [health_check, sigv4auth]

Understanding your Configurations

Understanding your Configurations

AMP Configurations for metrics

Configuring dimensions

Amazon CloudWatch Configuration for Application metrics

X-Ray Configuration for traces

Amazon CloudWatch and X-Ray Configuration for Metrics and Traces

AMP and X-Ray Configuration for Metrics and Traces

On this page