AWS Distro for OpenTelemetry

Getting Started with the OTLP Exporters

Getting Started with the OTLP Exporters

An exporter is a component in the OpenTelemetry Collector configured to send data to different systems/back-ends. Different exporters converts OpenTelemetry protocol (OTLP) formatted data to their respective predefined back-end format and exports this data to be interpreted by the back-end or system.



OTLP Protocol

The OpenTelemetry Protocol (OTLP) defines the encoding, transport, and delivery mechanism of telemetry data between telemetry sources, intermediate processes such as collectors and telemetry backends. OTLP is a protocol based on requests and responses, for example the client sends requests and the server corresponds with responses. OTLP is currently implemented over two types of transport systems, gRPC and HTTP, specifying the Protocol Buffers schema (protobuf) used for the payloads. The protobuf schema of the messages is the same for OTLP/HTTP and OTLP/gRPC.

OTLP/HTTP

The OTLP implementation transport system over HTTP uses protobuf payloads either in binary format or JSON. OTLP/HTTP uses HTTP POST requests to send telemetry data from clients to servers. Implementations may use HTTP/1.1 or HTTP/2 transports, if an HTTP/2 connection cannot be established it should fallback to HTTP/1.1 transport.

OTLP/gRPC

OTLP/gRPC sends telemetry data with unary requests in ExportTraceServiceRequest for traces, ExportMetricsServiceRequest for metrics, ExportLogsServiceRequest for logs. The language independent interface types for these mentioned pipeline data can be found here. The client will continuously send sequences of requests to the server and expects to receive a response with each request. You can learn more about the OTLP protocol here.



Setting up A Monitoring Backend

OpenTelemetry can export traces, logs, and metrics to various backends to analyze in order to understand an application’s performance and behavior. There are multiple monitoring backends (also known as end-points) which can support OpenTelemetry using the OTLP protocol.

In this section, we share getting started configurations to the Collector to export telemetry data to AppDynamics, Honeycomb, Lightstep, New Relic, and SumoLogic end-points.



Prerequisites

For using any backends supported by OpenTelemetry, make sure you have set up the Collector.



AppDynamics

AppDynamics supports OpenTelemetry by ingesting OTLP directly, so users of the AWS Distro for OpenTelemetry (ADOT) can send tracing data directly to  AppDynamics without the need for additional plugins or non-OTLP exporters.

Requirements

Before you can use the AWS Distro for OpenTelemetry with the AppDynamics endpoint, you need AppDynamics SaaS Controller >= v21.2.0.

Configuration (Collector)

The configuration takes place in the OTLP exporter in the Collector config YAML file.

  • Set the OTLP endpoint through the OTLP HTTP Exporter. To configure your AppDynamics Controller to work with the ADOT Collector, edit your otel-config.yml configuration file.
  • Set the AppDynamics API key <x-api-key> (Your AppDynamics API key must be defined as an HTTP header. To obtain your unique x-api-key, you should work closely with your AppDynamics account team.)
  • Use resource attributes to add your AppDynamics account information:
    • appdynamics.controller.account: Your AppDynamics Controller account name.
    • appdynamics.controller.host: your AppDynamics Controller host name.
    • appdynamics.controller.port: your AppDynamics Controller port number.
    • service.name your AppDynamics tier name. trace resource attribute for every service being monitored.
    • service.namespace: your AppDynamics application name. Set corresponding service.namespace trace resource attribute for every service being monitored.

For custom attributes, see Ingest OpenTelemetry Trace Data

Example 

processors:
  resource:
    attributes:
    - key: appdynamics.controller.account
      action: upsert
      value: "acme"
    - key: appdynamics.controller.host
      action: upsert
      value: "acme.saas.appdynamics.com"
    - key: appdynamics.controller.port
      action: upsert
      value: 443
  batch:
    timeout: 30s
    send_batch_size: 8192
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: localhost:4317
      http:
        endpoint: localhost:4318
exporters:
  otlphttp:
    endpoint: "https://pdx-sls-agent-api.saas.appdynamics.com"
    headers: {"x-api-key": "****************"}
service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [resource, batch]
      exporters: [otlphttp]

Dynatrace

Dynatrace supports native OpenTelemetry protocol (OTLP) ingest, so users of the AWS Distro for OpenTelemetry (ADOT) can send traces, metrics and logs directly to Dynatrace without the need for additional plugins or exporters.

Requirements

OTLP Trace Ingest

  • Dynatrace 1.222 or later
  • An API token with the Ingest OpenTelemetry traces scope

OTLP Metric Ingest

  • Dynatrace 1.254 or later
  • An API token with the Ingest metrics scope
  • Delta metrics: Dynatrace only supports ingesting metrics with delta aggregation temporality. To learn more about aggregation temporality and how to configure your applications or collectors, please refer to the Dynatrace documentation.

OTLP Logs Ingest

  • Dynatrace 1.269 or later
  • An API token with the Ingest logs scope

Configuration (Collector)

The configuration takes place in the OTLP exporter in the Collector config YAML file. Configure the OTLP HTTP Exporter with the following settings. For additional details see the Dynatrace documentation for the Collector.

endpoint

Your Dynatrace environment OTLP ingest endpoint https://{your-environment-id}.live.dynatrace.com/api/v2/otlp

headers

Set the Authorization header with the format "Api-Token <API_TOKEN>"

Example 

receivers:
  otlp:
    protocols:
      grpc:
      http:
exporters:
  otlphttp:
    endpoint: "https://{your-environment-id}.live.dynatrace.com/api/v2/otlp"
    headers:
      Authorization: "Api-Token <API_TOKEN>"
service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: []
      exporters: [otlphttp]
    metrics:
      receivers: [otlp]
      processors: []
      exporters: [otlphttp]
    logs:
      receivers: [otlp]
      processors: []
      exporters: [otlphttp]

Honeycomb

Honeycomb supports OpenTelemetry by ingesting OTLP directly, so users of the AWS Distro for OpenTelemetry (ADOT) can send tracing data directly to Honeycomb without the need for additional plugins or non-OTLP exporters.

Requirements

Before you can use the AWS Distro for OpenTelemetry with the Honeycomb endpoint, you need:

  • You will need a Honeycomb account, if you don’t currently have one you can sign up here.
  • An API key for the Honeycomb Environment you're sending data to

Configuration (Collector)

The configuration will take place in the OTLP exporter in the Collector config YAML file.

  • Set the OTLP endpoint to api.honeycomb.io:443
  • Add your Honeycomb API key as an OTLP header (you can find your API key under Environment settings)
  • The name of a dataset for metrics, if you're sending them

Example

To send trace data, all you need is the API key for your Environment:

1# Honeycomb Collector configuration
2exporters:
3  otlp/traces:
4    endpoint: api.honeycomb.io:443
5    headers:
6      # You can find your Honeycomb API key under Environment settings
7      "x-honeycomb-team":"<YOUR_API_KEY>"
8service:
9  extensions: []
10  pipelines:
11    traces:
12      receivers: [otlp]
13      processors: []
14      exporters: [otlp/traces]

To send metrics data, you also need to specify the dataset for metrics data:

1# Honeycomb Collector configuration
2exporters:
3  otlp/metrics:
4    endpoint: api.honeycomb.io:443
5    headers:
6      # You can find your Honeycomb API key under Environment settings
7      "x-honeycomb-team":"<YOUR_API_KEY>"
8      "x-honeycomb-dataset": "<YOUR_METRICS_DATASET>"
9service:
10  extensions: []
11  pipelines:
12    metrics:
13      receivers: [otlp]
14      processors: []
15      exporters: [otlp/metrics]

See Honeycomb's OpenTelemetry Collector docs to learn about additional configuration options.

Support

If you have any trouble using the AWS Distro for OpenTelemetry with Honeycomb, you can reach out to the ADOT support team or directly to the Honeycomb support page.



Lightstep

Lightstep supports OpenTelemetry natively, via OTLP. If you’re already set up with AWS Distro for OpenTelemetry, then getting data into Lightstep only requires an edit to the YAML config file for the Collector to get started.

Requirements

Before you can use the AWS Distro for OpenTelemetry with Lightstep, you need:

  • A Lightstep account. If you don't already have one, you can create a free account here.
  • An access token for your Lightstep project. This can be found in project settings (the gear icon in the sidebar).

Configuration (Collector)

The configuration will take place in the OTLP exporter in the Collector config YAML file.

  • Configure the Collector to export OTLP.
  • Set the OTLP endpoint to point to Lightstep.
  • Add your Lightstep access token as an OTLP header.
    • Header name: "lightstep-access-token"

Example

1# Lightstep Collector configuration
2exporters:
3  otlp:
4     # NOTE: if you are using private satellites, replace this public
5     # endpoint with the address of your satellite load balancer.
6    endpoint: ingest.lightstep.com:443
7     # Your access token can be found in the project settings page
8    headers: {"lightstep-access-token":"<YOUR_ACCESS_TOKEN>"}


New Relic

New Relic supports OpenTelemetry natively, via OTLP. If you’re already set up with the AWS Distro for OpenTelemetry, then sending data to New Relic can be accomplished with a simple change to the collector's YAML config file.

Requirements

Before you can use the AWS Distro for OpenTelemetry with New Relic, you will need:

Configuration (Collector)

The configuration will take place in the OTLP exporter section of the Collector config YAML file.

  • Set the OTLP endpoint to otlp.nr-data.net:4317
  • Add your New Relic Ingest-License key as an OTLP header.
    • Header name: "api-key"

Example

1# New Relic OTLP Collector configuration
2exporters:
3  otlp:
4    endpoint: otlp.nr-data.net:4317
5    headers: 
6      api-key: <YOUR_NEW_RELIC_LICENSE_KEY>


OpenSearch

OpenSearch supports ingesting enchriched trace data via Data Prepper, a standalone application that converts OLTP formatted data for use in OpenSearch. Data Prepper supports receiving trace data from OpenTelemetry natively via OTLP. Once you've set up a Data Prepper instance, completing the data pipeline is as simple as editing your YAML config file for the Collector and getting started.

Requirements

Before you can use the AWS Distro for OpenTelemetry with OpenSearch, you need:

  • A Data Prepper instance, configured to write to your OpenSearch cluster. Configuration documentation can be found here.

Configuration (Collector)

The configuration will take place in the OTLP exporter in the Collector config YAML file.

  • Configure the Collector to export OTLP.
  • Set the OTLP endpoint to that of your Data Prepper instance or cluster.

Example

1# Data Prepper Collector configuration
2exporters:
3  otlp/data-prepper:
4     # Port 21890 is the default port exposed by Data Prepper.
5    endpoint: <YOUR_DATA_PREPPER_ADDRESS>:21890
6

7service:
8  pipelines:
9    traces:
10      exporters: [otlp/data-prepper]


Sumo Logic

Sumo Logic supports tracing, metrics and logs telemetry signals from OpenTelemetry natively via OTLP. If you’re already set up with AWS Distro for OpenTelemetry, then exporting data into a SumoLogic backend is as simple as editing your YAML config file for the Collector and getting started.

Requirements

Before you can use the AWS Distro for OpenTelemetry with Sumo Logic you need:

  • A Sumo Logic account. If you don't already have one, you can create an account here.
  • An OTLP/HTTP endpoint URL. Instructions how to get one are available here.

Configuration (Collector)

The configuration will take place in the batch processor and OTLP/HTTP exporter in the Collector config YAML file.

Example

1# SumoLogic Collector configuration
2processors:
3  batch:
4    timeout: 5s
5    send_batch_size: 512
6    
7exporters:
8  otlphttp:
9    endpoint: https://YOUR_SUMOLOGIC_OTLP_HTTP_ENDPOINT_URL
10

11service:
12  pipelines:
13    traces:
14      exporters: [otlphttp]
15      processors: [batch]
16    metrics:
17      exporters: [otlphttp]
18      processors: [batch]
19    logs:
20      exporters: [otlphttp]
21      processors: [batch]

If you are instrumenting your application using OpenTelemetry JavaScript, Java, Python, Go, Ruby, .NET you can use the SumoLogic documentation to set up your application and obtain telemetry data.

Support

If you have any trouble using the AWS Distro for OpenTelemetry with Sumo Logic, you can reach out to the ADOT support team or directly to the Sumo Logic support page.



Questions, Feedback?

We would love to hear more common configuration scenarios or improvements to this documentation from you! Please submit an issue on the aws-otel community page to let us know.

On this page