Observability on Board

Episode 06traces

Ansible already traces every task

Ansible ships an OpenTelemetry callback plugin in community.general. One pip install and three lines in ansible.cfg turn every play into a trace, one span per task per host.

Ship telemetry toUse an OTLP endpoint you already have, from another vendor or your platform team. This one needs a one-time setup step: save your endpoint and the episodes fill it in.Set your endpointRun an OpenTelemetry Collector on your machine and watch telemetry arrive in its terminal. No account needed.More detailsShip straight to Bronto. You need an account and an API key with ingest permission.More details

Switch it on

The callback plugin ships in community.general, which the ansible package installs. It imports the OpenTelemetry Python SDK, which Ansible does not ship, so that gets installed first. The plugin is switched on in ansible.cfg; the endpoint and the auth header are the standard OpenTelemetry variables, because the plugin has no config-file key for them. Every play then becomes one trace, with a span per task per host that carries the host name, the module and the result. Needs community.general 9.0 or newer for the protocol line; older versions read OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf from the environment instead.

On the controller, once:

pip install opentelemetry-exporter-otlp

In the ansible.cfg next to the playbook:

[defaults]
callbacks_enabled = community.general.opentelemetry

[callback_opentelemetry]
otel_exporter_otlp_traces_protocol = http/protobuf
otel_service_name = ansible

In the shell or CI job that runs the play, a bastion’s .envrc for example:

export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://ingestion.eu.bronto.io/v1/traces
export OTEL_EXPORTER_OTLP_HEADERS="x-bronto-api-key=$BRONTO_API_KEY"

ansible-playbook site.yml

The protocol line is required: the default is gRPC, and without it the play exits 0 and exports nothing.

On the controller, once:

pip install opentelemetry-exporter-otlp

In the ansible.cfg next to the playbook:

[defaults]
callbacks_enabled = community.general.opentelemetry

[callback_opentelemetry]
otel_exporter_otlp_traces_protocol = http/protobuf
otel_service_name = ansible

In the shell or CI job that runs the play, a bastion’s .envrc for example:

export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:4318/v1/traces

ansible-playbook site.yml

This assumes a Collector listening on localhost:4318; the Local Collector setup shows how to start one.

This uses the endpoint and auth header you saved in the setup guide.

On the controller, once:

pip install opentelemetry-exporter-otlp

In the ansible.cfg next to the playbook:

[defaults]
callbacks_enabled = community.general.opentelemetry

[callback_opentelemetry]
otel_exporter_otlp_traces_protocol = http/protobuf
otel_service_name = ansible

In the shell or CI job that runs the play, a bastion’s .envrc for example:

export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=YOUR_OTLP_ENDPOINT/v1/traces
export OTEL_EXPORTER_OTLP_HEADERS="YOUR_AUTH_HEADER=YOUR_AUTH_VALUE"

ansible-playbook site.yml