I'm using open telemetry to export trace information of the following application:
- A nodejs kafka producer sends messages to
input-topic
. It useskafkajs
instrumented withopentelemetry-instrumentation-kafkajs
library. I'm using the example from AWS OTEL for NodeJS example. Here is mytracer.js
:
module.exports = () => {
diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.ERROR);
// create a provider for activating and tracking with AWS IdGenerator
const attributes = {
'service.name': 'nodejs-producer',
'service.namespace': 'axel'
}
let resource = new Resource(attributes)
const tracerConfig = {
idGenerator: new AWSXRayIdGenerator(),
plugins: {
kafkajs: { enabled: false, path: 'opentelemetry-plugin-kafkajs' }
},
resource: resource
};
const tracerProvider = new NodeTracerProvider(tracerConfig);
// add OTLP exporter
const otlpExporter = new CollectorTraceExporter({
url: (process.env.OTEL_EXPORTER_OTLP_ENDPOINT) ? process.env.OTEL_EXPORTER_OTLP_ENDPOINT : "localhost:55680"
});
tracerProvider.addSpanProcessor(new SimpleSpanProcessor(otlpExporter));
tracerProvider.addSpanProcessor(new SimpleSpanProcessor(new ConsoleSpanExporter()));
// Register the tracer with X-Ray propagator
tracerProvider.register({
propagator: new AWSXRayPropagator()
});
registerInstrumentations({
tracerProvider,
instrumentations: [new KafkaJsInstrumentation({})],
});
// Return a tracer instance
return trace.getTracer("awsxray-tests");
}
- A Java application that reads from
input-topic
and produces tofinal-topic
. Also instrumented with AWS OTEL java agent. Java app is launched like below:
export OTEL_TRACES_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
export OTEL_PROPAGATORS=xray
export OTEL_RESOURCE_ATTRIBUTES="service.name=otlp-consumer-producer,service.namespace=axel"
export OTEL_METRICS_EXPORTER=none
java -javaagent:"${PWD}/aws-opentelemetry-agent.jar" -jar "${PWD}/target/otlp-consumer-producer.jar"
- I'm using
otel/opentelemetry-collector-contrib
that has AWS XRay exporter:
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
exporters:
awsxray:
region: 'eu-central-1'
max_retries: 10
service:
pipelines:
traces:
receivers: [otlp]
exporters: [awsxray]
I can see from log messages and also from XRay console that traces are being published (with correct parent trace ids). NodeJS log message:
{
traceId: '60d0c7d4cfc2d86b2df8624cb4bccead',
parentId: undefined,
name: 'input-topic',
id: '3e289f00c4499ae8',
kind: 3,
timestamp: 1624295380734468,
duration: 3787,
attributes: {
'messaging.system': 'kafka',
'messaging.destination': 'input-topic',
'messaging.destination_kind': 'topic'
},
status: { code: 0 },
events: []
}
and Java consumer with headers:
Headers([x-amzn-trace-id:Root=1-60d0c7d4-cfc2d86b2df8624cb4bccead;Parent=3e289f00c4499ae8;Sampled=1])
As you see parent and root ids match each other. However the service map is constructed in a disconnected way:
What other configuration I'm missing here to compile a correct service map?