resilience • Intermediate

Distributed Tracing & Observability (OpenTelemetry & Jaeger)

Distributed Tracing tracks requests across microservice boundaries by propagating W3C TraceContext headers (TraceID, SpanID). Spans capture timing, errors, and database queries across the distributed call graph.

Intuitive Mental Model

The Postal Package Tracking Number: The single tracking number (TraceID) follows the package from the supplier to the airplane, customs warehouse, delivery truck, and your front door (Spans).

Architecture Blueprint & CodeProduction Standard

// W3C TraceContext HTTP Header Propagation:
// traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
// 00: Version
// 4bf92f35...: 128-bit Trace ID (Identifies the entire user request)
// 00f067aa...: 64-bit Parent Span ID
// 01: Trace Flags (Sampled = true)

Key Architectural Takeaways

•Root Cause Analysis: Instantly isolates which microservice in a 20-service dependency chain is adding 800ms latency.
•OpenTelemetry Standard: Universal vendor-neutral telemetry standard supported by Datadog, Jaeger, Honeycomb, and New Relic.

Common Architectural Pitfall

Sampling 100% of traces on high-throughput systems (100,000 QPS), overloading tracing storage backends.

Production Best Practice

Use Head-Based or Tail-Based Sampling (e.g. record 1% of normal requests and 100% of error/slow requests).