# Making OpenTelemetry and Service Mesh Traces Work Together: A Practical Guide
Modern cloud-native applications rely heavily on service meshes like Istio for traffic management, security, and observability. When combined with OpenTelemetry, you gain powerful distributed tracing capabilities—at least in theory. In practice, teams often encounter a common issue: traces break apart when mesh telemetry meets application telemetry. This article explains why this happens and how to fix it using the CNCF ecosystem.
## What Istio Already Gives You
Istio provides immediate value with minimal application changes. Once sidecars are injected, you automatically receive:
– Request volume between services
– Latency and error rate per workload
– Topology views in Kiali
– Access logs from the proxies
This telemetry is invaluable for platform teams needing to understand service health and performance at a glance.
## The Problem: Fragmented Telemetry
Many teams assume that adding OpenTelemetry will automatically solve all observability challenges. They install the mesh, enable tracing, add application instrumentation, and expect everything to work together seamlessly. What often happens instead is that traces become fragmented.
In a typical OpenTelemetry Demo setup, before Istio is installed, Jaeger shows complete end-to-end traces across demo services. After enabling Istio tracing, however, you see two separate versions of the same service:
– One from the OpenTelemetry SDK inside the application
– One from the Envoy sidecar
Rather than complementary views, you get two unrelated trace trees for the same request. This is the core integration problem: you have telemetry, but it’s not unified.
## Why Traces Split: The Root Cause
The fundamental issue is **propagation**. Applications using the OpenTelemetry SDK typically propagate context using W3C Trace Context headers. When Istio tracing is enabled, Envoy generates its own spans. If Envoy doesn’t extract and continue the same context, it creates a new root span.
This results in:
– An application trace containing business logic, RPCs, and database spans
– A mesh trace containing proxy hops, network latency, and retries
Both are useful independently, but neither provides the complete picture needed for effective debugging.
## The OpenTelemetry Solution
The solution involves two key components:
### 1. The Collector as Integration Point
Instead of treating the Collector as just another exporter target, position it as where the mesh and applications meet. Applications should send OTLP data to the Collector, which should also receive tracing data in formats compatible with Envoy.
### 2. Consistent Propagation
Both application and mesh must agree on propagation headers. While the application emits tracecontext and baggage, the mesh needs to understand these—or an additional compatible format.
## Practical Implementation Steps
### Switch Envoy to the Zipkin Tracer
Envoy’s built-in OTel tracer creates new root spans and ignores incoming traceparent headers. Instead, configure Envoy to use its Zipkin tracer:
“`yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
meshConfig:
extensionProviders:
– name: otel-tracing
opentelemetry:
service: otel-collector.otel-demo.svc.cluster.local
port: 4317
enableTracing: true
defaultConfig:
tracing:
zipkin:
address: otel-collector.otel-demo.svc.cluster.local:9411
“`
This configuration makes Envoy extract B3 trace context from incoming requests, allowing it to continue existing traces rather than creating new ones.
### Add B3 Propagation to Applications
Applications need to propagate headers that Envoy can understand. Add b3multi alongside tracecontext and baggage:
“`yaml
default:
envOverrides:
– name: OTEL_PROPAGATORS
value: “tracecontext,baggage,b3multi”
“`
This ensures applications emit both standard W3C headers and B3 headers that Envoy can use to join existing traces.
## Results
When properly configured, traces unify into a single view containing both application and mesh spans. Instead of seeing fragmented traces with approximately 14-2 spans per service, you’ll see comprehensive traces with 51-75 spans for services like checkout.
This transforms the debugging experience from “we have tracing” to “let’s debug this request” with a complete end-to-end view.
## Final Thoughts: The Collector’s Central Role
Every production system involves software from different projects at various versions. Even when all components are configured correctly, inconsistencies can still cause trace fragmentation. The OpenTelemetry Collector solves this by:
– Receiving telemetry in multiple formats
– Normalizing everything into the OpenTelemetry model
– Exporting unified data to backends
While Istio provides powerful out-of-the-box observability, achieving production-ready tracing often requires targeted adjustments. Having a central component that can normalize format inconsistencies makes the entire observability stack work effectively in practice.
## FAQ
**Q: Why do my traces break when I enable Istio after OpenTelemetry?**
A: Traces break because Envoy’s default OpenTelemetry tracer creates new root spans and ignores incoming traceparent headers, while your application continues using its own trace context. This results in two separate trace trees for the same request.
**Q: What is B3 propagation and why do I need it?**
A: B3 is a propagation format used by Zipkin. By adding B3 propagation alongside tracecontext, your applications can emit headers that Envoy understands, allowing the mesh to continue existing traces rather than creating new ones.
**Q: Is the Zipkin tracer better than the OpenTelemetry tracer for Envoy?**
A: For integration with OpenTelemetry applications, yes. The Zipkin extractor understands B3 headers and continues existing traces, while the OTel tracer ignores incoming context and starts new spans.
**Q: Can I use this approach with other backends besides Jaeger?**
A: Yes. The OpenTelemetry Collector normalizes data and can export to multiple backends. This solution is backend-agnostic.
**Q: Will this affect performance or add significant complexity?**
A: The changes are minimal configuration updates that don’t impact application code. The performance overhead is negligible compared to the value of unified tracing.
## Conclusion
Unifying application and service mesh telemetry requires understanding how propagation works across different components. By configuring Envoy to extract trace context properly and ensuring applications emit compatible headers, you create a seamless observability experience. The OpenTelemetry Collector serves as the central hub that normalizes diverse telemetry formats, making the entire CNCF observability stack work together effectively in production environments.
