<div id="readability-page-1" class="page"><div><img src="https://devstarsj.github.io/assets/https://images.unsplash.com/photo-1551288049-bebda4e38f71?w=1200&amp;q=80"><h2 id="the-observability-fragmentation-problem-past-tense">The Observability Fragmentation Problem (Past Tense)</h2>Three years ago, the observability landscape was a mess. You had Jaeger for tracing, Prometheus for metrics, and Elasticsearch for logs — each with its own SDK, its own instrumentation approach, and its own data model. Switching vendors meant rewriting instrumentation. Running multiple backends meant maintaining multiple agents.OpenTelemetry solved this. And by 2026, it has solved it so thoroughly that the question is no longer “should I use OTel?” but “how do I use it well?”<img src="https://images.unsplash.com/photo-1460925895917-afdab827c52f?w=1000&amp;q=80" alt="Distributed system monitoring dashboard"> Photo by Carlos Muza on Unsplash<hr><h2 id="what-opentelemetry-actually-is">What OpenTelemetry Actually Is</h2>OTel is three things:<ol><li>A specification — defining what traces, metrics, and logs look like</li><li>A set of SDKs — one per language, all following the same API</li><li>The Collector — a vendor-agnostic pipeline agent for receiving, processing, and exporting telemetry</li></ol>The key insight is separation of instrumentation from export. You instrument your code once using OTel’s API. Where that data goes — Datadog, Honeycomb, Grafana Cloud, Jaeger, your own Prometheus — is a deployment configuration, not a code change.<hr><h2 id="the-three-signals-status-in-2026">The Three Signals: Status in 2026</h2><table><thead><tr><th>Signal</th><th>API/SDK Status</th><th>Notes</th></tr></thead><tbody><tr><td>Traces</td><td>Stable</td><td>Production-ready since 2021</td></tr><tr><td>Metrics</td><td>Stable</td><td>Graduated stable in 2023</td></tr><tr><td>Logs</td><td>Stable</td><td>Finally GA in 2024</td></tr><tr><td>Profiles</td><td>Beta</td><td>Continuous profiling, expected GA Q4 2026</td></tr><tr><td>Events</td><td>Experimental</td><td>Structured log events with semantic conventions</td></tr></tbody></table>All four core signals are stable. Profiles (continuous profiling data) is the exciting frontier — it will let you correlate performance profiles directly with traces, answering “why was this trace slow?” with actual flame graphs.<hr><h2 id="instrumenting-a-nodejs-service">Instrumenting a Node.js Service</h2>Here’s the 2026 way to instrument a Node.js Express app with zero-code instrumentation:<div><pre><code>// otel.js - load BEFORE anything else
import { NodeSDK } from '@opentelemetry/sdk-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { OTLPMetricExporter } from '@opentelemetry/exporter-metrics-otlp-http';
import { PeriodicExportingMetricReader } from '@opentelemetry/sdk-metrics';

const sdk = new NodeSDK({
 serviceName: 'user-service',
 traceExporter: new OTLPTraceExporter({
 url: 'http://otel-collector:4318/v1/traces',
 }),
 metricReader: new PeriodicExportingMetricReader({
 exporter: new OTLPMetricExporter({
 url: 'http://otel-collector:4318/v1/metrics',
 }),
 exportIntervalMillis: 30000,
 }),
 instrumentations: [
 getNodeAutoInstrumentations({
 '@opentelemetry/instrumentation-http': { enabled: true },
 '@opentelemetry/instrumentation-express': { enabled: true },
 '@opentelemetry/instrumentation-pg': { enabled: true },
 '@opentelemetry/instrumentation-redis': { enabled: true },
 }),
 ],
});

sdk.start();
</code></pre></div><div><pre><code>// index.js
import './otel.js'; // Must be first
import express from 'express';
// ... your normal app code, no OTel changes needed
</code></pre></div>Auto-instrumentation handles HTTP spans, database queries, and Redis calls automatically. No manual <code>startSpan()</code> calls needed for common operations.<hr><h2 id="adding-custom-instrumentation">Adding Custom Instrumentation</h2>Auto-instrumentation gets you 80% of the way. For business-specific operations, add manual spans:<div><pre><code>import { trace, metrics } from '@opentelemetry/api';

const tracer = trace.getTracer('user-service');
const meter = metrics.getMeter('user-service');

// Custom metric
const orderCounter = meter.createCounter('orders.created', {
 description: 'Number of orders created',
});

const orderValueHistogram = meter.createHistogram('orders.value', {
 description: 'Order value in USD',
 unit: 'USD',
});

async function createOrder(userId: string, items: OrderItem[]) {
 return tracer.startActiveSpan('order.create', async (span) =&gt; {
 span.setAttributes({
 'user.id': userId,
 'order.item_count': items.length,
 });

 try {
 const order = await db.orders.create({ userId, items });
 const value = items.reduce((sum, i) =&gt; sum + i.price, 0);
 
 // Record metrics
 orderCounter.add(1, { 'order.status': 'success' });
 orderValueHistogram.record(value, { 'user.tier': user.tier });
 
 span.setStatus({ code: SpanStatusCode.OK });
 return order;
 } catch (error) {
 span.recordException(error);
 span.setStatus({ code: SpanStatusCode.ERROR });
 orderCounter.add(1, { 'order.status': 'error' });
 throw error;
 } finally {
 span.end();
 }
 });
}
</code></pre></div><hr><h2 id="the-otel-collector-your-observability-pipeline">The OTel Collector: Your Observability Pipeline</h2>The Collector is where the real power lives. It’s a standalone service that receives telemetry, applies transformations, and routes to multiple backends.<div><pre><code># collector-config.yaml
receivers:
 otlp:
 protocols:
 http:
 endpoint: 0.0.0.0:4318
 grpc:
 endpoint: 0.0.0.0:4317
 
 # Also scrape Prometheus endpoints
 prometheus:
 config:
 scrape_configs:
 - job_name: 'kubernetes-pods'
 kubernetes_sd_configs:
 - role: pod

processors:
 # Drop high-cardinality spans from health checks
 filter:
 traces:
 span:
 - 'attributes["http.route"] == "/health"'
 
 # Add resource attributes to everything
 resource:
 attributes:
 - key: deployment.environment
 value: production
 action: upsert
 
 # Smart tail-based sampling
 tail_sampling:
 decision_wait: 10s
 policies:
 - name: errors-policy
 type: status_code
 status_code: { status_codes: [ERROR] }
 - name: slow-traces-policy
 type: latency
 latency: { threshold_ms: 1000 }
 - name: sample-10pct
 type: probabilistic
 probabilistic: { sampling_percentage: 10 }
 
 batch:
 timeout: 5s
 send_batch_size: 1000

exporters:
 # Primary: Grafana Cloud
 otlp/grafana:
 endpoint: https://otlp-gateway-prod-us-central-0.grafana.net:443
 headers:
 authorization: "Basic ${GRAFANA_TOKEN}"
 
 # Secondary: Keep traces in-house with Tempo
 otlp/tempo:
 endpoint: http://tempo:4317
 
 # Metrics to Prometheus
 prometheus:
 endpoint: 0.0.0.0:9090

service:
 pipelines:
 traces:
 receivers: [otlp]
 processors: [filter, resource, tail_sampling, batch]
 exporters: [otlp/grafana, otlp/tempo]
 metrics:
 receivers: [otlp, prometheus]
 processors: [resource, batch]
 exporters: [otlp/grafana, prometheus]
 logs:
 receivers: [otlp]
 processors: [resource, batch]
 exporters: [otlp/grafana]
</code></pre></div>This config: filters health check noise, adds environment context, applies intelligent sampling (keeping all errors and slow traces, 10% sampling otherwise), and fans out to multiple backends.<hr>OTel’s real value beyond the SDK is semantic conventions — standardized attribute names for common operations. Instead of everyone calling the same thing <code>http.url</code>, <code>request_url</code>, <code>url</code>, and <code>req.url</code>, there’s one standard:<div><pre><code>http.request.method: GET
url.full: https://api.example.com/users/123
http.response.status_code: 200
server.address: api.example.com
</code></pre></div>The <a href="https://opentelemetry.io/docs/specs/semconv/">OTel semantic conventions spec</a> now covers HTTP, databases, messaging, RPC, cloud providers, Kubernetes, and more. When your spans follow conventions, your observability vendor can build dashboards and alerts without you doing any configuration.<hr><h2 id="otel-and-ai-observability">OTel and AI Observability</h2>A fast-emerging area in 2026: observability for LLM applications. The OTel working group has drafted semantic conventions for GenAI:<div><pre><code>span.setAttributes({
 'gen_ai.system': 'openai',
 'gen_ai.request.model': 'gpt-4o',
 'gen_ai.request.max_tokens': 1000,
 'gen_ai.response.model': 'gpt-4o-2024-11-20',
 'gen_ai.usage.input_tokens': 256,
 'gen_ai.usage.output_tokens': 512,
 'gen_ai.usage.total_tokens': 768,
});
</code></pre></div>Tools like LangFuse, Arize Phoenix, and Traceloop build on top of OTel to provide LLM-specific observability — prompt tracing, token cost tracking, and evaluation pipelines — all feeding into your existing OTel infrastructure.<hr><h2 id="getting-started-in-one-hour">Getting Started in One Hour</h2><ol><li>Add SDK to your service (pick your language from <a href="https://opentelemetry.io/">opentelemetry.io</a>)</li><li>Run the Collector locally with Docker: <code>docker run -p 4318:4318 otel/opentelemetry-collector-contrib</code></li><li>Point it at Jaeger for visualization (also runs in Docker)</li><li>Watch your first trace appear — it’ll take your breath away the first time</li></ol>The ecosystem has matured to the point where you don’t need to be an observability expert to get started. Start simple, add signals as you understand your needs.<hr>If you’re still running separate instrumentation for each backend, it’s time to consolidate. OTel isn’t just a nice-to-have in 2026 — it’s the foundation every modern observability stack is built on. <div>이 글이 도움이 되셨다면 공감 및 광고 클릭을 부탁드립니다 :) </div></div></div>

Dev Note

The Observability Fragmentation Problem (Past Tense)
Three years ago, the observability landscape was a mess. You had Jaeger for tracing, Prometheus for metrics, and Elasticsearch for logs — each with its own SDK, its own instrumentation approach, and its own data model. Switching vendors meant rewriting instrumentation. Running multiple backends meant maintaining multiple agents.
OpenTelemetry solved this. And by 2026, it has solved it so thoroughly that the question is no longer “should I use OTel?” but “how do I use it well?”
Photo by Carlos Muza on Unsplash
What OpenTelemetry Actually Is
OTel is three things:
A specification — defining what traces, metrics, and logs look like
A set of SDKs — one per language, all following the same API
The Collector — a vendor-agnostic pipeline agent for receiving, processing, and exporting telemetry
The key insight is separation of instrumentation from export. You instrument your code once using OTel’s API. Where that data goes — Datadog, Honeycomb, Grafana Cloud, Jaeger, your own Prometheus — is a deployment configuration, not a code change.
The Three Signals: Status in 2026
Signal
      API/SDK Status
      Notes
    
Traces
      Stable
      Production-ready since 2021
    
Metrics
      Stable
      Graduated stable in 2023
    
Logs
      Stable
      Finally GA in 2024
    
Profiles
      Beta
      Continuous profiling, expected GA Q4 2026
    
Events
      Experimental
      Structured log events with semantic conventions
    
All four core signals are stable. Profiles (continuous profiling data) is the exciting frontier — it will let you correlate performance profiles directly with traces, answering “why was this trace slow?” with actual flame graphs.
Instrumenting a Node.js Service
Here’s the 2026 way to instrument a Node.js Express app with zero-code instrumentation:
// otel.js - load BEFORE anything else
import { NodeSDK } from '@opentelemetry/sdk-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { OTLPMetricExporter } from '@opentelemetry/exporter-metrics-otlp-http';
import { PeriodicExportingMetricReader } from '@opentelemetry/sdk-metrics';
const sdk = new NodeSDK({
  serviceName: 'user-service',
  traceExporter: new OTLPTraceExporter({
    url: 'http://otel-collector:4318/v1/traces',
  }),
  metricReader: new PeriodicExportingMetricReader({
    exporter: new OTLPMetricExporter({
      url: 'http://otel-collector:4318/v1/metrics',
    }),
    exportIntervalMillis: 30000,
  }),
  instrumentations: [
    getNodeAutoInstrumentations({
      '@opentelemetry/instrumentation-http': { enabled: true },
      '@opentelemetry/instrumentation-express': { enabled: true },
      '@opentelemetry/instrumentation-pg': { enabled: true },
      '@opentelemetry/instrumentation-redis': { enabled: true },
    }),
  ],
});
sdk.start();
// index.js
import './otel.js';  // Must be first
import express from 'express';
// ... your normal app code, no OTel changes needed
Auto-instrumentation handles HTTP spans, database queries, and Redis calls automatically. No manual startSpan() calls needed for common operations.
Adding Custom Instrumentation
Auto-instrumentation gets you 80% of the way. For business-specific operations, add manual spans:
import { trace, metrics } from '@opentelemetry/api';
const tracer = trace.getTracer('user-service');
const meter = metrics.getMeter('user-service');
// Custom metric
const orderCounter = meter.createCounter('orders.created', {
  description: 'Number of orders created',
});
const orderValueHistogram = meter.createHistogram('orders.value', {
  description: 'Order value in USD',
  unit: 'USD',
});
async function createOrder(userId: string, items: OrderItem[]) {
  return tracer.startActiveSpan('order.create', async (span) => {
    span.setAttributes({
      'user.id': userId,
      'order.item_count': items.length,
    });
    try {
      const order = await db.orders.create({ userId, items });
      const value = items.reduce((sum, i) => sum + i.price, 0);
      
      // Record metrics
      orderCounter.add(1, { 'order.status': 'success' });
      orderValueHistogram.record(value, { 'user.tier': user.tier });
      
      span.setStatus({ code: SpanStatusCode.OK });
      return order;
    } catch (error) {
      span.recordException(error);
      span.setStatus({ code: SpanStatusCode.ERROR });
      orderCounter.add(1, { 'order.status': 'error' });
      throw error;
    } finally {
      span.end();
    }
  });
}
The OTel Collector: Your Observability Pipeline
The Collector is where the real power lives. It’s a standalone service that receives telemetry, applies transformations, and routes to multiple backends.
# collector-config.yaml
receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318
      grpc:
        endpoint: 0.0.0.0:4317
  
  # Also scrape Prometheus endpoints
  prometheus:
    config:
      scrape_configs:
        - job_name: 'kubernetes-pods'
          kubernetes_sd_configs:
            - role: pod
processors:
  # Drop high-cardinality spans from health checks
  filter:
    traces:
      span:
        - 'attributes["http.route"] == "/health"'
  
  # Add resource attributes to everything
  resource:
    attributes:
      - key: deployment.environment
        value: production
        action: upsert
  
  # Smart tail-based sampling
  tail_sampling:
    decision_wait: 10s
    policies:
      - name: errors-policy
        type: status_code
        status_code: { status_codes: [ERROR] }
      - name: slow-traces-policy
        type: latency
        latency: { threshold_ms: 1000 }
      - name: sample-10pct
        type: probabilistic
        probabilistic: { sampling_percentage: 10 }
  
  batch:
    timeout: 5s
    send_batch_size: 1000
exporters:
  # Primary: Grafana Cloud
  otlp/grafana:
    endpoint: https://otlp-gateway-prod-us-central-0.grafana.net:443
    headers:
      authorization: "Basic ${GRAFANA_TOKEN}"
  
  # Secondary: Keep traces in-house with Tempo
  otlp/tempo:
    endpoint: http://tempo:4317
  
  # Metrics to Prometheus
  prometheus:
    endpoint: 0.0.0.0:9090
service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [filter, resource, tail_sampling, batch]
      exporters: [otlp/grafana, otlp/tempo]
    metrics:
      receivers: [otlp, prometheus]
      processors: [resource, batch]
      exporters: [otlp/grafana, prometheus]
    logs:
      receivers: [otlp]
      processors: [resource, batch]
      exporters: [otlp/grafana]
This config: filters health check noise, adds environment context, applies intelligent sampling (keeping all errors and slow traces, 10% sampling otherwise), and fans out to multiple backends.
Semantic Conventions: The Hidden Secret
OTel’s real value beyond the SDK is semantic conventions — standardized attribute names for common operations. Instead of everyone calling the same thing http.url, request_url, url, and req.url, there’s one standard:
http.request.method: GET
url.full: https://api.example.com/users/123
http.response.status_code: 200
server.address: api.example.com
The OTel semantic conventions spec now covers HTTP, databases, messaging, RPC, cloud providers, Kubernetes, and more. When your spans follow conventions, your observability vendor can build dashboards and alerts without you doing any configuration.
OTel and AI Observability
A fast-emerging area in 2026: observability for LLM applications. The OTel working group has drafted semantic conventions for GenAI:
span.setAttributes({
  'gen_ai.system': 'openai',
  'gen_ai.request.model': 'gpt-4o',
  'gen_ai.request.max_tokens': 1000,
  'gen_ai.response.model': 'gpt-4o-2024-11-20',
  'gen_ai.usage.input_tokens': 256,
  'gen_ai.usage.output_tokens': 512,
  'gen_ai.usage.total_tokens': 768,
});
Tools like LangFuse, Arize Phoenix, and Traceloop build on top of OTel to provide LLM-specific observability — prompt tracing, token cost tracking, and evaluation pipelines — all feeding into your existing OTel infrastructure.
Getting Started in One Hour
Add SDK to your service (pick your language from opentelemetry.io)
Run the Collector locally with Docker: docker run -p 4318:4318 otel/opentelemetry-collector-contrib
Point it at Jaeger for visualization (also runs in Docker)
Watch your first trace appear — it’ll take your breath away the first time
The ecosystem has matured to the point where you don’t need to be an observability expert to get started. Start simple, add signals as you understand your needs.
If you’re still running separate instrumentation for each backend, it’s time to consolidate. OTel isn’t just a nice-to-have in 2026 — it’s the foundation every modern observability stack is built on.

OpenTelemetry in 2026: The Observability Standard That Finally Stuck

댓글