Why Run an OpenTelemetry Collector

Direct export couples every application to your backend. Change vendors, rotate an API key, or add span redaction, and you are redeploying 40 services. With a Collector in the middle, apps export plain OTLP to a local endpoint and everything else becomes pipeline config: receivers take data in, processors transform it, exporters fan it out. The processors are where the real value is. The batch processor cuts export overhead, memory_limiter stops the Collector from OOMing under a span flood, attributes and redaction processors strip PII before it leaves your cluster, and tail sampling can only happen here because no single app sees the whole trace. The Collector also buffers and retries when the backend is down, so a backend outage does not mean dropped telemetry or back-pressure into your apps. In Kubernetes the standard layout is two tiers. A DaemonSet agent on every node gives apps a node-local endpoint, adds k8s metadata like pod and namespace, and does cheap work like batching. A central gateway Deployment behind a Service handles the expensive, stateful work: tail sampling, filtering, authentication to external backends, and being the single egress point your firewall rules allow. Small clusters can skip the gateway and run just the DaemonSet pointed straight at the backend. You add the gateway when you need tail sampling or want one place to control egress and credentials.