Kernel-level eBPF can auto-capture RED metrics and traces on Linux 5.8+ without per-pod proxies. See the architecture, limits, and rollout plan. Read now.
What Kernel-Level Capture Changes
eBPF observability attaches small, verified programs at safe points in the Linux kernel so traffic and runtime signals can be measured without a sidecar or agent in every pod. On Linux 5.8 and newer, those hooks can observe sockets, system calls, and networking paths as the kernel already handles them. From that vantage point you can derive RED metrics—request rate, errors, and duration—and stitch short traces that follow a request as it crosses process and container boundaries.
The practical shift is ownership of instrumentation. Instead of scattering libraries and proxies through every service, you place one kernel-level collector on the host or node and let it auto-capture what the kernel already sees. Application code stays thinner; the common failure mode of “this service never got the sidecar” shrinks. You still need clear identity (which workload owns which sockets), but you no longer need a full userspace proxy mesh solely to export basic service health.
Architecture in Production
A typical design has three layers. Probes (or maps of probes) run in the kernel and record events into ring buffers or maps with fixed size and lifetime. A userspace agent on the same host reads those events, enriches them with container and process metadata, and exports metrics and traces to your existing backend. Control plane pieces decide which programs load, which targets they attach to, and how aggressive sampling or filtering should be under load.
Keep the kernel path narrow: count and time what you must, defer heavy work (string parsing, cardinality explosion, remote export) to userspace. Prefer well-known attach points over ad hoc tracing of every function. Map the export format to what your teams already use for dashboards and alerts so eBPF becomes another source, not a second observability stack. When you do add application spans later, treat kernel-derived RED and short traces as the baseline that fills gaps between services that never instrumented themselves.
Limits You Must Design Around
Kernel programs face hard constraints: instruction budgets, map memory, verifier rejection of unsafe patterns, and the cost of running on hot paths. High cardinality—unbounded labels for paths, headers, or user ids—will blow memory and backend cost even if the probe itself is cheap. Encrypted traffic and pure userspace protocols limit what the kernel can infer without cooperation from the application. Older kernels, restricted environments, or hosts without the right privileges simply cannot run the same attach model.
- Verify kernel version and privilege model on every node class before rollout.
- Cap map sizes, label sets, and sample rates; define drop and overflow behavior explicitly.
- Expect incomplete traces where work never hits observable kernel points; plan hybrid coverage with app instrumentation where it matters most.
- Treat verifier failures and probe load errors as first-class ops signals, not one-time install noise.
A Sensible Rollout Plan
Start with a single cluster or node pool, collect RED for a few well-understood services, and compare against any existing proxy or library metrics until numbers and labels match operational expectations. Expand attach coverage gradually—network first, then selected runtime hooks—while watching agent CPU, map pressure, and export lag. Encode load and unload as controlled changes: versioned program packages, canary nodes, and a clear rollback that detaches probes without restarting applications.
Operational readiness matters as much as the probes. Document who owns the agent, how secrets and export endpoints are configured, and how on-call distinguishes kernel-side drops from backend outages. Once stable, promote the same package through environments with the same policy: Linux 5.8+, no requirement for per-pod proxies for baseline RED and traces, and a written list of what eBPF will never see so product teams know when they still need explicit instrumentation. That is the path from a demo attach to production monitoring you can trust under real load.