Application performance monitoring

Application performance monitoring

Tracing application requests end to end — from front-end devices through service calls to databases and message queues — and turning the spans into service maps, latency distributions and error analysis. In the product menu this is Performance Monitoring (the APM module).

---

How APM data gets in

The shipping ingest path is OTLP: applications export OpenTelemetry traces (and derived metrics) to the agent or the platform on port 4317, and the APM module reads them back. sondar.otlp.enabled defaults to on.

• Collect-token auth is required by default (sondar.otlp.auth.required), so an exporter that reaches the OTLP port without a valid token is refused. The dead-port control separates auth from connectivity: a token problem and an unreachable port give different signals.
• Span data is stored at the configured sampling rate while keeping each sampled trace complete; performance summaries are collected in full, so metric pre-aggregation is unaffected by sampling.

Auto-injection is not the shipping path. The claim that the agent auto-injects into every application (JVM instrumentation by default) is not what ships: auto-injection is host-only and off by default (see the trace config panels in agent-machines.md), and the LD_PRELOAD auto-injection half was removed from the build. Application telemetry arrives over OTLP, from an exporter the application (or its SDK) already runs. Do not promise magic zero-code instrumentation on a fresh install.

---

The surfaces

Performance Monitoring → Service — the service list: service name, type, request rate (RPM), hostname, kind, scope, error rate, Apdex, average and P50–P99 response times, max, instances, request and error counts. The topology view renders the discovered service map (the “View topology” entry from a service); nodes are services, edges are calls. A business system groups related services.

Trace — search by trace ID, or browse spans: time, service, operation, response time, status code, span type, scope name/version, attributes, error count, call times. A trace opens as the call tree — each span’s flow and execution time down the chain, the root-cause view.

Config — the APM settings:

• Global config — latency bucket partitioning (affects response-time quantiles; finer buckets give more accurate percentiles, in milliseconds) and trace sampling.
• Business system and service config — which business systems exist, service display names, and per-service trace sampling.
• Trace sampling — store spans at a rate (0.0–1.0, default 1.0) while keeping sampled traces complete.

DB / MQ — the database and message-queue call analysis (DB type, call times, latency), the “which query is slow” surface.

Exception — collected error spans and their distribution.

Profiling (apm/trace/profiling) — the surface for eBPF-based profiling of running services.

Profiling needs a collector that is not included

The page, the permission (apm_profiling), the tracing_profiles index and the agent-side configuration all exist, and profiling is off by default. But the eBPF collector the configuration refers to is not part of this distribution — no such binary ships in the image, and the agent has no eBPF support of its own. Enabling the toggle writes configuration for a collector you would have to supply and run yourself. Treat profiling as unavailable unless you have separately arranged that collector.

Business System — manage the grouping of services.

External Service — calls to services outside the traced set.

---

The key metrics

• Apdex — the Apdex Alliance index of user satisfaction, 0–1: each operation is satisfactory, tolerable (response between the threshold and ~4× it), or unsatisfactory; (1×satisfactory + 0.5×tolerable) ÷ total. The threshold is what you configure as “tolerable”.
• Error rate — failed requests as a percentage of total.
• Percentiles — P50–P99 response times per service and per operation.