Skip to content

Host and infrastructure monitoring

Collecting host-level metrics — CPU, memory, disk, IO, load, network, process — plus host metadata and Kubernetes cluster/node metrics, and viewing them in the Infrastructure app. This is the infrastructure_monitoring app, and it is installed and collecting within minutes of an agent connecting.


The app installs seven collection templates. Together they write to three indexes:

repo contents
infrastructure_monitoring_metrics host, container and Kubernetes metrics (30s)
infrastructure_monitoring_meta host and Kubernetes metadata (1h)
infrastructure_monitoring_events Kubernetes events

All three have a 7-day retention. After installation the agent collects these by default — you do not normally need to send collection tasks yourself.

Collected every 30 seconds by the telegraf input family. The reader names carry the _ntel suffix, which selects the “new telegraf” implementation of the input:

readers:
- cpu_ntel:
interval: 30s
percpu: false
totalcpu: true
collect_cpu_time: false
report_active: false
core_tags: false
- disk_ntel:
interval: 30s
- diskio_ntel:
interval: 30s
- mem_ntel:
interval: 30s
- net_ntel:
interval: 30s
- system_ntel:
interval: 30s
- netstat_ntel:
interval: 30s
#- procstat:
# interval: 30s

Each is a telegraf input (the cpu, disk, diskio, mem, net, system and netstat plugins), registered under the <name>_ntel key. procstat exists but is commented out in the shipped config — process collection is available but off by default.

Reading the metric names. The app’s metric catalog names every metric with a localised key (host_cpu_idle_pct, host_disk_used_pct, …). In the English UI these render as the technical key, not a friendly label — the English dictionary maps each metric-name key to the key itself, while the Chinese UI gets the friendly name. So an English Metric Explorer shows host_cpu_idle_pct where the same metric in Chinese shows its translated name. The labels are a product decision to finish; see the audit record (D4).

Collected once an hour and used to answer asset questions — CPU model and clock speed, network cards, disks, memory. The mock_ticker reader fires once per interval and the metadata transformers attach the facts:

readers:
- mock_ticker:
interval: 1h
count: 1
transformers:
- add_metadata_agent
- add_metadata_host
- add_metadata_net to_string=true
- add_metadata_disk to_string=true
- add_metadata_memory
- add_metadata_cpu
- add_metadata_agent_tags

container reads container CPU/memory/network via the container runtime — Docker, containerd or CRI-O — each with its own socket path and connection settings. Docker’s defaults come from the environment (socket_path: ENV), containerd from /var/run/containerd/containerd.sock, CRI-O from /var/run/crio/crio.sock.

Three templates cover Kubernetes:

  • kubernetes_cluster — resource counts and state across the cluster via the API metric interface. A first reader (30s) collects the high-value resources — the shipped config excludes role, endpoint, service, ingress, clusterrole, serviceaccount, clusterrolebinding and rolebinding — and a second (10m) collects exactly those lower-value ones, so the resource list is complete without paying the poll cost every cycle. Authenticates with the in-cluster service-account token by default.
  • kubernetes_ntel — node and pod metrics via each node’s kubelet (https://127.0.0.1:10250). label_include / label_exclude control which pod labels become tags (empty = all, ["*"] = none).
  • kubernetes_event — cluster events via the API; throttle_window (default 1h) bounds how far behind now an event may be.
  • kubernetes_info (in the k8s meta template) — cluster object metadata, again via the in-cluster token.

The k8s metrics land in infrastructure_monitoring_metrics, meta in infrastructure_monitoring_meta, events in infrastructure_monitoring_events.


Infrastructure (the app, installed from the catalog) opens into three pages:

page what it shows
Host the host list and per-host views — CPU, memory, disk, network
Containers container resource usage
Kubernetes Analysis cluster, node, pod and workload views

The same metric data is queryable from the Metric app (Metric Explorer), so a host chart can be built anywhere a metric query can be written.

The app’s frontend strings are English (an en_US dictionary ships alongside the zh_CN one), and the app title resolves to Infrastructure through the app-i18n table.


  1. Host collection is on by default after install. The global settings in Collection → Agent toggle each family (host metrics, host metadata, containers, Kubernetes) on and off. You only hand-write these collection tasks when you want a subset — “host CPU only” — or when you want to add labels or conversions to the stream.
  2. The _ntel suffix selects the telegraf implementation. The same telegraf inputs also exist under the older _tel (and beat-family) kinds. The shipped configs use _ntel; there is no reason to prefer another kind, and mixing kinds for the same input in one task is a misconfiguration.
  3. Process collection is optional. procstat is present but commented out in the shipped host config. Enable it deliberately — per-process polling is the most expensive of the host inputs at scale.
  4. Metadata is an hour behind, by design. Host metadata updates on the 1h cycle. A host you just installed may take up to an hour to appear fully in asset views; its metrics appear within 30s.
  5. kubernetes_ntel needs kubelet reachability. If node metrics are empty, check the agent can reach https://127.0.0.1:10250 from the node and that the service-account token is valid — the two failures that look identical in the UI.