Log collection sources

The collection sources the Sondar agent ships with, and what each one is for.

The collection chapter documents eight sources — File, Script, JVM, HTTP Fetch, HTTP Listen, Prometheus remote r/w, Agent Metric/Log, Server Log. This page covers the rest: container, Kubernetes, Docker, Filebeat and Lumberjack readers are all built and shipping, and they are the sources a US enterprise reaches for first.

Container and Kubernetes

The largest gap in the older documentation. Six distinct readers ship for container logs alone.

reader	what it collects
`files/container_stdout`	stdout/stderr of running containers, the usual entry point for Docker and Kubernetes
`files/docker_log`	Docker’s own JSON log files
`files/container_tail`	tails a single container log file
`files/container_tailx`	tails many container log files by pattern
`files/container_dir`	watches a directory of container logs
`files/container_dirx`	as above, with pattern matching
`files/container_fileset`	multi-file container collection with an open-file ceiling
`autodiscover`	discovers containers and attaches collection as they start and stop

autodiscover is the one that makes Kubernetes practical: without it, every pod restart would need its log path re-registered by hand.

See Configuration — file readers below for the keys these accept; the container readers share the same option set.

Files

reader	what it collects
`files/fsnotify`	filesystem-event-driven collection, no polling
`files/aws_s3`	log objects from an S3 bucket

The general file modes — fileset, dir, dirx, tail, tailx — share the same option set, documented under Configuration below.

Agent-to-agent and shipper protocols

reader	what it collects
`beats/filebeat/v1`, `beats/filebeat/v2`	accepts data from Elastic Filebeat
`beats/heartbeat`	Elastic Heartbeat
`beats/packetbeat`	Elastic Packetbeat
`lumberjack`	the Lumberjack protocol, as used by Logstash forwarders

These matter for migration: a prospect already running Filebeat can point it at Sondar without replacing their agent fleet. None of it is documented today, so nobody knows it is possible.

The Filebeat collector family — twelve shipped, none documented

Separate from the beats/filebeat reader (which accepts data from a customer’s existing Filebeat), the integration catalog ships twelve Filebeat-based collectors as installable apps. All twelve are on the ship list — none are hidden — and none appear in any English document.

app	collects
`journald_fbt`	the systemd journal
`winlog_fbt`	Windows Event Log
`syslog_fbt`	syslog over the network
`tcp_fbt`, `udp_fbt`	arbitrary line streams over TCP or UDP
`unix_fbt`	a Unix domain socket
`kafka_fbt`	a Kafka topic
`filestream_fbt`, `log_fbt`	files, the modern and legacy Filebeat inputs
`container_fbt`, `docker_fbt`	container stdout/stderr
`stdin_fbt`	standard input

Two of these answer questions that come up in every enterprise evaluation. “Can you collect Windows Event Logs?” and “can you read the systemd journal?” are near-universal, the answer is yes for both, and nothing a prospect or a sales engineer can read says so. syslog_fbt, tcp_fbt and udp_fbt matter almost as much: network devices and appliances that cannot run an agent are a standard part of a log estate.

Verified rather than assumed: tools/scope/hidden-apps.json lists all twelve in ship (337 apps) and none in hidden (30).

Standards and platforms

reader	what it collects
`opentelemetry`	OTLP ingest
`vsphere/event`	VMware vSphere events
`script`	output of an arbitrary script
`metrics`, `telegraf`	agent and Telegraf metrics
`event`	internal agent event stream

Pulling from a system that already stores the data

The consumer family reads from an existing store rather than tailing a file — 31 readers, listed below.

Every SQL source comes in two modes, which is the distinction the handbook calls incremental and full:

append — incremental. Poll for rows added since the last checkpoint.
snapshot — full. Read the whole table each cycle.

source	readers
MySQL	`sql_consumer/{append,snapshot}/mysql`, plus `binlog_consumer/mysql_binlog_consumer` for change-data-capture
PostgreSQL	`sql_consumer/{append,snapshot}/postgres`
Oracle	`sql_consumer/{append,snapshot}/oracle`
SQL Server	`sql_consumer/{append,snapshot}/mssql`
Dameng	`sql_consumer/{append,snapshot}/dm`
Trino	`sql_consumer/{append,snapshot}/trino`
Druid	`sql_consumer/{append,snapshot}/druid`
MongoDB	`sql_consumer/mongodb_consumer/{append,snapshot}`
Elasticsearch	`es_consumer/{append,snapshot}` — two client implementations, `go-elasticsearch` and `olivere`
HBase	`hbase_consumer/{append,snapshot}`
Redis	`redis_consumer/{goredis,redigo}`
Kafka	`kafka_consumer/{sarama,segmentio}`
Sondar itself	`sondardb_consumer/{append,snapshot}` — read one index to write another
Zabbix	`zabbix_consumer/zabbix_mysql_binlog_consumer`
S3	`files/aws_s3` (in the file family, but the same job)

Which of these matter for logs. kafka_consumer is the common one — a great many organizations already put logs on a topic, and this reads them without changing that pipeline. es_consumer matters for migration: it reads an existing Elasticsearch cluster, so logs can move to Sondar without a re-instrumentation project. files/aws_s3 picks up logs already landed in object storage. sondardb_consumer is how you build a rolled-up index from a raw one.

Two client libraries, for Redis and Kafka. Nothing in the product says which to prefer, and the choice is not obviously reversible once a checkpoint exists. That is a documentation gap worth closing before a customer picks by coin-flip.

Event sources

Separate from log collection, the agent converts events from thirteen sources into Sondar events — Zabbix, Prometheus, cloud events, RESTful, and nine more, several of them worth a sales conversation rather than a footnote.

event source	registered as
Datadog	`datadog_event`
Dynatrace	`dynatrace_event`
New Relic	`newrelic_event`
ELK	`elk_event`
Prometheus	`prometheus_event`
Zabbix	`zabbix_event`
RESTful (generic)	`restful_event`
RESTful change events	`restful_change`
Sondar	`sondar_event`
Alibaba Cloud	`aliyun_event`
Huawei Cloud	`huawei_event`
Cloudwise	`cloudwise_event`
EMCC	`emcc_event`

Datadog, Dynatrace, New Relic and ELK are the interesting four. They mean Sondar can consume events from the monitoring platform a prospect already runs, which is a coexistence story rather than a rip-and-replace one — and it is currently invisible to anyone reading the documentation. Whether they stay is a scope question: aliyun_event, huawei_event, cloudwise_event and emcc_event are China-market integrations in the same family as the Alibaba Cloud log reader, and belong to the same decision.

All thirteen are registered in eventutils/all/all.go.

Out of scope for the US product

reader	note
`aliyun_sls`	Alibaba Cloud Log Service. Compiled into every agent and pulls `aliyun-log-go-sdk` plus `alibaba-cloud-sdk-go` into the build. Recommended for removal — see Q2 in the audit report.

Configuration — file readers

Every file-based reader (fileset, dir, dirx, tail, tailx, and the container_* variants) takes the keys below. Read from the reader source, not from any document: plugins/reader/files/fileset/types.go for the reader’s own keys and plugins/reader/files/inner/ for the shared ones, with defaults taken from the constructors (New() and NewOptions()) rather than from comments.

Selecting files

key	type	default	notes
`path`	string	—	Required. Glob patterns are allowed.
`valid_path_pattern`	list	all	Glob patterns to include.
`ignore_path_pattern`	list	none	Glob patterns to skip.
`max_open_files`	int	`10`	Ceiling on concurrently open files. Validated 1–10000.
`stat_interval`	duration	`10s`	How often the directory is rescanned.

Reading

key	type	default	notes
`read_from`	string	`oldest`	`oldest` or `newest`. `newest` skips everything already in the file — the right choice for a busy log you do not want to backfill.
`encoding`	string	`UTF-8`
`delimiter`	string	`\n`	Line separator.
`head_pattern`	regex	none	Multiline. A line matching this starts a new event; lines that do not are appended to the previous one. This is how a Java stack trace stays one event.
`max_line_len`	bytes	`1 MB`	Validated at a minimum of 100.
`runner_max_line_len`	bytes	`1 MB`	Per-runner ceiling.
`reader_buf_size`	bytes	`4096`
`read_limit`	MB	`-1`	Read rate cap; `-1` is unlimited.
`read_line_limit`	int	`-1`	Lines-per-interval cap; `-1` is unlimited.
`ignore_file_not_exist`	bool	`false`	Do not error when `path` matches nothing yet.
`channel_size`	int	`100`	Internal buffer between reader and pipeline.
`delete_after_read`	bool	`false`	Destructive. Removes the file once consumed. Only for a spool directory you own.

A worked example

Collecting a multiline application log, skipping the backlog, into a named index:

readers:
  - fileset:
      path: /var/log/myapp/*.log
      ignore_path_pattern: ["*.gz", "*.1"]
      read_from: newest
      # An event starts at an ISO-8601 timestamp; continuation lines -- the
      # stack trace -- are folded into the event above.
      head_pattern: '^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}'

transformers:
  - grok field="raw" pattern="%{TIMESTAMP_ISO8601:ts} %{LOGLEVEL:level} %{GREEDYDATA:msg}"

senders:
  - sondar:
      repo: myapp_logs
      sourcetype: myapp
      token: "${sondar_token}"

sondar:, not keta:. The shipped configs still use the old sender name. The agent accepts both — keta is kept as a deprecated alias precisely so those configs keep working — but write new configs against sondar.

Parsing happens here, in the agent. There is no server-side parser: events sent straight to the ingest API arrive with only the built-in fields. The transformers stage is where structure is created.

Configuration — non-file readers

The readers that do not tail files, with their verified config keys.

Script

script runs a command and collects its output. The command is a script file or a shell command; it must emit data on stdout.

readers:
  - script:
      script_cmd: bash            # required: a script file or command
      script_args: /opt/collect.sh   # arguments to the command
      script_timeout: 10s         # per-run timeout (default 10s)
      app: myapp                  # optional app label

Lumberjack (Logstash-forwarder)

lumberjack_listener accepts the Lumberjack protocol, so an existing Logstash forwarder (or Logstash itself) can ship to the agent without change. It listens on :5000 by default, both protocol versions enabled.

readers:
  - lumberjack_listener:
      address: ":5000"            # required; listen address
      enable_v1: true             # Lumberjack v1 frames (default on)
      enable_v2: true             # Lumberjack v2 frames (default on)
      keep_alive: 3s
      timeout: 30s
      encoding: utf-8
      tls_ca: /path/to/ca.pem     # optional server TLS

S3

s3 reads log objects from an S3-compatible bucket, downloading them to a local spool and tailing them. This is the log-archive ingestion path.

readers:
  - s3:
      endpoint: https://s3.example.com   # required
      region: us-east-1                  # required
      bucket: logs                        # required
      access_key_id: AKIA...              # required
      secret_access_key: "..."            # required
      prefix: prod/                       # object key prefix
      pattern: "*.log.gz"                 # filename match
      s3_delimiter: "/"
      # plus the file-reader keys (head_pattern, encoding, max_line_len, …)

OpenTelemetry (OTLP)

otlp_otel accepts OTLP traces, metrics and logs — this is the APM ingest path (see APM). skywalking_otel accepts the SkyWalking protocol. gRPC defaults to 0.0.0.0:4317, HTTP to 0.0.0.0:4318.

readers:
  - otlp_otel:
      receiver:
        protocols:
          grpc:
            endpoint: 0.0.0.0:4317
            # tls: { ca_file: …, cert_file: …, key_file: … }
            # auth: { authenticator: basicauth }   # see extensions below
          http:
            endpoint: 0.0.0.0:4318
      extensions:
        basicauth:
          htpasswd:
            inline: "user:password"
      processors:
        traces:
          - batch:
      timeout: 5s

The Filebeat collector family

The twelve *_fbt readers (filestream_fbt, log_fbt, journald_fbt, winlog_fbt, syslog_fbt, tcp_fbt, udp_fbt, unix_fbt, kafka_fbt, container_fbt, docker_fbt, stdin_fbt) are Filebeat inputs running inside the agent, so they take Filebeat’s own input configuration — the same paths/multiline/processors keys a standalone Filebeat config uses. The integration apps ship ready-made templates for these; the reader names above are the ones the agent registers.

Kubernetes — a worked example

Collecting pod logs on Kubernetes. The agent runs as a DaemonSet — one pod per node — with the node’s runtime and host paths mounted in. Install it with the Helm chart (Agent and machines); the chart takes SONDARD_RUNTIME (containerd, docker, or crio) and mounts /hostfs/… for the host’s logs and runtime socket.

Then a collection task reads container stdout with container_stdout, which tails the container runtime’s log files and emits one event per line:

readers:
  - container_stdout:
      interval: 30s
      codec: containerd           # docker, containerd, or cri
      read_from: newest           # skip the existing backlog
      head_pattern: '^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}'   # multiline: a new
                                                              # event starts at a timestamp

transformers:
  - grok field="raw" pattern="%{TIMESTAMP_ISO8601:ts} %{LOGLEVEL:level} %{GREEDYDATA:msg}"

senders:
  - sondar:
      repo: k8s_pod_logs
      sourcetype: container_stdout
      token: "${sondar_token}"

The autodiscover reader discovers containers as they start and stop and attaches collection automatically, so pod restarts do not need re-registering — the mechanism that makes this work at cluster scale without hand-editing paths. Its config is interfaces (the container runtimes to watch) and probe_timeout.

The same pod-log data, without the agent, can be pulled by the container_fbt/docker_fbt Filebeat collectors or pushed via the otlp_otel reader. The k8s metrics side (cluster, node, pod, event) is the infrastructure_monitoring app’s job.