Splunk HEC ingest

If your shippers already send to a Splunk HTTP Event Collector (HEC) — Fluent Bit’s splunk output, Vector’s splunk_hec_logs sink, the OpenTelemetry Collector’s splunkhec exporter, or your own code — they can send to Sondar by changing two settings: the URL and the token. Nothing else in the shipper changes.

Universal forwarders are the exception. A universal forwarder’s HTTP output ([httpout]) carries Splunk’s own forwarder protocol, not the HEC event API this endpoint implements. Point forwarders at one of the shippers above, or send their files with the Sondar agent instead.

This is compatibility for ingest only. The query language is SonQL, which is Splunk-inspired rather than Splunk-compatible; see Splunk → SonQL for the differences.

Endpoints

On the server’s normal HTTP port (9200 by default):

method	path	takes
POST	/services/collector	JSON events
POST	/services/collector/event	JSON events (the same thing)
POST	/services/collector/raw	plain text, one event per line
GET	/services/collector/health	nothing — answers {"text":"HEC is healthy","code":17}
GET	/services/collector/health/1.0	the same; a token you present is checked, and a bad one answers 400 {"text":"Invalid token","code":4} — this is the check Vector’s splunk_hec sinks run at startup

The token

Use a Sondar collect token — the same kind agents and OTLP exporters use (see Collection tokens) — in Splunk’s header:

Authorization: Splunk collect_<id>.<secret>

Bearer works too. Use the whole value, copied when the token was created; it is shown only once. A token created before October 2026 has no secret, and its value is just collect_<id>. A token in the query string (?token=) is refused: it would end up in proxy and access logs.

Sending events

The body is Splunk’s: one or more JSON objects, one after another (a JSON array of them is accepted too).

curl http://sondar.example.com:9200/services/collector \
  -H "Authorization: Splunk collect_<id>.<secret>" \
  -d '{"time": 1790150000, "host": "web-01", "source": "/var/log/app.log",
       "sourcetype": "_json", "event": {"message": "login ok", "user": "ada"},
       "fields": {"env": "prod"}}
      {"event": "a plain text line"}'

A successful batch answers {"text":"Success","code":0}.

HEC field	becomes	notes
event	_raw	required; a string is stored as is, an object as JSON text
time	_time	epoch seconds, fractional or as a string. Absent: the sourcetype’s timestamp rule, then the arrival time
index	the index	default default; must exist
sourcetype	sourcetype	default default; a new name is created on first use (below). Splunk’s _json is Sondar’s json
host	host	default: the address the request came from
source	origin	default hec
fields	extra fields	searchable like any other field

For /services/collector/raw, put index, sourcetype, host and source in the query string; every non-empty line is one event.

A sourcetype you have not created yet

As in Splunk, events under a sourcetype Sondar has never seen are indexed rather than refused: HEC creates the sourcetype on first sight, copying the built-in json sourcetype’s line and timestamp rules, and describes it as created by HEC so you can find it on the Sourcetype page and give it its own rules. Searches for sourcetype=<what you sent> find the events at once.

Two limits keep a typo from filling that page: HEC creates at most 50 new sourcetypes an hour (sondar.hec.auto_create_sourcetype.max_per_hour), and a name that breaks the naming rule below is refused. To require every sourcetype to be created by hand, set sondar.hec.auto_create_sourcetype: false; HEC then refuses an unknown one by name, with code 6.

Where Sondar is stricter than Splunk

• A batch is all or nothing. Every event is checked before any is written. Splunk writes the events before a bad one, so a shipper that retries the batch sends those events twice; a batch Sondar refuses as invalid wrote nothing, and a retry cannot duplicate it.
• Splunk’s sourcetype names work as they are. A sourcetype name may hold letters, digits and _ . : - / { }, so aws:cloudtrail, linux-secure, WinEventLog:Security and XmlWinEventLog:Microsoft-Windows-Sysmon/Operational can be created (on the Sourcetype page, or through the API) and your shippers keep sending them unchanged. Searches name them without quotes, as in Splunk: sourcetype=aws:cloudtrail, sourcetype=aws:*, sourcetype IN (aws:cloudtrail, linux-secure). Measured on r1226, 30 Sep 2026, on both engines: HEC accepted events under all three names, and each unquoted search returned the same count as its quoted form. A name with a space, a quote or a | is still refused. (Until that image, a name could not hold : or -, and HEC refused those events.)

Errors

Refusals use Splunk’s codes and texts, so a shipper’s logs read the same:

HTTP	code	text
401	2	Token is required
403	4	Invalid token
400	5	No data
400	6	Invalid data format — or a refused sourcetype: Invalid sourcetype '<name>' …, or Unknown sourcetype '<name>' … when creation is off or past its hourly limit
400	7	Incorrect index
400	12	Event field is required
400	13	Event field cannot be blank
400	15	Error in handling indexed fields
400	16	Query string authorization is not enabled
503	9	Server is busy

Per-event errors also carry invalid-event-number, counting from 0.

503 “Server is busy”

Sondar answers 503 with code 9, Splunk’s answer when its queues are full, in three cases. Each carries Retry-After: 1; make sure your shipper retries 503.

• The ingest parser queue is full, or the server is stopping: the text is just Server is busy. The batch wrote nothing, so a retry loses nothing and duplicates nothing.
• A write failed partway — a full disk, or part of an index unavailable while a node is lost. The text says why and how much was written, for example Server is busy: the index is temporarily unavailable (1673 of 5000 events were written). The retry sends the whole batch, so the lines already written are stored twice: see When a write is confirmed.

The queue holds 1,000 requests: eight times the 128 simultaneous senders that overflowed the old queue. A 503 therefore means more requests are waiting than the server can work through, rather than a burst of senders posting at the same moment. Two places show it on the server side:

• the log, at most once a minute: ingest refused N request(s) … the parser queue is full, with the count since the previous line;
• the series sondar_thread_pool_rejected_total with pool=sondar_event_parser (and pool=write for the shard write), in metrics_sondar, beside sondar_thread_pool_queue and sondar_thread_pool_active. Alert on its rate.

If it happens, the server needs more CPU or more nodes. A larger queue only delays the refusal. The size is the node setting thread_pool.sondar_event_parser.queue_size.

Earlier images held 50 requests in this queue. Many senders posting at the same moment were then refused well under capacity, and the server logged nothing: at 120,000 lines/s, under half of a 16 vCPU server’s ceiling, 128 senders got 19,864 refusals in three and a half minutes.