Interactive search

Searching logs — the query bar, the two engines, the modes, what you can do with a result, and Live Tail.

The query bar

A search is a set of predicates, then a pipeline of commands separated by |:

repo="app_logs" ERROR service="checkout"
| stats count() as errors by host
| sort -errors

A log store is an index to you and a repo internally. index= is accepted as an alias in search predicates, so index="app_logs" and repo="app_logs" are the same thing. The alias is predicate-only — the field is still called repo, so | stats count by index does not work and | stats count by repo does. That asymmetry is deliberate: index is also a legitimate customer field name.

The language is SonQL. It is Splunk-inspired and explicitly not Splunk-compatible: a query that works there will often work here, and when it does not, the difference is not a bug.

Two engines, and why you should name one

Sondar has two search engines and they are not interchangeable.

	`search` (search1)	`search2`
how it runs	parses to pivot operators, picks a datasource per operator	one batch/columnar datasource over the matrix engine
search-time field extraction	yes, every rule type	`regex` and `json` only — see below
`show`, `dbquery`, `mstats`	yes	no
sorting, pattern recognition, metric rules	no	yes
trading-day-only filtering	no	yes

A query that names neither runs on whichever engine the global Default Search Engine setting names. A fresh install persists search2 at first boot (InitializationService.setGlobalSearch()), so that is what unqualified queries use until an administrator changes the form. The code-level fallback for a blank setting is search1 (SearchSettings.acquireDefaultSetting(), SplSearchVersion.fromString("")) — which is where the older “the default is search1” wording came from, but nothing on a fresh install leaves it blank. Measured 22 Aug 2026: GET /api/v1/setting/global returns {"searchSettings":{"splSearchVersion":"search2"}} on an untouched install. But an administrator changing that dropdown changes what unqualified queries do, including saved ones and alert conditions. chart, mvcombine and mvexpand work on both engines since 14 Aug 2026 — mvexpand accepts both mvexpand <field> limit=<n> and mvexpand limit=<n> <field> on either engine, and transaction works on both, with search2 carrying the joined member fields as multivalue columns since 15 Aug 2026. mvcombine groups by the searched repo’s real columns on search2, so events differing only in the per-event identity fields (_time, _raw) merge into one row.

Name the engine in anything you save. search2 repo="…" or search repo="…". It costs seven characters and removes a class of surprise.

The search-time extraction trap

A field extraction is either accelerated (materialised into a column at write time) or search-time (applied to _raw as the query runs). search2 reads columns, so a search-time field is not simply there for it the way it is for search1 — and it used to return null for one silently, so a stats count by <search-time field> gave a number covering only the accelerated rows. It no longer does either.

For regex and json rules, search2 now applies them for you. When a query references such a field — anywhere: a stats … by key, a | where filter, a | fields list, a sort key, a join key — the engine reads the saved rule and runs it as part of the query. You do not have to know the rule exists. Two consequences worth knowing:

A filter on such a field cannot use the index, so it is applied after the scan. Same property as search1; it is a different performance profile from the same query on an accelerated field.
The cost is per row, and it lands hardest on a query that names no fields at all. Naming the fields you want lets the engine skip the rest; a bare search followed by sort, stats or dedup parses every scanned row.

The query says which of these it did: a search reading fields from _raw shows a notice above the results, and it stays there while you read the number.

For csv, kv and xml rules, it does not, and it says so with a warning naming the field and its source type rather than returning a plausible wrong number. Three ways out, in the order you should reach for them:

Run it on search. search1 applies the extraction rule for you.
Extract it inline. Write the extraction in the query and search2 runs it — the result is identical to search1’s:
```
search2 repo="app_logs"
| rex field=_raw "\"level\":\s*\"(?<level>[A-Z]+)\""
| stats count() as n by level
```
| jsonpath input="_raw" output="level" path="$.level" does the same for JSON. Both are vectorised, so this is not a slow fallback. The warning does not fire when you do this.
Turn on field acceleration for the extraction, which materialises the field at write time and so applies to newly ingested data only.

An administrator can turn the automatic behavior off entirely with sondar.matrix.query.search_time_extraction.enabled: false, which restores the warning for every rule type — the fix and its warning move together, so turning it off never leaves a field silently empty. It takes effect at startup, not live: set it in the node’s configuration and restart. There is no route that changes it on a running node, despite the setting being marked dynamic.

Trading-day filtering needs search2. The time picker offers “only trading-day data”, and on search1 it does not work: the search warns search1 does not yet support trading-day search. and returns unfiltered results. Name search2 if you need it.

Modes

The picker offers Fast Mode, Smart Mode and Verbose Mode. What they actually do, measured:

mode	`search` (search1)	`search2`
Fast	events carry only the built-in fields — extracted fields are suppressed	no effect
Smart	extracted fields are present; prunes what is read from the store when the query transforms	no effect
Verbose Mode	same result as Smart Mode	no effect

On search2 the mode picker does nothing. All three settings return identical results; the mode never reaches the matrix engine. If you are on search2, ignore the control.

On search1, the distinction that matters is Fast versus the other two: Fast is faster over a wide range because it does not apply your extraction rules to each event, and correspondingly the field sidebar is bare. Smart and Detailed differ only in how much is read from the store on a transforming query, which you cannot see in the result.

This table is measured against the product, not taken from documentation.

Reading a result

A search produces up to four sub-jobs: the events themselves (LIMIT), the timeline histogram (HISTOGRAM), the field summary (SUMMARY), and — for a query ending in a transforming command — the result table (RESULT).

Warnings appear as a toast. The search will tell you when it has done something you would not otherwise notice: a field it cannot see, an index you lack permission to read, a feature that is switched off. A zero-result search also offers the two dominant causes, because they are invisible in the query itself — a time range that excludes the data, and a bare term containing a dot.

Bare terms and dots. For indexes created before August 2026 the tokenizer did not split on ., so NullPointerException finds nothing in an index full of java.lang.NullPointerException. New indexes split on . by default.

What you can do with a search

From the result bar:

Download the events.
Save as → Alert — the query becomes an alert condition. Name the engine.
Save as → Dataset — a saved query, re-entered with | from dataset:name.
Save as → Report — a saved query with a schedule, re-entered with | from report:name.
Save as → Export task — a scheduled export. See exporting-log-data.md.

“Saved search” is a naming difference, not a missing feature. Sondar has no object called a saved search: the concept is Dataset (a query you re-use) and Report (a query on a schedule). Searching the docs for “saved search” finds nothing; this is what you want.

Mind the syntax — it is | from dataset:my_query, with a leading pipe and a colon. from is the one command whose grammar begins with a pipe, so from dataset "my_query" is a parse error, not a variant spelling.

A saved search does not get automatic search-time extraction. The regex/json rules described above are applied to queries you type, not to one re-entered through from dataset. You get the warning instead. Write the rex/jsonpath into the saved query itself if you need the field.

Search actions turn a field value into a jump — to another search, a dashboard, a report, an external URL, or a POST. They appear on an event when it matches the action’s data restriction, and the target is a template carrying $click.name$ , $click.value$ and $fieldname$ .

Search macros wrap a reusable fragment as `name`, or `name(arg)` with parameters. Define them under Settings → Analysis & Tools → Search macros. Note that a macro is expanded inside a quoted string — repo="mymacro" substitutes — which differs from Splunk.

Live Tail

A live view of logs as they arrive. It ships and works, but it is not a top-level navigation item: open it from the Live Tail link in the search results operation bar, which seeds it with the query you are running and opens it at /tail in a new tab.

Enter a query, and it polls once a second over a sliding window, appending new lines. Controls: start and stop, clear, a light/dark toggle, and a time offset (default 10 seconds) that trades latency for completeness — logs take a moment to arrive and be indexed, so tailing the literal present shows gaps.

Up to 10,000 lines are held; older ones are dropped from the view.

If the connection drops, Live Tail retries with backoff and, after five consecutive failures, stops and says so. It used to stop on the first failure while still displaying “Running”, so a stopped tail looked like a quiet system.

What was verified, and what was not

Verified running: the engine split and its effect on results; the search-time extraction warning; the permission warning; the index/repo alias; the pattern-recognition warning; macros, including argument substitution and quote expansion; search-action token substitution. Also the inline-extraction workaround above — rex and jsonpath on search2 both returned counts identical to search1’s automatic extraction over the same 800 records.

Also the automatic application of regex/json rules — measured against its own control: with sondar.matrix.query.search_time_extraction.enabled=false the same query over the same 800 records returns nothing and warns; with it on it returns search1’s counts exactly.

Two caveats worth knowing. Live Tail’s reconnection path is type-checked but has not been exercised against an induced network failure in a browser. And the search modes’ effect on the field summary specifically is unresolved — the summary sub-job endpoint returns the main job’s content for every mode, so that one dimension is untested rather than confirmed. The modes’ effect on returned fields is measured, above.