Skip to content
Sondar

Log pattern recognition

Log pattern recognition

Section titled “Log pattern recognition”

Collapsing thousands of log lines into the handful of shapes they actually have — and the two things that will otherwise make the answer confidently wrong.

What it does

Section titled “What it does”

Pattern recognition groups log messages by structure rather than content. Lines that differ only in a timestamp, an id or a number collapse into one pattern — a template with the varying parts masked — so a million lines become a few dozen shapes you can actually read.

It is the fastest way to answer “what is in this index that I have never looked at?”, and to notice a new error shape appearing.

The implementation is a Drain-style tree: messages are tokenised, walked down a fixed-depth tree and merged when they are similar enough. The defaults that matter are depth (7 on the agent-side config, 6 server-side), similar (0.4) and tokenizer (simple; json, smart and a CJK tokenizer also ship).

Two constraints that change the answer

Section titled “Two constraints that change the answer”

1. It runs on search2 only

Section titled “1. It runs on search2 only”

logcluster exists in the search2 grammar and nowhere else — there is no LOGCLUSTER token in the search1 parser at all. A pipeline using it must run on search2:

search2 repo="app_logs" | logcluster summary

If the engine is not named, the global Default Search Engine setting decides which one runs, so a saved search or dashboard panel using logcluster can stop working when that setting changes. Name the engine.

2. It is on for your own indexes, and off for ours

Section titled “2. It is on for your own indexes, and off for ours”

Pattern recognition is on by default for every non-internal event index, including default — the index events land in when a sender names none. A fresh install ships one target covering all of them, and because that target is a rule rather than a fixed list, an index you create next month is covered too without revisiting the setting.

It is off for the product’s own internal indexes (_internal, _alert_event and friends). Those are sondar’s own logs, not yours; clustering them costs parse time for no benefit on day one. Switch them on in Settings if you want them.

Enabling later does not backfill, so the default matters more than it looks. Patterns are computed at ingest and written as they arrive — nothing walks history afterwards. Logs that arrive while an index is switched off can never be clustered, even after you switch it on; only what arrives from that point forward is. This is why the feature ships on rather than waiting to be discovered.

That also means the first batch of a brand-new sourcetype usually yields no patterns: the parser for a sourcetype loads asynchronously, and the batch that triggers the load is itself passed through. The next one clusters.

Targets are set for named indexes or index groups in Settings, and turning the feature off removes them all. Once off, it stays off — the shipped default is applied once, at install, and never reasserted.

This used to be the sharpest edge in the feature. With it off, the query still returned a result: a single pattern covering 100% of the logs, with an empty template — a confident, plausible, wrong answer. Three corpora with obviously different message shapes all collapsed to the same thing, which is how it was caught.

It now says so instead:

Pattern recognition is not enabled for any index, so no log patterns can be detected. Enable it for an index in Settings; the result below is not a real pattern.

The check is instance-wide — if no index has pattern recognition enabled, no logcluster query can produce a real pattern, so the warning has no false positives. With the shipped default in place it should not normally fire; seeing it means the feature was switched off deliberately. The converse is not covered: with the feature on for index A and a query against index B, the query returns the empty-template result and no warning. Enable it for the index you intend to search.

Over the API the targets live at /api/v1/repo_logcluster_target (GET to list, PUT to set), addressed either by index (repo) or by index group.

The command

Section titled “The command”

The mode keyword is required — the grammar is logcluster (summary | search | trend | compare | alert | analyze) …, so a bare | logcluster is a parse error, not a default.

mode what it returns
summary the pattern list — template, count, share
search the raw events behind a pattern
trend pattern volume over time
compare one window against another
alert the shape used by a pattern-based alert rule
analyze the analysis view

Options are name=value after the mode. The ones worth knowing:

option default meaning
level 10 how many patterns to aim for, 1–10. Lower means fewer, broader patterns; 10 is the most granular and is the default
limit 100000 maximum patterns
max_events 100000 events considered
span auto bucket width for trend
sort_by num order the pattern list
thin false reduced output
timeshift — offset for compare

by groups patterns by a field, and asc/desc set the direction:

search2 repo="app_logs" | logcluster summary level=6 by service desc

Reading the result

Section titled “Reading the result”

A pattern’s template shows the parts that varied, masked: a run of differing text as *, a timestamp as its format, a number as [min,max]. The columns are the log count, the share of the total, and a sparkline of volume over time.

Opening one pattern gives its overview, the events behind it, and its trend — and a trend can be added to a dashboard or saved as an alert, which is how you turn “this shape is new” into something that pages you.

(This section describes the screens as the product presents them. Unlike the constraints above it was not re-verified against a running instance for this revision — the two facts that change the answer were.)

Tuning

Section titled “Tuning”

Defaults are reasonable; reach for these only when the patterns are obviously too coarse or too fine.

  • similar (0.4) — the merge threshold. Higher splits more.
  • depth — tree depth before templates form.
  • tokenizer — simple splits on whitespace and punctuation. Use json for structured payloads; smart and the CJK tokenizer also ship.
  • Masks — regexes whose matches are always treated as variable. The shipped config masks the agent’s own runner/sender/transformer identifiers so they do not fragment every pattern.
  • Memory — patterns are capped per node per source type (default 1,000,000). Patterns from different nodes can exceed that total.

These are commented in the rule configuration itself, under sondar.logcluster.parser.*.

When one pattern becomes many

Section titled “When one pattern becomes many”

If a sourcetype shows far more patterns than it has real message shapes — dozens of near-identical templates differing only by a hostname or a service name — this is the one tuning problem worth knowing about.

The parser builds a tree keyed on the leading tokens of each line, and it only generalises a token automatically when that token contains a digit. So web01 sshd: connection closed generalises, and alpha sshd: connection closed does not: each distinct alphabetic host opens its own branch. Measured: eight hosts sending one identical message produce eight patterns; the same corpus with host0–host7 produces one.

Syslog puts the host and the process at the start of every line, so this is the common case rather than an exotic one.

The quickest fix needs no configuration at all: the Pattern Count slider. On the search page’s Pattern tab, drag Pattern Count from Big towards Small. That re-clusters the results at a coarser granularity, and the fragments merge. Measured on 317 syslog lines from twelve alphabetic hosts: at Big it reports twelve patterns, one per host; at Small it reports one — <*> sshd: Accepted publickey for deploy from port <*>, 100% of the logs.

Try that first. It is a query-time dial, so it changes what this search shows and nothing else.

The durable fix is a mask. Where a sourcetype always has the problem, add a user-defined mask to its pattern rule so the variable is generalised at INGEST, permanently and for every future query:

[pattern](alpha|bravo|charlie|delta)[/pattern]

Anything the mask matches is treated as a variable before the tree is walked, so one pattern comes back instead of one per host. A regex covering your naming convention ([pattern](node-[a-z]+)[/pattern]) works the same way.

Timestamps need no help — both 2026-08-18T06:00:00Z and the digit-free Aug 18 06:00:00 already collapse.

Known gaps

Section titled “Known gaps”
  • Per-index scoping of the warning. As above, the “not enabled” warning is instance-wide; a query against an index that specifically lacks a target gets the empty-template result with nothing said. Narrowing it needs the query’s indexes at a layer that does not currently have them.
  • level is not a pattern count. It is a granularity dial, 1–10, not a pattern count.

See also

Section titled “See also”