External data and lookups

External data and lookups

Enrich search results with data from outside the index — a reference CSV, an external database, a key-value store, a geospatial dataset, or a script. A lookup joins an external source onto search results on a shared key, the way a SQL LEFT JOIN enriches a row.

This is the platform’s “external data” surface, reached under Configure → External Data.

Lookup tables (files)

A lookup table is an uploaded CSV — the reference data you want to join. Under Look Up File, upload the CSV, name the table, set the field separator, and declare its fields. The table is then available to lookup rules.

Lookup rules

A lookup rule maps search-result fields onto table fields and copies selected columns back into the results. Rules are of several types:

• File — join against an uploaded CSV table.
• External — join against an external database connection (see below).
• Kvstore — join against an app’s key-value store.
• Geospatial — enrich with location data.
• Command — run a Python script (placed under the app’s bins/) to compute the enrichment.

Two behaviors make a rule “automatic” or time-aware:

• Auto search — apply the rule automatically to a sourcetype: declare the input fields (in the search result), the output fields (from the table), and whether to rewrite (overwrite an existing field’s value with the table’s).
• Time search — a time-series lookup: match on a time field, with a time format and a shift window, for correlating events against a timeline.

In queries, the lookup SonQL command and the lookup_sql / lookup_sondardb transformers perform the same join at other stages — see SonQL and Transformers.

External databases

External Data → Database connects Sondar to a relational database so its tables can be queried and joined:

• Connection — name a connection, pick the type and driver, set host/port (the platform’s external query port is 9500), credentials, and an optional row limit.
• Driver — the installed JDBC drivers available for connections.
• Query — run SQL over a connection to see its data and build lookups.

An external connection is what an External lookup rule and the lookup_sql transformer point at.

External connections

The External Connect surface is the same connection mechanism for services other than databases — a named endpoint the platform can call out to.

---

Related

• Transformers — lookup_sql / lookup_sondardb do the same join in the collection pipeline.
• SonQL — the lookup command at search time.