bodyExpression: allow truncating the body (substring / left) for pattern mining #3650

Summary

Since #3426, bodyExpression on clickstack_event_patterns / clickstack_emerging_signals accepts a column, a map reference, or JSONExtractString over one. That covers mining a JSON field. One case is still blocked: mining only the start of a long body.

Many error logs are long multi-line stack traces or request/response dumps (several KB, sometimes up to 1 MiB per line). For pattern mining the first few hundred characters (exception class, message, first frames) are what tell patterns apart, and the rest:

• is transferred from ClickHouse for every sampled row (10K rows × large bodies), and
• makes Drain slower and the templates very long. Templates over 4,000 characters get no drill-down whereSnippet, so the
  most frequent error pattern is often the one that can't be drilled into (in one of our services, a 7 KB stack trace
  that was 94% of the error lines).

Agents tried bodyExpression: "substring(Body, 1, 2000)" and got the "single column or map reference" error.

Proposal

Extend the normalizeBodyExpression allowlist with one shape: substring(<allowed expression>, 1, <integer literal>) (and/or left(<allowed expression>, <integer literal>)), where the inner expression is anything already allowed (a column, a map reference, or JSONExtractString(...)). Integer literals only, with a sensible cap. No other functions, so the injection safety of the current allowlist is unchanged.

Alternative

A maxBodyChars option on the two tools, applied as substring(body, 1, maxBodyChars) in the sample query. Simpler for agents, and no expression parsing.

Happy to send a PR for whichever you prefer.

Related

• emerging_signals reports dated log lines as both "new" and "gone": Drain keeps the literal date, and each window is mined separately #3544 (option 3, deferred to this issue), fix(mcp): make event_patterns drill-down snippets executable #3426 (JSONExtractString in bodyExpression), MCP response trimming replaces the largest values whole (often the top result) and ignores topN; clickstack_table has no limit, having or default order #3542 (response trimming)