feat(spec): a field marked searchable: true is searched when its object declares no searchableFields

.changeset/29008-field-searchable-honoured.md

@@ -0,0 +1,19 @@
+---
+'@objectstack/spec': minor
+'@objectstack/metadata-protocol': minor
+'@objectstack/lint': minor
+---
+
+feat(spec): a field marked `searchable: true` is searched when its object declares no `searchableFields`
+
+Clause-②: yes (widening)
+Metadata-protocol: semantic
+
+`FieldSchema.searchable` was declared, defaulted and served, and nothing read it: `$search` skipped a field marked `searchable: true`, and a `$searchFields` override naming it was refused `400 INVALID_FIELD` "is not searchable" while the object's served schema said `searchable: true`. It is now honoured (ruling record 6573569376, letter A).
+
+- **Before.** With no `searchableFields` declared, the search set was the type-based auto-default only: the text-like and `select` columns. A `code`, `html`, `lookup` or `json` field marked `searchable: true` was not scanned, and naming it in `$searchFields` answered 400.
+- **Now.** With no `searchableFields` declared, the set is the type-based auto-default plus every field marked `searchable: true`, of any type except `secret`, `password` and `vector`, which stay excluded (a virtual `formula` field has no stored column and stays out too). The `$searchFields` ingress allow-list is the same set, so naming a marked field narrows the search to it instead of answering 400. The flag is read before the auto-default's own exclusions, so a marked `hidden` field or a marked field named like a system column joins too.
+- **Unchanged.** A declared `searchableFields` is still the whole search set, and no field's own flag is read there. A field without the flag (or with `searchable: false`, the default) resolves exactly as before. A credential field is still never scanned.
+- **What may now match more.** A field you already marked `searchable: true` on an object without `searchableFields` is now part of `$search`. For a `lookup` that means its stored record id, not the related record's title. Remove the flag, or declare `searchableFields` to pin the set.
+- **New export.** `SEARCH_FLAG_EXCLUDED_TYPES` (`@objectstack/spec/data`) names the three hard exclusions. `SearchFieldMeta` gains the optional `searchable`.
+- **Messages.** The `$searchFields` refusal for a field outside the auto-default, and the `searchable-field-unsearchable` lint verdict, now name the missing `searchable: true` and offer it as the fix, instead of calling the field's type unscannable.

content/docs/api/data-api.mdx

@@ -23,7 +23,7 @@ Query records with filtering, sorting, selection, and pagination.
 | `top` | query | Max records to return. No default — omitting it returns all matching records. |
 | `skip` | query | Offset |
 | `expand` | query | Comma-separated list of relations to eager-load. Must name a reference field (`lookup` / `master_detail` / `user` / `tree`) — otherwise `400 INVALID_FIELD`. |
-| `search` | query | Full-text search term, scanned case-insensitively across the object's searchable fields (its declared `searchableFields`, or a text-like auto-default when none are declared). When that set is empty — no `searchableFields` entry names a stored field, and the auto-default finds none — a non-blank term is `400 INVALID_FIELD`, never every row. |
+| `search` | query | Full-text search term, scanned case-insensitively across the object's searchable fields (its declared `searchableFields`, or, when none are declared, a text-like auto-default plus every field marked `searchable: true` except `secret` / `password` / `vector`). When that set is empty — no `searchableFields` entry names a stored field, and the auto-default finds none — a non-blank term is `400 INVALID_FIELD`, never every row. |
 | `searchFields` | query | Comma-separated subset of the searchable fields for `search` to scan — narrows the scan, never widens it. A name outside the searchable set is `400 INVALID_FIELD`. |
 
 > **Note:** OData-style `$`-prefixed parameters (`$filter`, `$select`, `$orderby`, `$top`, `$skip`, `$expand`, `$count`, `$search`) are also accepted directly on this same endpoint as aliases — they're normalized internally to the parameter names above. There is no separate standalone OData endpoint.

content/docs/data-modeling/field-types.mdx

@@ -1023,7 +1023,7 @@ These properties are available on **all** field types:
 | `required` | `boolean` | `false` | Whether the field is required |
 | `unique` | `boolean \| 'global' \| 'organization'` | `false` (`'organization'` on `autonumber`) | Unique constraint **and its scope** (ADR-0120). `'organization'` = one holder per organization; `true` = that same per-organization scope (positional synonym — prefer the explicit spelling in new code); `'global'` = one holder across the whole installation; `false` = no constraint. Omitted ⇒ `false`, except on an `autonumber` field, where omitted ⇒ `'organization'` — an auto-number is a business identifier, so the platform makes it unique per organization by default; write `unique: false` explicitly to opt out. `'tenant'`/`'org'` are rejected — the word is `'organization'`. Full rule in the `unique` doc block in [`FieldSchema`](/docs/references/data/field) |
 | `multiple` | `boolean` | `false` | Allow array of values (multi-record lookup, multi-select, multi-file). An emptied multi-value lookup reads back as `[]`, never `null` — the `multiple` doc block in [`FieldSchema`](/docs/references/data/field) |
-| `searchable` | `boolean` | `false` | Include in search index |
+| `searchable` | `boolean` | `false` | Search this field when the object declares no `searchableFields` — any type but `secret` / `password` / `vector`; a declared `searchableFields` is the whole set |
 | `sortable` | `boolean` | `true` | Allow sorting by this field |
 | `hidden` | `boolean` | `false` | Hide from default views |
 | `readonly` | `boolean` | `false` | Server contract — non-system writes to this field are stripped on **both** INSERT and UPDATE (the value falls back to `defaultValue`), not just hidden in the UI (#2948 / #3043) |

content/docs/data-modeling/fields.mdx

@@ -446,7 +446,7 @@ These properties are available on all field types:
 
 | Property | Type | Default | Description |
 | :--- | :--- | :--- | :--- |
-| `searchable` | `boolean` | `false` | Include in full-text search |
+| `searchable` | `boolean` | `false` | Search this field when the object declares no `searchableFields` — any type but `secret` / `password` / `vector`; a declared `searchableFields` is the whole set |
 | `externalId` | `boolean` | `false` | Not read: nothing matches on it. A seed dataset names its match field in its own `externalId`, an import in its mapping's `upsertKey` |
 
 Database indexes are declared at the object level in the object's `indexes[]` array — there is no per-field `index` flag. See [Indexing](/docs/data-modeling/indexing).

content/docs/data-modeling/queries.mdx

@@ -538,7 +538,8 @@ Only `query` and `fields` are implemented. The engine expands `search` into a dr
 <Callout type="info">
 `fields` (and its query-parameter spelling, `?searchFields=` / `?$searchFields=`) can only
 **narrow** the scan within the server-resolved searchable set — the object's declared
-`searchableFields`, or a text-like auto-default when none are declared. Over the
+`searchableFields`, or, when none are declared, a text-like auto-default plus every
+field marked `searchable: true` except `secret` / `password` / `vector`. Over the
 REST/protocol ingress a name outside that set is `400 INVALID_FIELD`, with distinct
 messages for a field that does not exist and a real field that is not searchable. An
 override that names no field at all (`?$searchFields=`, `?$searchFields=,`, an empty

content/docs/data-modeling/schema-design.mdx

@@ -87,7 +87,8 @@ enable: {
 ### Global search — `searchable` + `searchableFields`
 
 `searchable: true` includes an object's records in global search and the record
-picker. By default the search matches the name/title field plus short-text fields.
+picker. By default the search matches the name/title field plus short-text fields, and
+every field marked `searchable: true` (any type but `secret` / `password` / `vector`).
 To control exactly which fields are matched, set `searchableFields` on the object
 (ADR-0061):
 
@@ -102,7 +103,8 @@ To control exactly which fields are matched, set `searchableFields` on the objec
 Queries then pass a top-level `$search` parameter to match across `searchableFields`
 (e.g. `{ $search: 'acme' }`); a request may narrow the set with
 `$searchFields`. When `searchableFields` is unset, search falls back to the
-name/title field plus short-text fields. When neither leaves a stored field to scan — a
+name/title field plus short-text fields and the fields marked `searchable: true`; a
+declared `searchableFields` is the whole set, and no field's own flag is read. When neither leaves a stored field to scan — a
 junction object of lookups with a `formula` title, say — a list search is refused
 `400 INVALID_FIELD` rather than answered with every row, and global search skips the
 object; mirror the title onto a stored text field, as below.

content/docs/protocol/objectql/query-syntax.mdx

@@ -1186,7 +1186,8 @@ const structured: QueryAST = {
 Field resolution is server-side and never client-trusted: the requested fields
 (`searchFields`, or `search.fields` in the structured form) are **intersected** with the
 object's declared `searchableFields` (or, absent those, an auto-default of the name field
-plus short-text/enum fields), so naming a field outside that set can never widen the
+plus short-text/enum fields and every field marked `searchable: true` except `secret` /
+`password` / `vector`), so naming a field outside that set can never widen the
 search — and over the REST/protocol ingress it is `400 INVALID_FIELD` outright (#4254),
 because the engine-side intersection alone used to drop the unknown name and fall back to
 scanning the full searchable set. Internal callers reaching `engine.find()` directly keep

content/docs/references/data/field.mdx

@@ -57,7 +57,7 @@ const result = CurrencyConfigSchema.parse(data);
 | **format** | `string` | optional | Free-form string whose meaning depends on the field type and on the reader. The spec declares NO vocabulary for it and checks nothing but that it is a string, so any string parses on any field type. On an `autonumber` field it is the record-number PATTERN — the shorthand that predates `autonumberFormat`, which wins when both are present: `format: 'INV-{0000}'` mints `INV-0001` on both the query engine and the SQL driver, while a value carrying no `{...}` token is emitted as literal text with the bare counter appended (`format: 'email'` mints `email1`). Prefer `autonumberFormat` on a new field. On any other field type the server does not act on it: it picks no column type, coerces no value and runs no check from it. The Studio UI reads it as a display hint, in words and with defaults that its renderers own and declare. For example, the `date` and `datetime` cells read it as a display STYLE, and on a plain-text field the shared cell-renderer resolver reads a small set of words that promote the cell to a richer renderer, such as a link; each of those falls back silently to a default rendering when it does not recognise the word. To constrain a VALUE, use the field `type` (the write-time record validator's built-in email, url and phone checks key on `type`, never on this key) or a `format` validation rule, whose own `format` key is the closed set `email` \| `url` \| `phone` \| `json`. |
 | **required** | `boolean` | optional (default: `false`) | Write-time contract (ADR-0113): an insert must provide a non-null value, and an update may not null it out. On a multi-value lookup (`multiple: true`) required means NON-EMPTY array — an emptied required set fails validation loudly; `[]` does not satisfy it. NOT a column constraint — the physical NOT NULL is a separate explicit opt-in (`storage.notNull`), so tightening this on a deployed object is safe: existing null rows stay readable, and editable as long as the write does not touch this field. |
 | **storage** | `{ notNull?: boolean }` | optional | Physical storage constraints (ADR-0113). Owns the DDL the write contract deliberately does not imply. Absent = no storage-level constraint requested. |
-| **searchable** | `boolean` | optional (default: `false`) | Is searchable |
+| **searchable** | `boolean` | optional (default: `false`) | Search this field when its object declares no `searchableFields`. The set `search` scans, and the set a `$searchFields` override may name, is then the type-based default (the text-like and select columns) plus every field marked `searchable: true`, of any type except `secret`, `password` and `vector`, which stay excluded (a virtual `formula` field has no stored column, so it is never scanned either). `false`, the default, does not remove a field the type-based default already scans; to choose the set, declare `searchableFields`. When the object declares `searchableFields`, that list is the whole search set and this flag is not read (ADR-0061). |
 | **multiple** | `boolean` | optional (default: `false`) | Allow multiple values (Stores as Array/JSON). Declarable ONLY on the multi-capable types — select, lookup, user, file, image — and redundantly on the inherently-multi option types (multiselect, checkboxes, tags); `multiple: true` on any other type, `radio` included, is REFUSED at parse. An emptied multi-value lookup reads back as `[]`, never `null` — the rule binds every writer (cascade repair, form clears, API writes), not just cascade repair. |
 | **unique** | `boolean \| 'global' \| 'organization'` | optional | Unique constraint and its scope (ADR-0120). 'organization' = one holder per organization (NULL-safe composite with the organization key part on organization-scoped objects) — prefer this explicit spelling in new code; true = same per-organization scope (positional synonym, stays valid); 'global' = one holder across the whole installation. 'tenant'/'org' are rejected — the word is 'organization'. Omitted ⇒ false, EXCEPT on an `autonumber` field, where omitted ⇒ 'organization' (an auto-number is a business identifier, so the platform makes it unique per organization by default — the same tenant-composite shape an explicit `unique: true` produces). To opt an autonumber field out, write `unique: false` explicitly — legitimate only for a display-only sequence that is not used to identify the record; the platform's duplicate scan (`os migrate duplicates`) still treats every autonumber field as an identifier. |
 | **defaultValue** | `any` | optional | Default applied on INSERT when the field is omitted or null (`''` is a real value, not absence). Three legal shapes, discriminated in the engine's own order: a CEL Expression envelope `{ dialect: 'cel', source: 'today()' }` (accepted structurally; result type is a runtime concern); a runtime TOKEN — `NOW()` on `datetime`/`date`/`time` only, `current_user` on `user` or `lookup` with `reference: 'sys_user'` only, neither on a multi-value field; or a LITERAL, which must satisfy this field's own stored value contract (ADR-0104 D1 `valueSchemaFor`). Anything else is refused at parse time with a prescriptive message. |