fix(objectql,metadata-protocol)!: an in-process search-field override that names no searchable field is refused, never widened to a default set (#28120)

.changeset/28120-in-process-search-override-no-field.md

@@ -0,0 +1,20 @@
+---
+'@objectstack/objectql': patch
+'@objectstack/metadata-protocol': patch
+'@objectstack/spec': patch
+---
+
+fix(objectql,metadata-protocol)!: an in-process search-field override that names no searchable field is refused, never widened to a default set (#28120)
+
+Clause-②: no (narrowing) — two in-process tolerances the Data API doors already refuse are closed. The stored-metadata reader seam (`narrowStoredMetadataSearch`) no longer narrows an override that names no field (an empty list, a comma string with no name, a list with no string entry): it passes it to the engine, which refuses it. And the engine (`find`, `findOne`, `count`, `aggregate`) refuses a `searchFields` or object-form `search.fields` that names none of the fields the search scans, where `resolveSearchFields` used to drop the names and scan the whole default set. No export, answer member, status or error code is added: the refusals are the doors' `VALIDATION_ERROR` / 400 and `INVALID_FIELD` / 400. Folds #28121.
+Metadata-protocol: none
+
+<!-- adr-0087: not-required (no-migration-prescription) No key of any schema is added, removed, renamed or re-shaped, and no export or type changes: an engine option value the doors already refuse is now refused in process too, with the doors' codes. There is nothing for a migration to rewrite and no tombstone. -->
+
+**BREAKING** accept-set narrowing at the in-process search override (`@objectstack/objectql`, `@objectstack/metadata-protocol`).
+
+- **Before.** A host handler, hook or flow reading `sys_metadata` / `sys_metadata_history` through the reader seam with `searchFields: []` was served: the seam read the empty list as "names none" and wrote the narrowed default set over it. On any object, a search whose override named no searchable field (every name unknown, unsearchable or withheld) was resolved to the object's default search set.
+- **Now.** The empty override reaches the engine's refusal through the seam: `400 VALIDATION_ERROR`, before any driver read. An override naming no searchable field is refused `400 INVALID_FIELD` (`param: 'searchFields'` or `'search.fields'`, `field` / `fields` naming what the caller wrote) before any driver read, the code the REST door answers for the same value. `mapDataError` serves both with the door's status and code.
+- **Unchanged.** An absent (or `null`) override searches the default set. An override naming no searchable field sent without a `search` is still not judged in process, since nothing is scanned (the door refuses it with or without a search term). A named override narrows as before. An override naming SOME searchable field still has its other names dropped in process (the doors refuse them). An object with no searchable field at all keeps its own refusal (#24284).
+
+**The remedy.** Name at least one field the object searches (its `searchableFields`, or the default set), or leave the key out to search the default fields.

content/docs/data-modeling/queries.mdx

@@ -545,7 +545,9 @@ override that names no field at all (`?$searchFields=`, `?$searchFields=,`, an e
 array) is `400 VALIDATION_ERROR`, as `$select=` is; leave the parameter out to scan the
 default set. The engine used to drop unknown names and fall back to the full searchable
 set — a parameter that exists to narrow a search, silently widening it. Internal callers
-reaching `engine.find()` directly are unaffected.
+reaching `engine.find()` directly get the same two refusals for an override naming no field
+(`400 VALIDATION_ERROR`) and one naming none of the searchable fields (`400 INVALID_FIELD`);
+an override naming at least one searchable field there still has its other names dropped.
 </Callout>
 
 ### Searching by a related record's title — mirror the value

content/docs/protocol/objectql/query-syntax.mdx

@@ -1190,7 +1190,10 @@ plus short-text/enum fields), so naming a field outside that set can never widen
 search — and over the REST/protocol ingress it is `400 INVALID_FIELD` outright (#4254),
 because the engine-side intersection alone used to drop the unknown name and fall back to
 scanning the full searchable set. Internal callers reaching `engine.find()` directly keep
-the tolerant intersection, but not a search with nothing to scan: when no
+the tolerant intersection for an override naming at least one searchable field (the other
+names are dropped), but an override naming none of them is refused there too,
+`400 INVALID_FIELD`, rather than widened to the full set. Nor is a search with nothing to
+scan answered: when no
 `searchableFields` entry names a stored field and the auto-default finds none
 (`internal: true` fields never count), the engine itself refuses a non-blank `search`
 with `400 INVALID_FIELD` (`param: 'search'`) on `find`, `findOne`, `count` and

packages/metadata-protocol/src/protocol.ts

@@ -5186,18 +5186,28 @@ export interface StoredMetadataSearchSchema {
  *   string or an array) — naming one is refused, `INVALID_FIELD` / 400, the
  *   evaluate refusals' envelope, under the caller's wire spelling of the slot
  *   (`wireSpelling`, the door's; the bare canonical name when absent);
+ * - [#28120] an explicit list that names NO field — an empty array, a blank or
+ *   comma-only string, an array with no string entry — is not "names none": it
+ *   is passed through untouched, for the engine's own refusal to answer
+ *   (`400 VALIDATION_ERROR` for an empty one, #27732; `400 INVALID_FIELD` for
+ *   one naming no searchable field). Narrowing it here wrote a non-empty set
+ *   back over the caller's list, so an override the door and the engine both
+ *   refuse was served. The door never reaches this with one: its ingress gate
+ *   refuses it first;
  * - a search that names none is NARROWED: the object's resolved searchable set
  *   ({@link resolveSearchFieldResolution}) minus those columns is returned, for
  *   the caller to run as `searchFields` — the engine intersects an override
  *   with that set and never widens it;
  * - a set that narrows to nothing is refused rather than returned empty: an
- *   empty override is ABSENT to the engine, which would then scan the whole
- *   default set, these columns included.
+ *   empty override is refused by the engine as one naming no field, a
+ *   caller's mistake this caller did not make, so the family's own refusal
+ *   names the cause instead.
  *
  * Returns `undefined` when the query runs as it is — an object outside the
- * family, no search, an explicit list that passes, or no readable field map
- * (the engine has none to expand a search over either; the same rule the door's
- * field gates apply: a non-object, array or empty `fields`). Throws the refusal.
+ * family, no search, an explicit list that passes or names no field, or no
+ * readable field map (the engine has none to expand a search over either; the
+ * same rule the door's field gates apply: a non-object, array or empty
+ * `fields`). Throws the refusal.
  */
 export function narrowStoredMetadataSearch(
     object: string,
@@ -5222,6 +5232,8 @@ export function narrowStoredMetadataSearch(
         if (refusal) throw refusal;
         return undefined;
     }
+    // [#28120] A list naming no field is the engine's to refuse, never narrowed.
+    if (typeof explicit === 'string' || Array.isArray(explicit)) return undefined;
     if (search == null) return undefined;
     const fields = schema?.fields;
     if (!fields || typeof fields !== 'object' || Array.isArray(fields) || Object.keys(fields).length === 0) {

packages/objectql/src/engine-empty-search-fields-refusal.test.ts

@@ -221,3 +221,84 @@ describe('[#27732] the in-process engine refuses a search-field override naming
     expect(await answered(engine, 'find', { search: 'harbour', searchFields: null })).toEqual(DEFAULT_ANSWER.find);
   });
 });
+
+/**
+ * [#28120] …and an override that names no SEARCHABLE field — every name
+ * unknown, or no entry a field name at all — is refused `400 INVALID_FIELD`,
+ * the code the door answers the same value with (`assertSearchFieldsAreSearchable`,
+ * #4254). `resolveSearchFields` dropped the unknown names and fell back to the
+ * whole default set, so a typo or a renamed field searched every default
+ * column instead of the one the caller meant. An override naming at least one
+ * searchable field keeps the engine's tolerance: the other names are dropped.
+ */
+describe('[#28120] the in-process engine refuses a search-field override naming no searchable field', () => {
+  let engine: ObjectQL;
+  let reads: Array<{ via: string; where: unknown }>;
+
+  beforeEach(async () => {
+    engine = new ObjectQL();
+    const made = makeDriver();
+    reads = made.reads;
+    engine.registerDriver(made.driver, true);
+    await engine.init();
+    engine.registry.registerObject(account, 'test-package');
+  });
+
+  describe.each(VERBS)('%s', (verb) => {
+    it('an all-unknown searchFields list is refused 400 INVALID_FIELD, before any driver read', async () => {
+      const answer = await wireAnswer(run(engine, verb, { search: 'harbour', searchFields: ['nmae', 'town'] }));
+      expect(answer).toMatchObject({ status: 400, code: 'INVALID_FIELD', param: 'searchFields' });
+      expect(reads).toEqual([]);
+    });
+
+    it('the comma-string and object-form spellings are refused the same way', async () => {
+      expect(await wireAnswer(run(engine, verb, { search: 'harbour', searchFields: 'nmae' })))
+        .toMatchObject({ status: 400, code: 'INVALID_FIELD', param: 'searchFields' });
+      expect(await wireAnswer(run(engine, verb, { search: { query: 'harbour', fields: ['nmae'] } })))
+        .toMatchObject({ status: 400, code: 'INVALID_FIELD', param: 'search.fields' });
+      expect(reads).toEqual([]);
+    });
+
+    it('a list with no field-name entry at all is refused too', async () => {
+      expect(await wireAnswer(run(engine, verb, { search: 'harbour', searchFields: [42] })))
+        .toMatchObject({ status: 400, code: 'INVALID_FIELD', param: 'searchFields' });
+      expect(reads).toEqual([]);
+    });
+
+    it('CONTROL: a partly-known override still narrows to its known names (the tolerance kept)', async () => {
+      expect(await answered(engine, verb, { search: 'harbour', searchFields: ['name', 'nmae'] })).toEqual(NAME_ONLY_ANSWER[verb]);
+    });
+
+    it('CONTROL: a valid override narrows and an absent one searches the default set, as before', async () => {
+      expect(await answered(engine, verb, { search: 'harbour', searchFields: ['name'] })).toEqual(NAME_ONLY_ANSWER[verb]);
+      expect(await answered(engine, verb, { search: 'harbour' })).toEqual(DEFAULT_ANSWER[verb]);
+    });
+  });
+
+  it('the refusal names the field the caller wrote', async () => {
+    const err: any = await engine.find(account.name, { search: 'harbour', searchFields: ['nmae'] }).catch((e: unknown) => e);
+    expect(err).toMatchObject({ field: 'nmae', fields: ['nmae'], object: account.name });
+    expect(err.message).toContain("names no searchable field in 'searchFields'");
+  });
+
+  it('the REST door answers the same status and code for the same override', async () => {
+    const protocol = new ObjectStackProtocolImplementation(engine);
+    const door = await wireAnswer(protocol.findData({ object: account.name, query: { search: 'harbour', searchFields: 'nmae' } as any }));
+    const inProcess = await wireAnswer(engine.find(account.name, { search: 'harbour', searchFields: ['nmae'] }));
+    expect({ status: inProcess.status, code: inProcess.code }).toEqual({ status: door.status, code: door.code });
+    expect(door).toMatchObject({ status: 400, code: 'INVALID_FIELD' });
+    expect(reads).toEqual([]);
+  });
+
+  it('CONTROL: an object with no searchable field keeps its own refusal (#24284), naming that cause', async () => {
+    engine.registry.registerObject({
+      name: 'link_28120',
+      label: 'Link',
+      fields: { id: { name: 'id', label: 'ID', type: 'text' as const, primaryKey: true } },
+      searchableFields: ['gone'],
+    } as any, 'test-package');
+    const err: any = await engine.find('link_28120', { search: 'harbour', searchFields: ['nmae'] }).catch((e: unknown) => e);
+    expect(err).toMatchObject({ code: 'INVALID_FIELD', param: 'search' });
+    expect(err.message).toContain('has no searchable field');
+  });
+});

packages/objectql/src/engine-search-empty-set.test.ts

@@ -176,14 +176,14 @@ describe('[#24284] a search over an empty resolved search set is refused', () =>
         }
     });
 
-    it('control: an object with a stored text field still filters, and an override intersected to nothing falls back to its set', async () => {
+    it('control: an object with a stored text field still filters, and an override intersected to nothing is refused', async () => {
         expect((await engine.find('lib_book', { search: 'Dune' })).map((r) => r.id)).toEqual(['b1']);
         expect(await engine.find('lib_book', { search: 'zzzz-nothing' })).toEqual([]);
         expect(await engine.count('lib_book', { search: 'zzzz-nothing' })).toBe(0);
-        // A direct engine caller naming only fields outside the allowed set: the
-        // tolerant intersection falls back to the allowed set — a FILTERED
-        // answer, never every row.
-        expect((await engine.find('lib_book', { search: 'emma', searchFields: ['nope'] })).map((r) => r.id)).toEqual(['b2']);
+        // [#28120] A direct engine caller naming only fields outside the allowed
+        // set is refused, never answered over the whole allowed set.
+        await expect(engine.find('lib_book', { search: 'emma', searchFields: ['nope'] }))
+            .rejects.toMatchObject({ status: 400, code: 'INVALID_FIELD', param: 'searchFields', field: 'nope' });
     });
 });
 

packages/objectql/src/engine.ts

@@ -5,7 +5,7 @@ import { QueryAST, QueryInput, HookContext, ServiceObject } from '@objectstack/s
 // [commit 74155c735] The defaulting node schema `fillQueryAstDefaults` runs author input
 // through — the declared `.default()` stays in `packages/spec`, the engine
 // only invokes it.
-import { SortNodeSchema, FullTextSearchSchema } from '@objectstack/spec/data';
+import { SortNodeSchema, FullTextSearchSchema, resolveSearchFieldResolution } from '@objectstack/spec/data';
 import {
   EngineQueryOptions,
   DataEngineInsertOptions,
@@ -769,6 +769,53 @@ function refuseSearchOverrideNamingNoField(object: string, requested: unknown, p
   throw err;
 }
 
+/**
+ * [#28120] …and an override that names no field the search can scan is refused
+ * `400 INVALID_FIELD`, the code the door answers the same value with
+ * (`assertSearchFieldsAreSearchable`, #4254): every name unknown, unsearchable,
+ * or withheld from the scan (`internal: true`, credential types), or no entry a
+ * field name at all.
+ *
+ * `resolveSearchFields` drops the names outside the allowed set and, when none
+ * is left, falls back to the whole set — the typo or the renamed field the
+ * caller wrote to narrow the search ran over the object's default fields
+ * instead, untold. Judged on the set the expansion resolves over (`allowed`),
+ * so this refuses exactly the overrides that fallback would have widened. An
+ * override naming at least one allowed field is left to the intersection (the
+ * engine's tolerance for the other names stands), and an empty `allowed` is
+ * left to the expansion's own refusal (#24284), which names that cause.
+ */
+function refuseSearchOverrideNamingNoSearchableField(
+  object: string,
+  requested: unknown,
+  param: string,
+  allowed: readonly string[],
+): void {
+  const entries = typeof requested === 'string'
+    ? requested.split(',').map((s) => s.trim()).filter(Boolean)
+    : Array.isArray(requested) ? requested : undefined;
+  if (entries === undefined || entries.length === 0 || allowed.length === 0) return;
+  const allowSet = new Set(allowed);
+  if (entries.some((f) => typeof f === 'string' && allowSet.has(f))) return;
+  const names = entries.filter((f): f is string => typeof f === 'string');
+  const err: any = new Error(
+    `ObjectQL search on '${object}' names no searchable field in '${param}': got ${JSON.stringify(requested)}. `
+    + 'The query was not run: an override naming none of the fields the search scans was dropped, which '
+    + "scans the object's default search fields. Name a searchable field, or leave the override out to "
+    + 'search the default fields.',
+  );
+  err.status = 400;
+  err.httpStatus = 400;
+  err.code = 'INVALID_FIELD';
+  if (names.length > 0) {
+    err.field = names[0];
+    err.fields = names;
+  }
+  err.param = param;
+  err.object = object;
+  throw err;
+}
+
 /**
  * Rows per page for the autonumber seeding scan (#6249). This is a PAGE size,
  * not a cap: the walk pages until the scope is exhausted. The number is the one
@@ -13453,8 +13500,9 @@ export class ObjectQL implements IObjectQLEngine {
       // data door instead (`assertSearchFieldsAreSearchable`,
       // `@objectstack/metadata-protocol`), the #4254 posture — an unhonoured
       // narrowing must be a 400, not a silent widening. A direct engine caller
-      // naming one has it intersected away here, the engine's existing tolerant
-      // posture for an unknown requested search field.
+      // naming one beside a searchable field has it intersected away here, the
+      // engine's existing tolerant posture for an unknown requested search
+      // field; naming only such fields is refused below (#28120).
       // [ADR-0100] …and never a `secret` / `password` column either, by the
       // collector the credential guards read: the auto-default already skips
       // those types, and an author-declared `searchableFields` naming one is
@@ -13475,6 +13523,12 @@ export class ObjectQL implements IObjectQLEngine {
         // `displayNameField` is the deprecated alias (still honored).
         displayField: (schema as any).nameField ?? (schema as any).displayNameField,
       };
+      // [#28120] An override naming none of the fields the expansion would
+      // scan is refused, never widened back to the whole set.
+      refuseSearchOverrideNamingNoSearchableField(
+        ast.object, requestedFields, (ast as any).searchFields != null ? 'searchFields' : 'search.fields',
+        resolveSearchFieldResolution(expandOptions).allowed,
+      );
       // [#25672] …and never a column the CALLER may not query. The field-read
       // predicate guard (plugin-security) runs after this expansion and judges
       // the expanded `$or` as the caller's own predicate, so one hidden column
@@ -13503,9 +13557,9 @@ export class ObjectQL implements IObjectQLEngine {
         // set the expansion itself resolved (no second resolver), so it holds
         // for every verb that reaches here (`find` / `findOne` / `aggregate` /
         // the count) and every door (REST, dispatcher, SDK, a hook's
-        // `ctx.api`). An explicit override intersected to nothing falls back
-        // to the allowed set inside `resolveSearchFields`, so it reaches this
-        // only when that set is empty too. [#25672] A set the caller's
+        // `ctx.api`). An explicit override intersected to nothing is refused
+        // before the expansion (#28120) unless the allowed set is empty too, so
+        // it reaches this only then. [#25672] A set the caller's
         // queryable fields withheld down to nothing gets the same answer: the
         // declared set expands without the narrowing, so the refusal says so.
         const withheld = queryableFields !== undefined && expandSearchToFilter(raw, expandOptions) !== null;