Namespace JSON schema helpers under JSON ingestion by jwils · Pull Request #1203 · block/elasticgraph

jwils · 2026-05-20T19:57:06Z

Why

After the move-only PR preserves rename detection, put the moved helper constants under the ElasticGraph::JSONIngestion namespace.

What

Namespace the moved JSON Schema helper classes and RBS signatures
Update schema_definition references and tests to use the new qualified constants

Risk Assessment

Medium - namespace-only behavior should be unchanged, but it updates call sites across schema_definition.

References

Stacked on Move JSON schema helpers to json ingestion gem #1201.
Local lint, type check, schema artifact checks, and documentation coverage passed while building this split.

Stack

Current PR is marked with ->.

## Why Split the JSON ingestion extraction into reviewable steps so the gem packaging can be checked independently before code moves. ## What - Add the `elasticgraph-json_ingestion` gem shell - Wire the new path gem into the bundle without changing runtime behavior - Update generated dependency diagrams and gem READMEs for the new empty gem ## Risk Assessment Low — this only adds an empty extension gem package, lockfile wiring, and generated dependency docs. ## References - First PR in the JSON ingestion extraction stack. - Local spellcheck, lint, type-check, schema artifact, and dependency diagram checks passed. ## Stack Current PR is marked with `->`. - -> [#1199 Add JSON ingestion gem shell](#1199) - [#1200 Add schema definition extension plumbing](#1200) - [#1201 Move JSON schema helpers to json ingestion gem](#1201) - [#1203 Namespace JSON schema helpers under JSON ingestion](#1203) - [#1204 Add JSON ingestion indexing extensions](#1204) - [#1215 Wire JSON ingestion schema extension modules](#1215) - [#1205 Use JSON ingestion as schema definition extension](#1205) - [#1202 Document JSON ingestion schema APIs](#1202) - [#1224 Migrate JSON ingestion specs into gem](#1224) - [#1220 Add configurable indexing event decoder](#1220)

## Why Prepare schema definition for ingestion serializer extensions before moving JSON Schema code into its own gem. ## What - Add schema definition `extension_modules` plumbing for APIs, factories, and rake tasks - Apply duplicate extension modules only once - Replace the rake-task sidecar enforcement option with schema-level `schema.enforce_json_schema_version false` - Update project templates, tests, and docs to configure JSON schema enforcement in schema definitions ## Risk Assessment Medium — this touches schema definition setup, but JSON Schema behavior remains in `elasticgraph-schema_definition` in this PR. ## References - Stacked on #1199. - Local spellcheck, lint, type-check, targeted specs, and schema artifact checks passed. ## Stack Current PR is marked with `->`. - [#1199 Add JSON ingestion gem shell](#1199) - -> [#1200 Add schema definition extension plumbing](#1200) - [#1201 Move JSON schema helpers to json ingestion gem](#1201) - [#1203 Namespace JSON schema helpers under JSON ingestion](#1203) - [#1204 Add JSON ingestion indexing extensions](#1204) - [#1215 Wire JSON ingestion schema extension modules](#1215) - [#1205 Use JSON ingestion as schema definition extension](#1205) - [#1202 Document JSON ingestion schema APIs](#1202) - [#1224 Migrate JSON ingestion specs into gem](#1224) - [#1220 Add configurable indexing event decoder](#1220)

## Why Keep the first extraction layer reviewable by moving the JSON Schema helper files without changing their contents, so GitHub can render them as renames instead of delete/add rewrites. ## What - Move JSON Schema helper Ruby and RBS files into `elasticgraph-json_ingestion` - Update core requires/type references to load the helpers from the new gem - Keep `elasticgraph-json_ingestion` optional at runtime by using a schema_definition development dependency instead of a runtime dependency - Add the Docker/Apollo build-context wiring needed for bundles that include the new path gem explicitly ## Risk Assessment Low to medium - mostly file moves plus require/dependency wiring. ## References - Stacked on #1200. - `git diff -M --summary` reports the moved helper files as 100% renames, including `json_schema_with_metadata.rb`. - Local bundle install, lint, type check, spellcheck, dependency diagram verification, schema artifact check, and targeted unit specs passed on the restacked branch. ## Stack Current PR is marked with `->`. - [#1199 Add JSON ingestion gem shell](#1199) - [#1200 Add schema definition extension plumbing](#1200) - -> [#1201 Move JSON schema helpers to json ingestion gem](#1201) - [#1203 Namespace JSON schema helpers under JSON ingestion](#1203) - [#1204 Add JSON ingestion indexing extensions](#1204) - [#1215 Wire JSON ingestion schema extension modules](#1215) - [#1205 Use JSON ingestion as schema definition extension](#1205) - [#1202 Document JSON ingestion schema APIs](#1202) - [#1224 Migrate JSON ingestion specs into gem](#1224) - [#1220 Add configurable indexing event decoder](#1220)

## Why Keep the JSON ingestion indexing extensions separate from the schema-extension wiring so the indexing behavior is easier to review. ## What - Add JSON ingestion extension modules for indexing field references and field types - Make core indexing field references and enum/scalar/union field types extendable via the existing MemoizableData helper, matching the object field type - Keep JSON ingestion behavior in extension modules; no wrapper classes ## Risk Assessment Low to medium - this changes internal indexing value objects to use the existing MemoizableData wrapper so JSON ingestion can attach extension modules, without changing core mapping behavior. ## References - Stacked on #1203. - `bundle exec rspec elasticgraph-schema_definition/spec/unit/elastic_graph/schema_definition/json_schema_spec.rb elasticgraph-schema_definition/spec/unit/elastic_graph/schema_definition/json_schema_field_metadata_spec.rb elasticgraph-schema_definition/spec/unit/elastic_graph/schema_definition/indexing/json_schema_with_metadata_spec.rb elasticgraph-schema_definition/spec/unit/elastic_graph/schema_definition/factory_spec.rb` passed. - `script/type_check` passed. - `script/lint` passed. ## Stack Current PR is marked with `->`. - [#1199 Add JSON ingestion gem shell](#1199) - [#1200 Add schema definition extension plumbing](#1200) - [#1201 Move JSON schema helpers to json ingestion gem](#1201) - [#1203 Namespace JSON schema helpers under JSON ingestion](#1203) - -> [#1204 Add JSON ingestion indexing extensions](#1204) - [#1215 Wire JSON ingestion schema extension modules](#1215) - [#1205 Use JSON ingestion as schema definition extension](#1205) - [#1202 Document JSON ingestion schema APIs](#1202) - [#1224 Migrate JSON ingestion specs into gem](#1224) - [#1220 Add configurable indexing event decoder](#1220)

## Why Introduce the JSON ingestion schema-definition extension modules after the indexing extension points exist, but before making JSON ingestion the default implementation. ## What - Add `ElasticGraph::JSONIngestion::SchemaDefinition::APIExtension` and supporting factory/results/artifact/state/schema-element extension modules - Wire JSON ingestion through the factory so core indexing field references and field types are extended in place - Keep the existing core JSON Schema behavior active in this intermediate layer - Add doctest support for JSON ingestion schema-definition examples without making the extension the default yet ## Risk Assessment Medium - this adds new extension code, but the default behavior remains the existing core JSON Schema implementation in this PR. ## References - Stacked on #1204. - `bundle exec rspec elasticgraph-schema_definition/spec/unit/elastic_graph/schema_definition/json_schema_spec.rb elasticgraph-schema_definition/spec/unit/elastic_graph/schema_definition/json_schema_field_metadata_spec.rb elasticgraph-schema_definition/spec/unit/elastic_graph/schema_definition/indexing/json_schema_with_metadata_spec.rb elasticgraph-schema_definition/spec/unit/elastic_graph/schema_definition/factory_spec.rb` passed. - `script/type_check` passed. - `script/lint` passed. ## Stack Current PR is marked with `->`. - [#1199 Add JSON ingestion gem shell](#1199) - [#1200 Add schema definition extension plumbing](#1200) - [#1201 Move JSON schema helpers to json ingestion gem](#1201) - [#1203 Namespace JSON schema helpers under JSON ingestion](#1203) - [#1204 Add JSON ingestion indexing extensions](#1204) - -> [#1215 Wire JSON ingestion schema extension modules](#1215) - [#1205 Use JSON ingestion as schema definition extension](#1205) - [#1202 Document JSON ingestion schema APIs](#1202) - [#1224 Migrate JSON ingestion specs into gem](#1224) - [#1220 Add configurable indexing event decoder](#1220)

## Why Complete the extraction by loading JSON ingestion through the schema-definition extension API and removing the duplicated core JSON Schema implementation. ## What - Wire `ElasticGraph::JSONIngestion::SchemaDefinition::APIExtension` into app, template, doctest, and spec-helper schema definition setup - Move JSON Schema artifact/results behavior out of core schema_definition - Remove JSON Schema field/type/index logic from core classes now provided by the extension - Keep JSON schema version enforcement configurable through the JSON ingestion extension - Update the generated JSON ingestion dependency diagram ## Risk Assessment Medium to high - this is the behavior-transfer layer, but it sits after the move/namespace/wrapper/wiring PRs so the remaining diff is mostly deletion and wiring. ## References - Stacked on #1215. - Local `script/quick_build` passed before the generated diagram update; `script/update_dependency_diagrams --verify` and README spellcheck passed after it. ## Stack Current PR is marked with `->`. - [#1199 Add JSON ingestion gem shell](#1199) - [#1200 Add schema definition extension plumbing](#1200) - [#1201 Move JSON schema helpers to json ingestion gem](#1201) - [#1203 Namespace JSON schema helpers under JSON ingestion](#1203) - [#1204 Add JSON ingestion indexing extensions](#1204) - [#1215 Wire JSON ingestion schema extension modules](#1215) - -> [#1205 Use JSON ingestion as schema definition extension](#1205) - [#1202 Document JSON ingestion schema APIs](#1202) - [#1224 Migrate JSON ingestion specs into gem](#1224) - [#1220 Add configurable indexing event decoder](#1220)

## Why Document the JSON ingestion schema-definition APIs after extracting JSON ingestion into its own extension and making it part of generated projects by default. ## What - Document the default JSON ingestion setup and explicit extension-module wiring - Document `json_schema_version`, `json_schema_strictness`, scalar JSON Schema metadata, field/type validations, and field-level `nullable: false` ## Risk Assessment Low - README-only documentation change. Snippets are validated. ## References - Stacked on #1205. - Local `script/quick_build` passed before the generated diagram restack; `script/update_dependency_diagrams --verify` and README spellcheck passed after it. ## Stack Current PR is marked with `->`. - [#1199 Add JSON ingestion gem shell](#1199) - [#1200 Add schema definition extension plumbing](#1200) - [#1201 Move JSON schema helpers to json ingestion gem](#1201) - [#1203 Namespace JSON schema helpers under JSON ingestion](#1203) - [#1204 Add JSON ingestion indexing extensions](#1204) - [#1215 Wire JSON ingestion schema extension modules](#1215) - [#1205 Use JSON ingestion as schema definition extension](#1205) - -> [#1202 Document JSON ingestion schema APIs](#1202) - [#1224 Migrate JSON ingestion specs into gem](#1224) - [#1220 Add configurable indexing event decoder](#1220)

## Why The JSON ingestion gem should own the JSON-schema tests that cover its extracted implementation so the gem can enforce full coverage without a SimpleCov skip. ## What - Move JSON schema generation, metadata merge, pruner, and matcher specs into `elasticgraph-json_ingestion`. - Add JSON-ingestion artifact-manager integration coverage for versioned schema artifacts, version-bump enforcement, metadata refresh, and schema evolution diagnostics. - Add focused wrapper/scalar specs for the remaining extracted JSON-ingestion paths. - Remove the temporary SimpleCov filter for `elasticgraph-json_ingestion`. ## Verification - `script/run_gem_specs elasticgraph-json_ingestion` - `script/type_check elasticgraph-json_ingestion` - `BUNDLE_GEMFILE=Gemfile bundle exec rspec --format progress spec/integration/elastic_graph/schema_definition/rake_tasks_spec.rb spec/unit/elastic_graph/schema_definition/runtime_metadata/scalar_types_by_name_spec.rb` from `elasticgraph-schema_definition/` - `bundle exec standardrb ...` on touched specs/support `script/run_gem_specs elasticgraph-schema_definition` and `script/run_gem_specs elasticgraph-json_ingestion` both pass with 100% coverage (with the test datastore booted). ## Follow-ups - The migrated spec files keep the compact `module JSONIngestion::SchemaDefinition` form so that git/GitHub detects them as file moves. A follow-up PR (after this stack merges) will convert them to the standard nested module style and re-indent. - The JSON-schema-versioning scenarios that the new `schema_artifact_manager_extension_spec.rb` covers have been removed from the core `rake_tasks_spec.rb`; the core lines they exercised are now covered by focused unit specs instead. ## Stack Current PR is marked with `->`. - [#1199 Add JSON ingestion gem shell](#1199) - [#1200 Add schema definition extension plumbing](#1200) - [#1201 Move JSON schema helpers to json ingestion gem](#1201) - [#1203 Namespace JSON schema helpers under JSON ingestion](#1203) - [#1204 Add JSON ingestion indexing extensions](#1204) - [#1215 Wire JSON ingestion schema extension modules](#1215) - [#1205 Use JSON ingestion as schema definition extension](#1205) - [#1202 Document JSON ingestion schema APIs](#1202) - -> [#1224 Migrate JSON ingestion specs into gem](#1224) - [#1220 Add configurable indexing event decoder](#1220)

jwils mentioned this pull request May 20, 2026

Add JSON ingestion indexing extensions #1204

Merged

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch from 7bc0045 to 6267113 Compare May 20, 2026 20:03

jwils force-pushed the joshuaw/json-ingestion-extension branch from 461ba24 to 64ff2af Compare May 20, 2026 20:03

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch from 6267113 to 040cea2 Compare May 20, 2026 20:15

myronmarston reviewed May 20, 2026

View reviewed changes

Comment thread ...json_ingestion/lib/elastic_graph/json_ingestion/schema_definition/indexing/event_envelope.rb Outdated

myronmarston reviewed May 20, 2026

View reviewed changes

Comment thread ...json_ingestion/lib/elastic_graph/json_ingestion/schema_definition/indexing/event_envelope.rb

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch from 040cea2 to e6a4787 Compare May 21, 2026 02:49

jwils force-pushed the joshuaw/json-ingestion-extension branch from 64ff2af to 7d4becc Compare May 21, 2026 02:49

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch from e6a4787 to eb21230 Compare May 21, 2026 13:43

jwils marked this pull request as ready for review May 22, 2026 18:23

jwils requested review from BrianSigafoos-SQ, ayousufi, bsorbo and jwondrusch as code owners May 22, 2026 18:23

myronmarston reviewed May 22, 2026

View reviewed changes

jwils force-pushed the joshuaw/json-ingestion-extension branch from 7d4becc to e64b589 Compare May 22, 2026 22:41

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch 2 times, most recently from 2afdcd4 to b03c1a4 Compare May 24, 2026 13:33

jwils force-pushed the joshuaw/json-ingestion-extension branch from e64b589 to 13a3306 Compare May 24, 2026 13:33

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch from b03c1a4 to a3171ef Compare May 24, 2026 14:09

jwils force-pushed the joshuaw/json-ingestion-extension branch from 13a3306 to a43c868 Compare May 24, 2026 14:09

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch from a3171ef to bf0dd3b Compare May 24, 2026 14:24

jwils force-pushed the joshuaw/json-ingestion-extension branch 2 times, most recently from 8956645 to be6a891 Compare May 24, 2026 14:51

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch from bf0dd3b to 34a576b Compare May 24, 2026 14:51

jwils force-pushed the joshuaw/json-ingestion-extension branch from be6a891 to c3c9a96 Compare May 24, 2026 15:00

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch 2 times, most recently from 5920f3b to dbaf8a1 Compare May 24, 2026 15:36

jwils force-pushed the joshuaw/json-ingestion-extension branch from c3c9a96 to 2417f3e Compare May 24, 2026 23:47

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch from dbaf8a1 to 00faf9c Compare May 24, 2026 23:47

This was referenced Jun 11, 2026

Make the JSON ingestion spec extension fully opt-in #1249

Open

Remove remaining JSON schema usage from schema definition specs #1250

Closed

jwils force-pushed the joshuaw/json-ingestion-extension branch from 1bc3fca to 31c849f Compare June 15, 2026 20:58

jwils requested review from ellisandrews-toast, marcdaniels-toast and rossroberts-toast as code owners June 15, 2026 20:58

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch from 2561384 to 0c1ef88 Compare June 15, 2026 20:59

myronmarston approved these changes Jun 16, 2026

View reviewed changes

jwils force-pushed the joshuaw/json-ingestion-extension branch from 31c849f to 2305606 Compare June 16, 2026 18:05

Base automatically changed from joshuaw/json-ingestion-extension to main June 16, 2026 18:27

Namespace JSON schema helpers under JSON ingestion

8d63094

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch from 0c1ef88 to 8d63094 Compare June 16, 2026 18:40

jwils enabled auto-merge (squash) June 16, 2026 18:40

jwils merged commit 5e75e96 into main Jun 16, 2026
25 checks passed

jwils deleted the joshuaw/json-ingestion-helper-namespace branch June 16, 2026 19:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Namespace JSON schema helpers under JSON ingestion#1203

Namespace JSON schema helpers under JSON ingestion#1203
jwils merged 1 commit into
mainfrom
joshuaw/json-ingestion-helper-namespace

jwils commented May 20, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

jwils commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What

Risk Assessment

References

Stack

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jwils commented May 20, 2026 •

edited

Loading