Add JSON ingestion indexing extensions by jwils · Pull Request #1204 · block/elasticgraph

jwils · 2026-05-20T19:57:16Z

Why

Keep the JSON ingestion indexing extensions separate from the schema-extension wiring so the indexing behavior is easier to review.

What

Add JSON ingestion extension modules for indexing field references and field types
Make core indexing field references and enum/scalar/union field types extendable via the existing MemoizableData helper, matching the object field type
Keep JSON ingestion behavior in extension modules; no wrapper classes

Risk Assessment

Low to medium - this changes internal indexing value objects to use the existing MemoizableData wrapper so JSON ingestion can attach extension modules, without changing core mapping behavior.

References

Stacked on Namespace JSON schema helpers under JSON ingestion #1203.
bundle exec rspec elasticgraph-schema_definition/spec/unit/elastic_graph/schema_definition/json_schema_spec.rb elasticgraph-schema_definition/spec/unit/elastic_graph/schema_definition/json_schema_field_metadata_spec.rb elasticgraph-schema_definition/spec/unit/elastic_graph/schema_definition/indexing/json_schema_with_metadata_spec.rb elasticgraph-schema_definition/spec/unit/elastic_graph/schema_definition/factory_spec.rb passed.
script/type_check passed.
script/lint passed.

Stack

Current PR is marked with ->.

## Why Split the JSON ingestion extraction into reviewable steps so the gem packaging can be checked independently before code moves. ## What - Add the `elasticgraph-json_ingestion` gem shell - Wire the new path gem into the bundle without changing runtime behavior - Update generated dependency diagrams and gem READMEs for the new empty gem ## Risk Assessment Low — this only adds an empty extension gem package, lockfile wiring, and generated dependency docs. ## References - First PR in the JSON ingestion extraction stack. - Local spellcheck, lint, type-check, schema artifact, and dependency diagram checks passed. ## Stack Current PR is marked with `->`. - -> [#1199 Add JSON ingestion gem shell](#1199) - [#1200 Add schema definition extension plumbing](#1200) - [#1201 Move JSON schema helpers to json ingestion gem](#1201) - [#1203 Namespace JSON schema helpers under JSON ingestion](#1203) - [#1204 Add JSON ingestion indexing extensions](#1204) - [#1215 Wire JSON ingestion schema extension modules](#1215) - [#1205 Use JSON ingestion as schema definition extension](#1205) - [#1202 Document JSON ingestion schema APIs](#1202) - [#1224 Migrate JSON ingestion specs into gem](#1224) - [#1220 Add configurable indexing event decoder](#1220)

## Why Prepare schema definition for ingestion serializer extensions before moving JSON Schema code into its own gem. ## What - Add schema definition `extension_modules` plumbing for APIs, factories, and rake tasks - Apply duplicate extension modules only once - Replace the rake-task sidecar enforcement option with schema-level `schema.enforce_json_schema_version false` - Update project templates, tests, and docs to configure JSON schema enforcement in schema definitions ## Risk Assessment Medium — this touches schema definition setup, but JSON Schema behavior remains in `elasticgraph-schema_definition` in this PR. ## References - Stacked on #1199. - Local spellcheck, lint, type-check, targeted specs, and schema artifact checks passed. ## Stack Current PR is marked with `->`. - [#1199 Add JSON ingestion gem shell](#1199) - -> [#1200 Add schema definition extension plumbing](#1200) - [#1201 Move JSON schema helpers to json ingestion gem](#1201) - [#1203 Namespace JSON schema helpers under JSON ingestion](#1203) - [#1204 Add JSON ingestion indexing extensions](#1204) - [#1215 Wire JSON ingestion schema extension modules](#1215) - [#1205 Use JSON ingestion as schema definition extension](#1205) - [#1202 Document JSON ingestion schema APIs](#1202) - [#1224 Migrate JSON ingestion specs into gem](#1224) - [#1220 Add configurable indexing event decoder](#1220)

## Why Keep the first extraction layer reviewable by moving the JSON Schema helper files without changing their contents, so GitHub can render them as renames instead of delete/add rewrites. ## What - Move JSON Schema helper Ruby and RBS files into `elasticgraph-json_ingestion` - Update core requires/type references to load the helpers from the new gem - Keep `elasticgraph-json_ingestion` optional at runtime by using a schema_definition development dependency instead of a runtime dependency - Add the Docker/Apollo build-context wiring needed for bundles that include the new path gem explicitly ## Risk Assessment Low to medium - mostly file moves plus require/dependency wiring. ## References - Stacked on #1200. - `git diff -M --summary` reports the moved helper files as 100% renames, including `json_schema_with_metadata.rb`. - Local bundle install, lint, type check, spellcheck, dependency diagram verification, schema artifact check, and targeted unit specs passed on the restacked branch. ## Stack Current PR is marked with `->`. - [#1199 Add JSON ingestion gem shell](#1199) - [#1200 Add schema definition extension plumbing](#1200) - -> [#1201 Move JSON schema helpers to json ingestion gem](#1201) - [#1203 Namespace JSON schema helpers under JSON ingestion](#1203) - [#1204 Add JSON ingestion indexing extensions](#1204) - [#1215 Wire JSON ingestion schema extension modules](#1215) - [#1205 Use JSON ingestion as schema definition extension](#1205) - [#1202 Document JSON ingestion schema APIs](#1202) - [#1224 Migrate JSON ingestion specs into gem](#1224) - [#1220 Add configurable indexing event decoder](#1220)

## Why After the move-only PR preserves rename detection, put the moved helper constants under the `ElasticGraph::JSONIngestion` namespace. ## What - Namespace the moved JSON Schema helper classes and RBS signatures - Update schema_definition references and tests to use the new qualified constants ## Risk Assessment Medium - namespace-only behavior should be unchanged, but it updates call sites across schema_definition. ## References - Stacked on #1201. - Local lint, type check, schema artifact checks, and documentation coverage passed while building this split. ## Stack Current PR is marked with `->`. - [#1199 Add JSON ingestion gem shell](#1199) - [#1200 Add schema definition extension plumbing](#1200) - [#1201 Move JSON schema helpers to json ingestion gem](#1201) - -> [#1203 Namespace JSON schema helpers under JSON ingestion](#1203) - [#1204 Add JSON ingestion indexing extensions](#1204) - [#1215 Wire JSON ingestion schema extension modules](#1215) - [#1205 Use JSON ingestion as schema definition extension](#1205) - [#1202 Document JSON ingestion schema APIs](#1202) - [#1224 Migrate JSON ingestion specs into gem](#1224) - [#1220 Add configurable indexing event decoder](#1220)

jwils mentioned this pull request May 20, 2026

Use JSON ingestion as schema definition extension #1205

Merged

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch from 7bc0045 to 6267113 Compare May 20, 2026 20:03

jwils force-pushed the joshuaw/json-ingestion-extension-modules branch 2 times, most recently from 024c4ef to d8c3c70 Compare May 20, 2026 20:15

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch 2 times, most recently from 040cea2 to e6a4787 Compare May 21, 2026 02:49

jwils force-pushed the joshuaw/json-ingestion-extension-modules branch from d8c3c70 to b78686f Compare May 21, 2026 02:49

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch from e6a4787 to eb21230 Compare May 21, 2026 13:43

jwils force-pushed the joshuaw/json-ingestion-extension-modules branch from b78686f to 0b3844f Compare May 21, 2026 13:43

jwils marked this pull request as ready for review May 22, 2026 18:23

jwils requested review from BrianSigafoos-SQ, ayousufi, bsorbo, jwondrusch and myronmarston as code owners May 22, 2026 18:23

jwils force-pushed the joshuaw/json-ingestion-extension-modules branch from 0b3844f to 65f9a60 Compare May 22, 2026 22:41

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch from eb21230 to 2afdcd4 Compare May 22, 2026 22:41

jwils force-pushed the joshuaw/json-ingestion-extension-modules branch from 65f9a60 to 6be803f Compare May 24, 2026 13:33

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch 2 times, most recently from b03c1a4 to a3171ef Compare May 24, 2026 14:09

jwils force-pushed the joshuaw/json-ingestion-extension-modules branch 2 times, most recently from 159afa8 to 29e7e8e Compare May 24, 2026 14:24

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch 2 times, most recently from bf0dd3b to 34a576b Compare May 24, 2026 14:51

jwils force-pushed the joshuaw/json-ingestion-extension-modules branch 2 times, most recently from 31d7828 to 87c88c1 Compare May 24, 2026 15:00

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch from 34a576b to 5920f3b Compare May 24, 2026 15:00

jwils force-pushed the joshuaw/json-ingestion-extension-modules branch from 87c88c1 to e99c65f Compare May 24, 2026 15:36

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch from 5920f3b to dbaf8a1 Compare May 24, 2026 15:36

jwils force-pushed the joshuaw/json-ingestion-extension-modules branch from e99c65f to 6aea7ea Compare May 24, 2026 18:05

jwils force-pushed the joshuaw/json-ingestion-extension-modules branch 3 times, most recently from 7cd0f7d to 3d879aa Compare May 30, 2026 14:38

myronmarston reviewed May 30, 2026

View reviewed changes

jwils force-pushed the joshuaw/json-ingestion-extension-modules branch 2 times, most recently from 01d66b2 to 4d549ec Compare May 30, 2026 20:17

myronmarston approved these changes May 30, 2026

View reviewed changes

This was referenced Jun 11, 2026

Make the JSON ingestion spec extension fully opt-in #1249

Open

Remove remaining JSON schema usage from schema definition specs #1250

Closed

jwils force-pushed the joshuaw/json-ingestion-extension-modules branch from 4d549ec to d097cc0 Compare June 15, 2026 20:58

jwils requested review from ellisandrews-toast, marcdaniels-toast and rossroberts-toast as code owners June 15, 2026 20:58

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch from 2561384 to 0c1ef88 Compare June 15, 2026 20:58

myronmarston approved these changes Jun 16, 2026

View reviewed changes

jwils force-pushed the joshuaw/json-ingestion-helper-namespace branch from 0c1ef88 to 8d63094 Compare June 16, 2026 18:40

Base automatically changed from joshuaw/json-ingestion-helper-namespace to main June 16, 2026 19:04

Add JSON ingestion indexing extensions

a906b8f

jwils force-pushed the joshuaw/json-ingestion-extension-modules branch from d097cc0 to a906b8f Compare June 16, 2026 19:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add JSON ingestion indexing extensions#1204

Add JSON ingestion indexing extensions#1204
jwils merged 1 commit into
mainfrom
joshuaw/json-ingestion-extension-modules

jwils commented May 20, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

jwils commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What

Risk Assessment

References

Stack

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jwils commented May 20, 2026 •

edited

Loading