IGNITE-28520 Auto-generate message serialization / marshalling / deployment#13095
Open
anton-vinogradov wants to merge 330 commits into
Open
IGNITE-28520 Auto-generate message serialization / marshalling / deployment#13095anton-vinogradov wants to merge 330 commits into
anton-vinogradov wants to merge 330 commits into
Conversation
anton-vinogradov
force-pushed
the
ignite-28520
branch
from
May 4, 2026 16:28
cb7ec5a to
802f7e1
Compare
|
# Conflicts: # modules/codegen/src/main/java/org/apache/ignite/internal/MessageSerializerGenerator.java # modules/codegen/src/main/java/org/apache/ignite/internal/idto/IDTOSerializerGenerator.java # modules/zookeeper/src/main/java/org/apache/ignite/spi/discovery/zk/internal/ZkMessageFactory.java # modules/zookeeper/src/main/java/org/apache/ignite/spi/discovery/zk/internal/ZookeeperDiscoveryImpl.java
Contributor
Author
|
/runall |
Contributor
Author
|
Ignite PR Checker verdict · RunAll build 9216359 · 147 suites ran, 0 reused
🔍 2 suite(s) ran fewer tests than on master (tests that never ran can't fail):
✅ No test blockers otherwise; 26 pre-existing/flaky filtered out. ♻️ Settled after 2 auto re-run wave(s): #1 — 2 broken suite(s); #2 — 1 broken suite(s). |
Possible compatibility issues. Please, check rolling upgrade casesThis PR modifies protected classes (with Order annotation). Affected files:
|
Contributor
Author
|
/runall 🚀 RunAll queued — build 9220444. The verdict lands here when the run finishes. |
Contributor
Author
|
/runall 🚀 RunAll queued — build 9220592. The verdict lands here when the run finishes. |
Contributor
Author
|
Ignite PR Checker verdict · RunAll build 9220592 · 147 suites ran, 0 reused
✅ No test blockers otherwise; 67 pre-existing/flaky filtered out. ♻️ Settled after 2 auto re-run wave(s): #1 — 1 broken suite(s); #2 — 1 broken suite(s). |
Vladsz83
reviewed
Jul 22, 2026
|
|
||
| import org.apache.ignite.plugin.extensions.communication.Message; | ||
|
|
||
| public class NioFieldOnNonMessageMessage implements Message { |
Contributor
There was a problem hiding this comment.
Suggestion: NioFieldOnFieldMessage
| val.prepareMarshal(ctx); | ||
|
|
||
| /** {@inheritDoc} */ | ||
| @Override public void marshal(Marshaller marsh) throws IgniteCheckedException { |
Contributor
There was a problem hiding this comment.
Good Place to fire a 'revise' ticket. Revise certain message logic or shared message preparation/finalization.
|
|
||
| /** */ | ||
| public class CalciteErrorMessage extends ErrorMessage { | ||
| public class CalciteErrorMessage extends ErrorMessage implements DeferredUnmarshalMessage { |
Contributor
There was a problem hiding this comment.
Why it is a DeferredUnmarshalMessage?
| * | ||
| */ | ||
| public class QueryCloseMessage implements Message { | ||
| public class QueryCloseMessage implements DeferredUnmarshalMessage { |
Contributor
There was a problem hiding this comment.
Why it is a DeferredUnmarshalMessage? And many other who implements DeferredUnmarshalMessage
| public @Nullable OperationContextMessage opCtxMsg; | ||
|
|
||
| /** Set once the payload is marshalled; guards double marshal and unmarshalled transmit. Not on the wire. */ | ||
| private transient boolean marshalled; |
| // peer-deployment loader), so the response topic is restored here; it carries only internal classes. | ||
| try { | ||
| if (req.resTopicMsg != null) | ||
| MessageMarshalling.unmarshal(req.resTopicMsg, ctx); |
Contributor
There was a problem hiding this comment.
We should not manually marshall. Cat we revise the design?
|
|
||
| if (addDepInfo) | ||
| prepareObject(key, cctx); | ||
| deployObject(key, cctx); |
Contributor
There was a problem hiding this comment.
Why manually-deployed? No by a generated deployer
|
|
||
| if (preloadEntries != null) | ||
| unmarshalInfos(preloadEntries, ctx.cacheContext(cacheId), ldr); | ||
| deployInfos(preloadEntries, cctx); |
Contributor
There was a problem hiding this comment.
Why manually-deployed? Not by a generated deployer.
|
|
||
| assert added.dhtLocal(); | ||
|
|
||
| if (added.ownerVersion() != null) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Extends the annotation-driven message codegen (previously serialization-only) to
marshalling and deployment: a message's wire read/write, marshalling and
deployment are now generated from declarative field annotations, replacing
hand-written writeTo/readFrom, prepareMarshal/finishUnmarshal and prepareDeployment.
Main directions
Generated marshaller & deployer companions.
MessageProcessornow emits<Msg>Marshallerand<Msg>Deployernext to<Msg>Serializer, driven by@Marshalled/@MarshalledCollection/@MarshalledMap(with@Order/@NioField). Hand-written marshalling/deployment hooks are removed from messages.Uniform factory dispatch.
MessageFactoryresolves all three by direct type —serializer()/marshaller()/deployer()— via oneregister(directType, supplier, serializer, marshaller, deployer). Callers use thestatic facades
MessageSerializer.writeTo/readFrom,MessageMarshaller.marshal/unmarshal,GridCacheMessageDeployer.deploy; an ArchUnit rule (MessageSerializationArchitectureTest)enforces it.
Comm & discovery wired to it.
GridIoManager/GridCacheIoManager/IgniteTxManagerand the TCP/ZK discovery I/O callMessageMarshaller.marshal/unmarshal;discovery custom messages (
MetadataUpdateProposedMessage,BinaryMetadataVersionInfo, …)move from
MarshallableMessagehooks to@Marshalled.Deployment collapsed.
GridCacheMessageDeployeris now only the codegen interfaceplus the factory-resolving facade; per-field static bridges dropped,
GridCacheMessage#deploy*accessed directly.
Marshalling stays off the NIO threads. The NIO codec does only serialization
(
writeTo/readFrom); the actual marshal/unmarshal runs on the sender and workerthreads.
@NioField+unmarshalNioare the explicit, minimal exception — only thehandful of fields needed for early dispatch are unmarshalled on the NIO thread.
Side effects (mechanical — skim)
prepareMarshal→marshal,finishUnmarshal→unmarshal,finishUnmarshalNio→unmarshalNio(onMarshallableMessage/MessageMarshaller);CacheObject#prepareMarshal(ctx)/finishUnmarshal(ctx)→marshal/unmarshal;prepareDeployment+prepare*Deploymenthelpers →deploy/deploy*;MarshallerCacheFreeFinishTest→…UnmarshalTest,MessageFinishUnmarshalOnceTest→MessageUnmarshalOnceTest..mvn/jvm.configopensjdk.compilerfor the codegen tests(Google compile-testing);
commit-check.ymlnow surfaces the real compile errorbehind the generated-class "cannot find symbol" cascade.
Performance
JMH-verified: serialization hot path unchanged (Δ ≈ 0 vs master); marshalling adds
~1.8 ns/message of factory dispatch (<0.5% of the actual
U.marshal, ~µs); zero extraallocations.