feat: implement gossipsub partial messages extension #577
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…sipsub-partial-messages
MarcoPolo
reviewed
Sep 16, 2025
protocols/gossipsub/src/behaviour.rs
Outdated
| { | ||
| // Return err if trying to publish the same partial message state we currently have. | ||
| if existing.available_parts() == partial_message.available_parts() { | ||
| return Err(PublishError::Duplicate); |
There was a problem hiding this comment.
I don't think this is correct.
- Imagine you have parts 1,2,3.
- You tell your peers about those parts.
- A peer comes back and says I want part 2.
- You republish with the same parts in order to respond.
- You get this error and fail to respond.
Member
Author
There was a problem hiding this comment.
thanks for explaining Marco, updated as we spoke
protocols/gossipsub/src/behaviour.rs
Outdated
| data_transform: D, | ||
|
|
||
| /// Partial messages received. | ||
| partial_messages: HashMap<TopicHash, HashMap<Vec<u8>, P>>, |
There was a problem hiding this comment.
Do you need to store P here? I think it's better if P is owned solely by the application.
Member
Author
There was a problem hiding this comment.
yup, you are right, thanks Marco!
protocols/gossipsub/src/types.rs
Outdated
| pub(crate) struct PartialData { | ||
| pub(crate) ihave: Vec<u8>, | ||
| pub(crate) iwant: Vec<u8>, | ||
| pub(crate) message: Vec<u8>, |
There was a problem hiding this comment.
Is it useful to store the message here? It seems like wasted space, you only use it to check if the peer is sending you a duplicate.
Might be simpler to let the application handle dupes.
Member
Author
There was a problem hiding this comment.
thanks Marco, I only left wanted and has wanted to avoid sending the same message and has to avoid notifying the application layer of duplicates.
Thanks!
jxs
force-pushed
the
gossipsub-partial-messages
branch
5 times, most recently
from
cb0e925 to
e6f1ae4
Compare
jxs
force-pushed
the
gossipsub-partial-messages
branch
from
e6f1ae4 to
69c2d95
Compare
and create a paralel list for the partial_only topics.
Pull-Request: libp2p#6171.
Pull-Request: libp2p#6198.
will address the intra doc links issue on a subsequent PR, the remaining [issue](https://github.com/libp2p/rust-libp2p/actions/runs/19040315461/job/54375441096?pr=6195#step:6:760) seems like a bug, opened rust-lang/rust-clippy#16023. Pull-Request: libp2p#6195.
The rustwasm organization was archived this year. Projects have moved to new URLs to reflect that. This PR just updates those URLs. See https://blog.rust-lang.org/inside-rust/2025/07/21/sunsetting-the-rustwasm-github-org/ Pull-Request: libp2p#6199.
From libp2p#2217. Renaming UDS Config symbol from `UdsConfig` in the doc comment and from `TokioUdsConfig`, for better referencing when re-exported. Left deprecated alias for backward compatibility. Pull-Request: libp2p#6190.
This reverts commit aa94317.
This reverts commit a3ba227.
This removes a `debug_assert` from `poll_flush` because it can be valid for the user to recheck the flush state. In that case the operation should be noop. A real life code that triggers this assertition is [this](https://github.com/libp2p/rust-asynchronous-codec/blob/c818a83906891caf8aadcae8f899727c2c8393a8/src/framed_write.rs#L263-L266). Fixes libp2p#5618 Pull-Request: libp2p#6193.
Adds a fast path in `remove_data_messages` to avoid unnecessary work when the caller provides an empty `message_ids` slice. Pull-Request: libp2p#6208.
Pull-Request: libp2p#6219.
jxs
force-pushed
the
gossipsub-partial-messages
branch
2 times, most recently
from
927d36c to
ac9f618
Compare
This change improves peer removal during `heartbeat` by switching from a two-pass remove logic to an in-place `retain` with very small size intermediate variable `removed_peers_count`. Pull-Request: libp2p#6209.
it's not required for the `peerid` feature Pull-Request: libp2p#6226.
Signed-off-by: Marco Munizaga <git@marcopolo.io>
jxs
force-pushed
the
gossipsub-partial-messages
branch
from
33d825a to
5673256
Compare
mergify bot
pushed a commit
to libp2p/rust-libp2p
that referenced
this pull request
Dec 18, 2025
Improve gossipsub behavior tests structure. Split the already big enough file into multiple test files according to their `Behaviour` tests category. Document the shared test primitives and rename structures, `InjectNodes` to `BehaviourTestBuilder` and remove the `inject_nodes` functions to use `Default`. This will help with the new tests for partial messages on sigp#577 which will be eventually moved here Pull-Request: #6232.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This is a draft implementation of partial messages for gossipsub following the spec PR and based on the Go implementation. Still WIP but should give a good idea of the direction we're heading.