Fix delivery report callback: librdkafka passes the message by pointer - #8
Open
stellarthemes wants to merge 1 commit into
Open
Fix delivery report callback: librdkafka passes the message by pointer#8stellarthemes wants to merge 1 commit into
stellarthemes wants to merge 1 commit into
Conversation
The dr_msg_cb binding declared the callback as taking Message by value, but librdkafka invokes it with `const rd_kafka_message_t *`. On x86-64 Linux the by-value declaration makes the callback read stack memory the caller never wrote, so every field is garbage (issue BT-OpenSource#2's garbled payload). On Apple Silicon a large by-value struct is passed by reference, which happens to line up with the pointer librdkafka actually passes - which is why the bug only shows on Linux. Also build the payload string with the message length (the payload is not NUL-terminated) and log the error string when err is set, since librdkafka reuses the payload field for it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Author
|
do i need to do anything to get this merged? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
librdkafka invokes the delivery report callback as
but the binding declares the callback as taking
Messageby value:On x86-64 Linux, a struct this size passed by value is expected in stack memory, so the callback ends up reading memory the caller never wrote — every field in the message is garbage. That's the garbled payload in #2.
On Apple Silicon, a large by-value struct is passed by reference instead, which happens to line up exactly with librdkafka passing a pointer. So the same code looks fine on a Mac — which is why #2 only reproduces on Linux servers.
Changes:
Message*and reads fields through.valuelen— the payload is not NUL-terminated, soString.new(pointer)could over-read even with the calling convention fixederris set, log at error level instead — librdkafka reuses the payload field for the error string in that caseChecked the struct layout against librdkafka 1.8 headers; unit specs pass. Note this changes the callback type, so anyone who registered their own delivery callback with the old signature will need the same one-line change — probably worth a version bump.
Fixes #2.