Take RTDE data types from the robot's setup acknowledgement - #560
Draft
urrsk wants to merge 18 commits into
Draft
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #560 +/- ##
==========================================
+ Coverage 80.78% 81.22% +0.44%
==========================================
Files 117 117
Lines 7146 7378 +232
Branches 3201 3298 +97
==========================================
+ Hits 5773 5993 +220
- Misses 987 1002 +15
+ Partials 386 383 -3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. |
There was a problem hiding this comment.
Pull request overview
Moves RTDE field typing from a static table to robot setup acknowledgements while preserving preallocated data paths.
Changes:
- Adds negotiated
DataTypehandling across packages, parsers, clients, and writers. - Adds fake-server, behavior, and allocation tests.
- Fixes
TCPServer::writeUnchecked()initialization.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
tests/test_rtde_writer.cpp |
Tests negotiated writer types and validation. |
tests/test_rtde_parser.cpp |
Tests typed parser behavior. |
tests/test_rtde_data_package.cpp |
Tests dynamic field typing. |
tests/test_rtde_client.cpp |
Updates invalid-recipe expectations. |
tests/test_rtde_client_fake_server.cpp |
Adds robot-free client tests. |
tests/test_rtde_allocations.cpp |
Adds real-time allocation checks. |
tests/test_producer.cpp |
Supplies negotiated parser types. |
tests/test_pipeline.cpp |
Supplies negotiated pipeline types. |
tests/rtde_test_helpers.h |
Adds test access to internal typing APIs. |
tests/resources/generate_rtde_outputs.py |
Sources fields from the fake server. |
tests/fake_rtde_server.cpp |
Emulates recipe type acknowledgements. |
tests/CMakeLists.txt |
Registers new test executables. |
src/rtde/rtde_writer.cpp |
Validates and merges typed input packages. |
src/rtde/rtde_parser.cpp |
Parses using negotiated output types. |
src/rtde/rtde_client.cpp |
Propagates acknowledged recipe types. |
src/rtde/data_package.cpp |
Implements dynamic RTDE typing. |
src/comm/tcp_server.cpp |
Initializes the write counter. |
include/ur_client_library/rtde/rtde_writer.h |
Declares writer type setup. |
include/ur_client_library/rtde/rtde_parser.h |
Declares parser type setup. |
include/ur_client_library/rtde/data_package.h |
Adds DataType and untyped storage. |
examples/rtde_client.cpp |
Removes obsolete string-allocation guidance. |
doc/migration_notes.rst |
Documents behavioral changes. |
doc/examples/rtde_client.rst |
Updates real-time usage guidance. |
doc/architecture/rtde_client.rst |
Documents negotiated typing architecture. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 24 out of 24 changed files in this pull request and generated 2 comments.
Suppressed comments (1)
include/ur_client_library/rtde/data_package.h:124
std::stringis not one of the RTDE protocol types, but retaining it here letssetData()change an untyped field frommonostatetostd::string. Such a package then passesisTyped()andserializePackage()emits a malformed variable-length payload, whilegetDataType()reports no type. Restrict untyped fields to the alternatives represented byDataType.
using _rtde_type_variant = std::variant<std::monostate, bool, uint8_t, uint32_t, uint64_t, int32_t, double,
vector3d_t, vector6d_t, vector6int32_t, vector6uint32_t, std::string>;
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 27 out of 27 changed files in this pull request and generated 2 comments.
Suppressed comments (3)
Previously missed (1) — in code that hasn't changed since the last review.
tests/test_rtde_allocations.cpp:52
- This thread-local design excludes not only the fake server but also the client's background-reader and writer threads. Therefore
background_receive_does_not_allocatedoes not observe the thread that parses received packages, andsending_input_data_does_not_allocatedoes not observe the thread that serializes and writes them; both tests can pass with allocations in the paths their names claim to cover. Instrument all client-owned threads while excluding only the server thread.
// Counting is per-thread: the fake server and, in the background-read case, the client's read
// thread run in the same process, and their allocations are none of this test's business.
thread_local std::size_t g_allocation_count = 0;
thread_local bool g_count_allocations = false;
src/rtde/rtde_parser.cpp:161
- A caller can make a preallocated package “typed” with
setData()before the first blocking receive, so this check does not prove that its types came from the robot. For example, a timestamp-only package set asuint64_tskipsinitEmpty(recipe_types_)and parses the robot's DOUBLE bytes as an integer while reporting success. Validate both the recipe and every existing field type againstrecipe_/recipe_types_, retyping or replacing packages that do not match.
if (!data_package->isTyped())
{
// A package built from a recipe alone doesn't know its field types yet. Applying the ones
// the robot reported doesn't allocate, so this happens right here rather than by handing
// the caller a replacement package.
tests/test_rtde_data_package.cpp:460
- This test does not measure allocations, so an allocation introduced inside
initEmpty(types)would still pass despite the test name and the PR's real-time guarantee. Surround this call with the allocation counter (or move this case into the allocation-test binary) and assert that its count remains zero.
package.initEmpty({ "DOUBLE", "VECTOR6D" });
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.
Suppressed comments (3)
Previously missed (1) — in code that hasn't changed since the last review.
tests/test_rtde_allocations.cpp:277
- The counter is thread-local, while
sendPackage()only queues data andRTDEWriter::run()performs serialization and socket writing on its own thread. Consequently this measured block cannot see allocations in the actual asynchronous send path, sosending_input_data_does_not_allocatecan pass despite a send-thread regression. Add writer-thread instrumentation or a same-thread serialization allocation test.
{
AllocationCounter counter;
for (int i = 0; i < g_MEASURED_CYCLES; ++i)
{
all_sent &= input_pkg.setData("speed_slider_fraction", 0.5);
all_sent &= client_->getWriter().sendPackage(input_pkg);
}
src/rtde/rtde_parser.cpp:157
isTyped()also becomes true when the caller has populated every field withsetData(), so it does not prove these are the negotiated types. A preallocated one-fieldtimestamppackage set asuint64_t, for example, skipsinitEmpty(recipe_types_)and parses the robot's DOUBLE bytes as UINT64. This path also never verifies field names, so an untyped same-length package with a different recipe is typed by position. Validate the recipe and always reapply the acknowledged types before parsing; replace the package only when its recipe differs.
DataPackage* data_package = dynamic_cast<DataPackage*>(result.get());
data_package->setProtocolVersion(protocol_version_);
if (!data_package->isTyped())
tests/test_rtde_data_package.cpp:460
- This test never observes allocations: it only verifies that the package remains usable. The counters in
test_rtde_allocations.cppstart afterRTDEClient::init()and warmup, so an allocation added toinitEmpty(types)would pass the suite even though no-allocation in-place typing is a central guarantee. Measure this call while allocation counting is active.
TEST(rtde_data_package, applying_types_does_not_reallocate)
{
std::vector<std::string> recipe{ "timestamp", "actual_q" };
test::TestableDataPackage package(recipe);
double timestamp = 0.0;
ASSERT_FALSE(package.getData("timestamp", timestamp));
package.initEmpty({ "DOUBLE", "VECTOR6D" });
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 27 out of 27 changed files in this pull request and generated no new comments.
Suppressed comments (2)
Previously missed (2) — in code that hasn't changed since the last review.
tests/test_rtde_allocations.cpp:184
- This guard can pass while the allocation tests miss the allocations they are intended to detect. As the comment above notes, some libstdc++/musl
std::allocatorimplementations callmallocdirectly; invoking::operator newhere proves only that this replacement works, while vector/string growth in the measured RTDE paths can bypass it and leave the count at zero. Validate the counter with a representative standard-container allocation and either intercept that platform's allocation path or fail/skip when it cannot be observed.
// Guards the tests below: if the counter stopped seeing allocations, they would pass vacuously.
// Call operator new directly rather than writing `new int`: a new-expression may be omitted even
// when the pointer escapes, which is what Alpine's gcc 15 does at -O2. Allocate with operator new
// rather than a container: on some libstdc++ / musl builds std::allocator uses malloc and would
// never hit the replaced operator new that the RTDE tests count.
TEST(AllocationCounterTest, counts_allocations)
{
std::size_t allocations = 0;
{
AllocationCounter counter;
g_allocation_sink = ::operator new(sizeof(int));
src/rtde/data_package.cpp:212
getDataType()does not necessarily report a robot-acknowledged type as the new API promises. On an application-created input package,setData()changes the variant frommonostateto the caller's type, so this function then returns that inferred type—even whenRTDEWriter::sendPackage()later rejects it because the robot reported a different type. Consumers therefore cannot tell whether this result is authoritative. Track the acknowledged type separately from the value/inferred type, or explicitly expose this as the stored value type and provide the robot-reported type through the client/writer API.
std::optional<rtde_interface::DataType> rtde_interface::DataPackage::getDataType(const std::string_view name) const
{
const auto it =
std::find_if(data_.begin(), data_.end(), [&name](const std::pair<std::string, _rtde_type_variant>& element) {
return element.first == name;
});
if (it == data_.end())
{
return std::nullopt;
}
return typeOf(it->second);
There was a problem hiding this comment.
🟡 Changes recommended
Exception safety, real-time allocation guarantees, and direct protocol-v2 parsing compatibility remain unresolved.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 32/32 changed files
- Comments generated: 3
- Review effort level: Balanced
There was a problem hiding this comment.
🟡 Changes recommended
The new type-reporting contract is inconsistent with caller-established types, and reconnect tests contain synchronization and coverage defects.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (2)
Previously missed (2) — in code that hasn't changed since the last review.
examples/rtde_writer.cpp:93
- Correct the typo in this user-facing example comment: “may” should be “many.”
tests/test_rtde_client_reconnect.cpp:320 - These two point-in-time state checks do not establish that reconnecting stopped. During each retry the client repeatedly returns to
UNINITIALIZED, so an implementation that retries forever can satisfy both assertions. Wait for the expected request count and then verifyrequestedProtocolVersions()remains unchanged (or expose a completion signal) to cover exhaustion rather than an incidental state between attempts.
- Files reviewed: 32/32 changed files
- Comments generated: 3
- Review effort level: Balanced
DataPackage carried a hand-maintained table of every subscribable field name and its data type. Every field the controller gained had to be added to it, and a field missing from it could not be used even though the robot would happily serve it. The robot already reports the data type of every field it acknowledges, so the table was a second source of truth for information we were being told anyway. The library now keeps only the list of data types the protocol defines and takes the per-field types from the setup acknowledgement. Applications do not have to change. A DataPackage is still constructed from a recipe, that is still the only place it allocates, and the robot's answer is applied to it afterwards in place. Because every RTDE type is trivially copyable and lives inline in the variant, changing which type a field holds touches no memory, so a package can be created before the connection exists and still be used in a real-time loop. tests/test_rtde_allocations.cpp pins this down by counting allocations across the receive and send paths. - Add DataPackage::getDataType(), reporting the type the robot gave a field so it does not have to be hardcoded, as an rtde_interface::DataType with toString() for the protocol's name. This is the only entry point the change adds; applying types, relaying them to the parser and writer, and asking whether a package is ready are all internal. - Remove DataPackage::g_type_list. - A field name the robot does not know is now reported by RTDEClient::init() instead of the constructor, still as an RTDEInvalidKeyException, since it is the robot that decides which names exist. - RTDEWriter::sendPackage() validates the package it is given against the robot's types, naming the field and both types on a mismatch, and sends unset fields as zeros. setData() can no longer catch a wrong type on its own because it is what establishes an input field's type. - getData() returns false rather than throwing std::bad_variant_access on a type mismatch, and takes the field name as a string_view so passing a literal does not allocate. Also fixes comm::TCPServer::writeUnchecked() reading its written out-parameter before assigning it, which made the fake RTDE server silently send nothing and is why the new robot-free tests could not run before. See doc/migration_notes.rst for the three behavioural differences.
A thrown init() left the client INITIALIZING so a retry reported success, packages kept protocol v2 after a v1 fallback, and sendPackage() cleared the store buffer before validation. Disconnect on those exceptions, apply the negotiated version when typing, and validate input first.
A failed init, a refused start or pause, and a handshake that never completes were only exercised on the success path; the fake server can now refuse those replies so those failures stay failures.
Clang rejects deleting RTDEServer through unique_ptr unless the destructor is virtual, and Alpine's allocator never hits the replaced operator new from vector::resize, so the smoke test now allocates with new.
…n allocating type cannot be reintroduced.
…ocation tests honest. The docs-vs-copy comparison never asked a robot whether it accepts the names. The allocation guard was compiled away on Alpine and aborted under ASAN, so the zero-allocation assertions could pass without measuring. The parser now always re-applies the robot's types rather than trusting isTyped() after setData().
…ite burst is not dropped.
Each data package now carries an FNV-1a hash of its field names and of each field's type, and stores its values contiguously. sendPackage() is therefore a hash compare plus a memcpy of the whole recipe, rather than a per-field walk that allocated a fresh package every cycle, and copying became a deterministic full overwrite instead of merging in only the fields the caller had set. The parser recognises a pre-allocated package that already has the negotiated layout and parses straight into it, allocating a correctly typed one only otherwise. Looking a field up by name goes through a name-to-index map instead of a linear scan of the recipe.
setProtocolVersion() was already public on both, and the two are set from the same handshake, so keeping only setRecipeTypes() behind a friend declaration drew a line that did not correspond to anything. The subclasses the tests used to reach the protected member are gone with it: TestableRTDEParser and TestableRTDEWriter held nothing but a using declaration, and TestableDataPackage only inherited constructors that were public anyway. Tests now name the library types directly.
Applications still construct a DataPackage from the input recipe and write only the fields they care about. That package does not have the robot's layout hash, so requiring a full match rejected the flow that already worked. copyFrom() now falls back to a recipe-hash compare and two positional loops: validate types, then take written values and send the rest as zeros. A package built from a different recipe is still rejected.
A DataPackage built from a recipe carries no data types, since those only arrive with the robot's setup acknowledgement. The parser used to reject such a package, discard it and allocate a typed replacement with a warning. That made the first getDataPackageBlocking() call allocate for every application following the documented pattern, including our own rtde_client example. The types can simply be applied to the package that was handed in: its storage is already the right shape, and setTypes() only overwrites the variant alternatives, which allocates nothing. The parser now does that whenever the package's recipe hash matches the negotiated one, and keeps allocating a replacement only for a package built from a genuinely different recipe, where parsing on would silently write fields under the wrong names. RTDEParser holds the negotiated layout as a typed template package rather than a bare hash, so the recipe and layout identities and the blueprint for packages it has to allocate itself all come from one object that cannot go stale. Also adds an RTDE writer example covering sendPackage() with the typed input package, and corrects the documentation that this behaviour touches: - The claims that the read path never allocates were true for getDataPackage() but not for the blocking read; they hold again now, and the architecture doc explains which call types the package. - The writer example's sample output was not reproducible from its own code. It is replaced with a run captured against URSim 5.25.1, and the lag and comparison descriptions are corrected to match what the code checks. - Two parser tests asserting that a package "is replaced" now assert that it is typed in place, and an allocation test pins the first parse into an untyped package at zero allocations.
The parser now takes the layout identity from its typed template package, so this static helper had no production caller left.
…ocks. ~TCPServer ran after those members were gone, which aborted the handshake-retry test on macOS arm64 with mutex lock failed: Invalid argument.
…ional. A v2 caller passing the same bytes as before would otherwise read the recipe-id as field data. Type names are validated before any field is written, and the slow-copy warning waits until destruction so a partial send does not allocate.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…type. Reconnect writes that state from another thread, so polling it was a data race. getDataType already reports a type setData() established, which the docs now match.
There was a problem hiding this comment.
🟡 Changes recommended
Public setup mutators can race with serialization, and additional undefined-behavior and test-lifetime issues remain.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 32/32 changed files
- Comments generated: 4
- Review effort level: Balanced
Comment on lines
+104
to
+108
| void RTDEWriter::setRecipeTypes(const std::vector<std::string>& types) | ||
| { | ||
| std::lock_guard<std::mutex> lock_guard(store_mutex_); | ||
| data_buffer0_->setTypes(types); | ||
| data_buffer1_->setTypes(types); |
| * \throws UrException if the number of types doesn't match the recipe or if a type is unknown. | ||
| * Every name is checked before any field is written, so a failure leaves the package unchanged. | ||
| */ | ||
| void setTypes(const std::vector<std::string>& types); |
Member
Author
There was a problem hiding this comment.
@urfeex It seems it would like to hide the new set function. Which mean use 'friend' pattern again :-/
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DataPackage carried a hand-maintained table of every subscribable field name and its data type. Every field the controller gained had to be added to it, and a field missing from it could not be used even though the robot would happily serve it. The robot already reports the data type of every field it acknowledges, so the table was a second source of truth for information we were being told anyway.
The library now keeps only the list of data types the protocol defines and takes the per-field types from the setup acknowledgement.
Applications do not have to change. A DataPackage is still constructed from a recipe, that is still the only place it allocates, and the robot's answer is applied to it afterwards in place. Because every RTDE type is trivially copyable and lives inline in the variant, changing which type a field holds touches no memory, so a package can be created before the connection exists and still be used in a real-time loop. tests/test_rtde_allocations.cpp pins this down by counting allocations across the receive and send paths.
Also fixes comm::TCPServer::writeUnchecked() reading its written out-parameter before assigning it, which made the fake RTDE server silently send nothing and is why the new robot-free tests could not run before.
See doc/migration_notes.rst for the four behavioural differences.