Bug report
Steps to reproduce
- Run the test suite so ctest schedules
launch_testing tests in parallel (for example the conda-based jazzy unit job: colcon test --ctest-args -LE 'linter|integration').
- The
ros2_medkit_log_bridge and ros2_medkit_action_status_bridge integration tests run at the same time.
test_01_error_log_creates_attributed_fault in ros2_medkit_log_bridge fails: AssertionError: unexpectedly None : expected a LOG_PLANNER_SERVER_* fault.
Expected behavior
Each integration test runs on its own DDS domain and only discovers its own nodes.
Actual behavior
The add_launch_test integration tests in ros2_medkit_fault_manager, ros2_medkit_diagnostic_bridge, ros2_medkit_fault_reporter, ros2_medkit_action_status_bridge and ros2_medkit_log_bridge do not set ROS_DOMAIN_ID, so they all run on the default domain 0. Only ros2_medkit_integration_tests assigns a unique domain per launch test.
Under parallel ctest, tests on domain 0 discover each other's nodes. In the observed failure the ros2_medkit_log_bridge fault manager picked up a fault from another test (ACTION_TEST_ACTION_ABORTED from /test_action_status_client, a node owned by the ros2_medkit_action_status_bridge integration test). That extra traffic shifted timing so the planner_server ERROR log was not turned into a fault in time, and the assertion failed.
It is non-deterministic: it only triggers when ctest schedules two colliding tests at the same time and DDS discovery is fast enough. Most runs pass, which is why the same commit passes the Docker jazzy job and the conda humble job but fails the conda jazzy job.
Related: the ros2_medkit_integration_tests domain pool declares an upper bound of 229, which overruns the gtest ranges assigned to ros2_medkit_log_bridge (210-214) and ros2_medkit_action_status_bridge (215-219) once it has more than 70 tests (it currently has 74+).
Environment
- ros2_medkit version: main
- ROS 2 distro: Jazzy (also latent on Humble and Lyrical)
- OS: Ubuntu 24.04
Additional information
add_launch_test targets are not covered by the existing medkit_set_test_domain calls, which are applied only to the gtest targets in those packages.
Bug report
Steps to reproduce
launch_testingtests in parallel (for example the conda-based jazzy unit job:colcon test --ctest-args -LE 'linter|integration').ros2_medkit_log_bridgeandros2_medkit_action_status_bridgeintegration tests run at the same time.test_01_error_log_creates_attributed_faultinros2_medkit_log_bridgefails:AssertionError: unexpectedly None : expected a LOG_PLANNER_SERVER_* fault.Expected behavior
Each integration test runs on its own DDS domain and only discovers its own nodes.
Actual behavior
The
add_launch_testintegration tests inros2_medkit_fault_manager,ros2_medkit_diagnostic_bridge,ros2_medkit_fault_reporter,ros2_medkit_action_status_bridgeandros2_medkit_log_bridgedo not setROS_DOMAIN_ID, so they all run on the default domain0. Onlyros2_medkit_integration_testsassigns a unique domain per launch test.Under parallel ctest, tests on domain 0 discover each other's nodes. In the observed failure the
ros2_medkit_log_bridgefault manager picked up a fault from another test (ACTION_TEST_ACTION_ABORTEDfrom/test_action_status_client, a node owned by theros2_medkit_action_status_bridgeintegration test). That extra traffic shifted timing so theplanner_serverERROR log was not turned into a fault in time, and the assertion failed.It is non-deterministic: it only triggers when ctest schedules two colliding tests at the same time and DDS discovery is fast enough. Most runs pass, which is why the same commit passes the Docker jazzy job and the conda humble job but fails the conda jazzy job.
Related: the
ros2_medkit_integration_testsdomain pool declares an upper bound of 229, which overruns the gtest ranges assigned toros2_medkit_log_bridge(210-214) andros2_medkit_action_status_bridge(215-219) once it has more than 70 tests (it currently has 74+).Environment
Additional information
add_launch_testtargets are not covered by the existingmedkit_set_test_domaincalls, which are applied only to the gtest targets in those packages.