Skip to content

feat(graph_watchdog): orphan detector (GRAPH_ORPHAN) - #578

Merged
bburda merged 1 commit into
mainfrom
feat/graph-watchdog-orphan
Jul 29, 2026
Merged

feat(graph_watchdog): orphan detector (GRAPH_ORPHAN)#578
bburda merged 1 commit into
mainfrom
feat/graph-watchdog-orphan

Conversation

@bburda

@bburda bburda commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Summary

Second of the seven silent-fault detectors. Raises GRAPH_ORPHAN for a topic with publishers but no subscribers, or the reverse, when a topic with the complementary shape exists nearby: same namespace, same message type, name within a very small edit distance.

That is what a remap typo looks like from the graph. One node publishes /scan_raw, another subscribes /scan_row, both are healthy, and neither is talking to the other.

Two deliberate choices:

The fault names both sides and calls it a candidate. Nothing in the graph can tell a typo from a counterpart that has just exited or has not started yet, so the description says both and asks the operator to check before renaming.

A pair must hold for more than grace sweeps, default 10. During a staged bringup one side legitimately exists without the other, which is indistinguishable from a typo until the second node appears. Reporting on a single sweep would fire on every launch.


Issue

Part of #570.


Type

  • Bug fix
  • New feature or tests
  • Breaking change
  • Documentation only

Testing

21 tests green locally on Jazzy, up from 18. New: a policy unit test for the near-miss matching and the edit distance, an integration test driving real publishers and subscribers over real DDS, and an e2e against a real gateway asserting over /faults.

colcon build --packages-select ros2_medkit_graph_watchdog
colcon test --packages-select ros2_medkit_graph_watchdog && colcon test-result --verbose

Checklist

  • Breaking changes are clearly described (and announced in docs / changelog if needed)
  • Tests were added or updated if needed
  • Docs were updated if behavior or public API changed

Copilot AI review requested due to automatic review settings July 29, 2026 12:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds the second “silent-fault” detector to the ros2_medkit_graph_watchdog plugin: an orphan/near-miss topic-name detector that raises GRAPH_ORPHAN when a pub-only topic and a sub-only topic look like a likely typo pair (same namespace + type + small edit distance) and persist beyond a configurable grace period.

Changes:

  • Introduces orphan detector implementation and pure matching policy (find_orphans).
  • Adds unit/integration/e2e tests covering matching behavior, persistence gate, and full gateway + /faults surface behavior.
  • Updates graph watchdog documentation and wires new tests into CMake.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/ros2_medkit_plugins/ros2_medkit_graph_watchdog/include/ros2_medkit_graph_watchdog/orphan_policy.hpp Adds pure near-miss matching + Levenshtein helper for orphan pairing.
src/ros2_medkit_plugins/ros2_medkit_graph_watchdog/src/detectors/orphan_detector.cpp Implements orphan detector, persistence gate, and aggregated fault emission.
src/ros2_medkit_plugins/ros2_medkit_graph_watchdog/test/test_orphan_policy.cpp Unit tests for find_orphans() matching/guards/dedup.
src/ros2_medkit_plugins/ros2_medkit_graph_watchdog/test/test_orphan_integration.cpp Integration test driving real pub/sub over DDS and asserting ReportFault traffic.
src/ros2_medkit_plugins/ros2_medkit_graph_watchdog/test/e2e/test_orphan_e2e.test.py End-to-end launch test asserting GRAPH_ORPHAN raise/clear via GET /api/v1/faults.
src/ros2_medkit_plugins/ros2_medkit_graph_watchdog/README.md Documents the new detector and its configuration keys/test tiers.
src/ros2_medkit_plugins/ros2_medkit_graph_watchdog/CMakeLists.txt Registers new gtests and launch test targets.

Comment thread src/ros2_medkit_plugins/ros2_medkit_graph_watchdog/CMakeLists.txt Outdated
Comment thread src/ros2_medkit_plugins/ros2_medkit_graph_watchdog/CMakeLists.txt Outdated
Comment thread src/ros2_medkit_plugins/ros2_medkit_graph_watchdog/README.md Outdated
Comment thread src/ros2_medkit_plugins/ros2_medkit_graph_watchdog/README.md Outdated
@bburda
bburda force-pushed the feat/graph-watchdog-orphan branch from 6f7445c to 5aa7b71 Compare July 29, 2026 13:53
Raises GRAPH_ORPHAN for a topic that has publishers but no subscribers, or the
reverse, when a topic with the complementary shape exists in the same namespace
with the same message type and a name within a very small edit distance. That is
what a remap typo looks like from the graph: both nodes healthy, neither talking
to the other.

The fault names both sides and says plainly that it is a candidate, either a typo
or a counterpart that has gone away, because nothing in the graph can tell those
apart and the operator should check both before renaming anything.

Three guards keep the detector quiet on healthy graphs. The leaf and the namespace
carry separate edit budgets and the namespace one defaults to zero, so two robots
are never paired. A pair differing only in digits is an enumeration of sensors, not
a typo, so /lidar_1 next to /lidar_2 is not reported; the cost is that a typo in a
digit is not reported either, which is deliberate. A pair must also hold for more
than grace consecutive sweeps, default 10, because during a staged bringup one side
legitimately exists without the other for a while.

Raising namespace_edit_distance reports a misspelled namespace, /robott/scan against
/robot/scan, which is invisible by default. It stays opt-in because a fleet named
with letters puts every robot one edit from its neighbour, and only a numbered fleet
is saved by the digit rule.

An operator who still gets a false alarm can silence a topic with the allowlist
key, matched exactly and never by prefix.

The e2e carries three pairs at once, one reported, one allowlisted and one visible
only under a raised namespace budget, so it also proves a string array and an integer
survive the YAML to ROS parameter to nested-config path into configure(); both C++
tiers hand configure() a JSON object directly and cannot show that.

ROS system topics are matched by exact name rather than by prefix, so a user topic
that merely starts with the same characters is not skipped.
@bburda
bburda force-pushed the feat/graph-watchdog-orphan branch from 5aa7b71 to f71bacc Compare July 29, 2026 13:58
@bburda
bburda merged commit 961f94f into main Jul 29, 2026
14 checks passed
@bburda
bburda deleted the feat/graph-watchdog-orphan branch July 29, 2026 14:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A topic-name typo leaves both sides healthy and silently disconnected

3 participants