Skip to content

feat: Migrate LangGraph runner to AgentGraphRunnerResult; clean up legacy shape detection#156

Merged
jsonbailey merged 4 commits intomainfrom
jb/aic-2174/langchain-graph-runner
May 4, 2026
Merged

feat: Migrate LangGraph runner to AgentGraphRunnerResult; clean up legacy shape detection#156
jsonbailey merged 4 commits intomainfrom
jb/aic-2174/langchain-graph-runner

Conversation

@jsonbailey
Copy link
Copy Markdown
Contributor

@jsonbailey jsonbailey commented Apr 28, 2026

Summary

  • Updates LangGraphAgentGraphRunner.run() to return AgentGraphRunnerResult (with GraphMetrics) instead of the legacy AgentGraphResult shape
  • Adds collect_node_metrics() to LDMetricsCallbackHandler for pure data extraction (no LD tracker calls from the runner)
  • Removes the transitional isinstance(raw_result, AgentGraphRunnerResult) detection branch from ManagedAgentGraph — now that both OpenAI and LangGraph runners return the new shape, the bridge is dead code
  • All graph-level and per-node tracking events are driven exclusively by ManagedAgentGraph._flush_graph_tracking()
  • Updates test_langgraph_agent_graph_runner.py to verify runner returns AgentGraphRunnerResult and does NOT call graph tracker directly
  • Updates test_tracking_langgraph.py to verify events via handler flush() or result shape; removes direct runner → graph tracker assertions

Stack position

This is PR 11-langchain, stacked on:

After this PR, the legacy AgentGraphResult shape is fully retired from production code.

Test plan

  • make test passes in packages/ai-providers/server-ai-langchain (81 tests)
  • uv run pytest packages/sdk/server-ai/tests/ passes (149 tests)
  • make test passes in packages/ai-providers/server-ai-openai (40 tests)

🤖 Generated with Claude Code


Note

Medium Risk
Medium risk because it changes the LangGraph runner’s public result shape and shifts when/where tracking metrics are produced, which could break callers or alter emitted metrics if assumptions differ.

Overview
Updates LangGraphAgentGraphRunner.run() to return the new AgentGraphRunnerResult with GraphMetrics (content, duration, path, aggregated usage, and per-node metrics) instead of the legacy AgentGraphResult/direct tracker calls.

Refactors LDMetricsCallbackHandler to build per-node LDAIMetrics incrementally and expose them via node_metrics, removing the old flush() path that emitted LaunchDarkly events.

Tests are updated to assert the new result shape and that graph-level tracking is driven via ManagedAgentGraph (runner no longer calls graph trackers directly), with callback-handler tests now validating pure metric accumulation.

Reviewed by Cursor Bugbot for commit 95442cd. Bugbot is set up for automated code reviews on this repo. Configure here.

@jsonbailey jsonbailey requested a review from a team as a code owner April 28, 2026 23:46
@jsonbailey jsonbailey marked this pull request as draft April 28, 2026 23:46
@jsonbailey jsonbailey force-pushed the jb/aic-2174/openai-graph-runner branch from 9733a28 to 44501e3 Compare April 29, 2026 13:15
@jsonbailey jsonbailey force-pushed the jb/aic-2174/langchain-graph-runner branch from 0539ba1 to 404670d Compare April 29, 2026 13:15
@jsonbailey jsonbailey force-pushed the jb/aic-2174/openai-graph-runner branch from 44501e3 to 142e041 Compare April 29, 2026 13:19
@jsonbailey jsonbailey force-pushed the jb/aic-2174/langchain-graph-runner branch from 404670d to f132154 Compare April 29, 2026 13:19
@jsonbailey jsonbailey force-pushed the jb/aic-2174/openai-graph-runner branch from 142e041 to fb3c0f6 Compare April 29, 2026 13:22
@jsonbailey jsonbailey force-pushed the jb/aic-2174/langchain-graph-runner branch from f132154 to eb1004c Compare April 29, 2026 13:22
@jsonbailey jsonbailey force-pushed the jb/aic-2174/openai-graph-runner branch from fb3c0f6 to b3547b0 Compare April 29, 2026 13:52
@jsonbailey jsonbailey force-pushed the jb/aic-2174/langchain-graph-runner branch from eb1004c to 8a049e2 Compare April 29, 2026 13:53
@jsonbailey jsonbailey force-pushed the jb/aic-2174/openai-graph-runner branch from b3547b0 to 1d4ddb2 Compare April 29, 2026 13:57
@jsonbailey jsonbailey force-pushed the jb/aic-2174/langchain-graph-runner branch from 8a049e2 to cea3780 Compare April 29, 2026 13:57
@jsonbailey jsonbailey force-pushed the jb/aic-2174/openai-graph-runner branch from 1d4ddb2 to 6201d09 Compare April 29, 2026 14:38
@jsonbailey jsonbailey force-pushed the jb/aic-2174/langchain-graph-runner branch from cea3780 to f27f9b8 Compare April 29, 2026 14:39
@jsonbailey jsonbailey force-pushed the jb/aic-2174/openai-graph-runner branch from 6201d09 to ef4216c Compare April 29, 2026 16:34
@jsonbailey jsonbailey force-pushed the jb/aic-2174/langchain-graph-runner branch from f27f9b8 to d892533 Compare April 29, 2026 16:34
@jsonbailey jsonbailey force-pushed the jb/aic-2174/openai-graph-runner branch from ef4216c to 8ecce16 Compare April 30, 2026 14:05
@jsonbailey jsonbailey force-pushed the jb/aic-2174/langchain-graph-runner branch from d892533 to 13ee088 Compare April 30, 2026 14:06
@jsonbailey jsonbailey force-pushed the jb/aic-2174/openai-graph-runner branch from 8ecce16 to 09af502 Compare April 30, 2026 14:23
@jsonbailey jsonbailey force-pushed the jb/aic-2174/langchain-graph-runner branch from 13ee088 to 3159524 Compare April 30, 2026 14:24
@jsonbailey jsonbailey force-pushed the jb/aic-2174/openai-graph-runner branch from 09af502 to 43bc879 Compare April 30, 2026 14:47
@jsonbailey jsonbailey force-pushed the jb/aic-2174/langchain-graph-runner branch from 3159524 to 2c5671d Compare April 30, 2026 14:48
@jsonbailey jsonbailey force-pushed the jb/aic-2174/openai-graph-runner branch from 43bc879 to f6df746 Compare May 4, 2026 17:10
@jsonbailey jsonbailey force-pushed the jb/aic-2174/langchain-graph-runner branch from 2c5671d to 9318539 Compare May 4, 2026 17:15
@jsonbailey jsonbailey marked this pull request as ready for review May 4, 2026 19:14
Base automatically changed from jb/aic-2174/openai-graph-runner to main May 4, 2026 21:05
jsonbailey and others added 3 commits May 4, 2026 16:06
…cy shape detection

Updates LangGraphAgentGraphRunner to return AgentGraphRunnerResult with GraphMetrics
(success, path, duration_ms, usage, node_metrics) instead of the legacy AgentGraphResult.
Adds collect_node_metrics() to LDMetricsCallbackHandler for pure data extraction.

Removes the transitional AgentGraphResult detection branch from ManagedAgentGraph now
that both the OpenAI and LangGraph runners return AgentGraphRunnerResult.
All graph-level and per-node tracking events are driven exclusively by the managed layer.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove the flush() async method and update all tests to use
collect_node_metrics() which is the correct data-extraction API. The
managed layer (ManagedAgentGraph) owns actual tracking; flush() was
retained only for tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jsonbailey jsonbailey force-pushed the jb/aic-2174/langchain-graph-runner branch from 4411d93 to ba4b5dd Compare May 4, 2026 21:09
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ba4b5dd. Configure here.

@jsonbailey jsonbailey merged commit efa8e00 into main May 4, 2026
45 checks passed
@jsonbailey jsonbailey deleted the jb/aic-2174/langchain-graph-runner branch May 4, 2026 21:45
@github-actions github-actions Bot mentioned this pull request May 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants