-
Notifications
You must be signed in to change notification settings - Fork 114
Add LangGraph workflow streams sample #315
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
brianstrauch
wants to merge
7
commits into
main
Choose a base branch
from
langgraph-streaming-sample
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
12bc724
Add LangGraph workflow streams sample
brianstrauch 63e53b4
Add langgraph_plugin code owners
brianstrauch 3ec3095
Merge branch 'main' into langgraph-streaming-sample
brianstrauch c4dc3e8
Trigger CI
brianstrauch 346b49f
Add README for graph_api/streaming sample
brianstrauch 9948014
Dedupe streaming tokens on a sequence id
brianstrauch 4f23187
Fix mypy error in streaming test dedupe
brianstrauch File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| # Streaming (Graph API) | ||
|
|
||
| Streams a LangGraph run to an external client while the workflow is still running, using Temporal's durable, offset-addressed [`WorkflowStream`](https://docs.temporal.io/). The graph writes a short story about a topic and emits both fine-grained tokens and node-completion progress on separate topics. | ||
|
|
||
| ## What This Sample Demonstrates | ||
|
|
||
| - **Node token streaming** — the `write_story` node calls LangGraph's `get_stream_writer()` to emit tokens. The plugin's `streaming_topic="tokens"` routes those writes onto the `"tokens"` topic. | ||
| - **Workflow-side `astream` publish** — the workflow drives the graph with `app.astream(...)` and publishes each node-completion chunk onto a `"progress"` topic it owns. | ||
| - A single client subscribing to all topics and demultiplexing on `item.topic`. | ||
| - Waiting for the client to acknowledge (via signal) before completing, since the stream disappears when the workflow ends. | ||
| - **Idempotent consumption** — each token chunk carries a monotonic sequence id so the client can dedupe, because streaming is at-least-once per activity attempt (a retried node re-runs and re-publishes its writes). | ||
|
|
||
| ## Running the Sample | ||
|
|
||
| Prerequisites: `uv sync --group langgraph` and a running Temporal dev server (`temporal server start-dev`). | ||
|
|
||
| ```bash | ||
| # Terminal 1 | ||
| uv run langgraph_plugin/graph_api/streaming/run_worker.py | ||
|
|
||
| # Terminal 2 | ||
| uv run langgraph_plugin/graph_api/streaming/run_workflow.py | ||
| ``` | ||
|
|
||
| ## Files | ||
|
|
||
| | File | Description | | ||
| |------|-------------| | ||
| | `workflow.py` | Graph node functions, graph definition, and `StreamingWorkflow` that publishes to the stream | | ||
| | `run_worker.py` | Registers graph with `LangGraphPlugin` (`streaming_topic="tokens"`), starts worker | | ||
| | `run_workflow.py` | Starts the workflow, subscribes to the stream, prints tokens and progress, then acks | |
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| """Worker for the streaming sample (Graph API).""" | ||
|
|
||
| import asyncio | ||
| import os | ||
|
|
||
| from temporalio.client import Client | ||
| from temporalio.contrib.langgraph import LangGraphPlugin | ||
| from temporalio.worker import Worker | ||
|
|
||
| from langgraph_plugin.graph_api.streaming.workflow import ( | ||
| StreamingWorkflow, | ||
| make_streaming_graph, | ||
| ) | ||
|
|
||
|
|
||
| async def main() -> None: | ||
| client = await Client.connect(os.environ.get("TEMPORAL_ADDRESS", "localhost:7233")) | ||
| # streaming_topic routes node get_stream_writer() output onto the "tokens" topic. | ||
| plugin = LangGraphPlugin( | ||
| graphs={"streaming": make_streaming_graph()}, | ||
| streaming_topic="tokens", | ||
| ) | ||
|
|
||
| worker = Worker( | ||
| client, | ||
| task_queue="langgraph-streaming", | ||
| workflows=[StreamingWorkflow], | ||
| plugins=[plugin], | ||
| ) | ||
| print("Worker started. Ctrl+C to exit.") | ||
| await worker.run() | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| asyncio.run(main()) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| """Start the streaming workflow and subscribe to its Workflow Stream (Graph API).""" | ||
|
|
||
| import asyncio | ||
| import os | ||
| from datetime import timedelta | ||
|
|
||
| from temporalio.client import Client | ||
| from temporalio.contrib.workflow_streams import WorkflowStreamClient | ||
|
|
||
| from langgraph_plugin.graph_api.streaming.workflow import StreamingWorkflow | ||
|
|
||
|
|
||
| async def main() -> None: | ||
| client = await Client.connect(os.environ.get("TEMPORAL_ADDRESS", "localhost:7233")) | ||
|
|
||
| handle = await client.start_workflow( | ||
| StreamingWorkflow.run, | ||
| "a brave robot", | ||
| id="streaming-workflow", | ||
| task_queue="langgraph-streaming", | ||
| ) | ||
|
|
||
| # Subscribe to all topics on the workflow's stream and demultiplex on topic. | ||
| ws = WorkflowStreamClient.create(client, handle.id) | ||
| # Streaming is at-least-once per activity attempt, so a retried node may | ||
| # re-publish tokens. Dedupe on the chunk's seq to consume idempotently. | ||
| seen_tokens: set[int] = set() | ||
| async for item in ws.subscribe( | ||
| from_offset=0, | ||
| result_type=dict, | ||
| poll_cooldown=timedelta(milliseconds=50), | ||
| ): | ||
| if item.topic == "tokens": | ||
| seq = item.data["seq"] | ||
| if seq in seen_tokens: | ||
| continue # duplicate from a node retry; already consumed. | ||
| seen_tokens.add(seq) | ||
| print(item.data["token"], end="", flush=True) | ||
|
brianstrauch marked this conversation as resolved.
|
||
| elif item.topic == "progress": | ||
| if item.data.get("done"): | ||
| # Let the workflow know we are done consuming so it can complete. | ||
| await handle.signal(StreamingWorkflow.ack_stream) | ||
| break | ||
| print(f"\n[progress] {item.data}") | ||
|
|
||
| result = await handle.result() | ||
| print(f"\n\nFinal result: {result}") | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| asyncio.run(main()) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,97 @@ | ||
| """Streaming with the LangGraph Graph API and Temporal Workflow Streams. | ||
|
|
||
| A workflow's :class:`WorkflowStream` is a durable, offset-addressed event channel | ||
| external clients can subscribe to while the workflow is still running. This sample | ||
| demonstrates both ways the LangGraph plugin produces stream items: | ||
|
|
||
| - **Node token streaming** -- the ``write_story`` node calls LangGraph's | ||
| ``get_stream_writer()`` to emit fine-grained tokens. The plugin is configured with | ||
| ``streaming_topic="tokens"`` (see ``run_worker.py``), which routes those writes onto | ||
| the ``"tokens"`` topic. | ||
| - **Workflow-side ``astream`` publish** -- the workflow drives the graph with | ||
| ``app.astream(...)`` and publishes each node-completion chunk onto a ``"progress"`` | ||
| topic it owns. | ||
|
|
||
| A single client subscribes to all topics and demultiplexes on ``item.topic``. | ||
| """ | ||
|
|
||
| from datetime import timedelta | ||
|
|
||
| from langgraph.config import get_stream_writer | ||
| from langgraph.graph import START, StateGraph | ||
| from temporalio import workflow | ||
| from temporalio.contrib.langgraph import graph as temporal_graph | ||
| from temporalio.contrib.workflow_streams import WorkflowStream | ||
| from typing_extensions import TypedDict | ||
|
|
||
|
|
||
| class State(TypedDict): | ||
| topic: str | ||
| story: str | ||
|
|
||
|
|
||
| async def outline(state: State) -> dict[str, str]: | ||
| """Produce a short opening line. Runs first so ``astream`` emits an early chunk.""" | ||
| return {"story": f"A story about {state['topic']}:"} | ||
|
|
||
|
|
||
| async def write_story(state: State) -> dict[str, str]: | ||
| """Write the story, emitting each word as a token via the stream writer. | ||
|
|
||
| Streaming is at-least-once per activity attempt: if this node retries | ||
| (transient failure, worker restart) it re-runs from scratch and re-publishes | ||
| its writes, so subscribers may see the same token twice. Each chunk therefore | ||
| carries a monotonic ``seq`` so consumers can dedupe idempotently. A retry | ||
| re-emits the same ``seq`` values, letting the client drop the duplicates. | ||
| """ | ||
| writer = get_stream_writer() | ||
| words = f"{state['story']} Once upon a time, there was {state['topic']}.".split() | ||
| for seq, word in enumerate(words): | ||
| writer({"seq": seq, "token": word + " "}) | ||
| return {"story": " ".join(words)} | ||
|
|
||
|
|
||
| def make_streaming_graph() -> StateGraph: | ||
| g = StateGraph(State) | ||
| activity_metadata = { | ||
| "execute_in": "activity", | ||
| "start_to_close_timeout": timedelta(seconds=10), | ||
| } | ||
| g.add_node("outline", outline, metadata=activity_metadata) | ||
| g.add_node("write_story", write_story, metadata=activity_metadata) | ||
| g.add_edge(START, "outline") | ||
| g.add_edge("outline", "write_story") | ||
| return g | ||
|
|
||
|
|
||
| @workflow.defn | ||
| class StreamingWorkflow: | ||
| def __init__(self) -> None: | ||
| # WorkflowStream must be constructed during workflow initialization. | ||
| self.stream = WorkflowStream() | ||
| self._stream_acked = False | ||
|
|
||
| @workflow.signal | ||
| def ack_stream(self) -> None: | ||
| """Signalled by the client once it has finished consuming the stream.""" | ||
| self._stream_acked = True | ||
|
|
||
| @workflow.run | ||
| async def run(self, topic: str) -> str: | ||
| app = temporal_graph("streaming").compile() | ||
| progress = self.stream.topic("progress") | ||
|
|
||
| story = "" | ||
| async for chunk in app.astream({"topic": topic, "story": ""}): | ||
| # Each chunk is {node_name: {state updates}}. Forward it as progress. | ||
| progress.publish(chunk) | ||
| for node_update in chunk.values(): | ||
| if "story" in node_update: | ||
| story = node_update["story"] | ||
|
|
||
| progress.publish({"done": True}) | ||
|
|
||
| # The stream disappears when the workflow completes, so wait until the | ||
| # client acknowledges it has finished consuming before returning. | ||
| await workflow.wait_condition(lambda: self._stream_acked) | ||
| return story |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| import uuid | ||
| from datetime import timedelta | ||
| from typing import Any | ||
|
|
||
| from temporalio.client import Client | ||
| from temporalio.contrib.langgraph import LangGraphPlugin | ||
| from temporalio.contrib.workflow_streams import WorkflowStreamClient | ||
| from temporalio.worker import Worker | ||
|
|
||
| from langgraph_plugin.graph_api.streaming.workflow import ( | ||
| StreamingWorkflow, | ||
| make_streaming_graph, | ||
| ) | ||
|
|
||
|
|
||
| async def test_streaming_graph_api(client: Client) -> None: | ||
| task_queue = f"streaming-test-{uuid.uuid4()}" | ||
| plugin = LangGraphPlugin( | ||
| graphs={"streaming": make_streaming_graph()}, | ||
| streaming_topic="tokens", | ||
| ) | ||
|
|
||
| async with Worker( | ||
| client, | ||
| task_queue=task_queue, | ||
| workflows=[StreamingWorkflow], | ||
| plugins=[plugin], | ||
| ): | ||
| handle = await client.start_workflow( | ||
| StreamingWorkflow.run, | ||
| "a brave robot", | ||
| id=f"streaming-{uuid.uuid4()}", | ||
| task_queue=task_queue, | ||
| ) | ||
|
|
||
| ws = WorkflowStreamClient.create(client, handle.id) | ||
| tokens: list[dict[str, Any]] = [] | ||
| progress: list[dict[str, Any]] = [] | ||
| async for item in ws.subscribe( | ||
| from_offset=0, | ||
| result_type=dict, | ||
| poll_cooldown=timedelta(milliseconds=10), | ||
| ): | ||
| if item.topic == "tokens": | ||
| tokens.append(item.data) | ||
| elif item.topic == "progress": | ||
| if item.data.get("done"): | ||
| await handle.signal(StreamingWorkflow.ack_stream) | ||
| break | ||
| progress.append(item.data) | ||
|
|
||
| result = await handle.result() | ||
|
|
||
| # Tokens reassemble into the final story. Streaming is at-least-once per | ||
| # activity attempt, so dedupe on seq (keeping first-seen order) before | ||
| # reassembling, exactly as an idempotent consumer would. | ||
| assert tokens, "expected at least one token" | ||
| assert all("seq" in t and "token" in t for t in tokens) | ||
| seen: set[int] = set() | ||
| deduped: list[dict[str, Any]] = [] | ||
| for t in tokens: | ||
| if t["seq"] not in seen: | ||
| seen.add(t["seq"]) | ||
| deduped.append(t) | ||
| assembled = "".join(t["token"] for t in deduped).strip() | ||
| assert assembled == result | ||
|
|
||
| # Workflow-side astream publish: one chunk per node, in order. | ||
| assert [list(chunk)[0] for chunk in progress] == ["outline", "write_story"] | ||
| assert result == progress[-1]["write_story"]["story"] | ||
| assert "a brave robot" in result |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.