Testing · LangGraph · Advanced
Testing Protection Against Stale Human Approvals in LangGraph

A human approval is only safe when it is applied to the exact question the human saw. This article shows how to test that invariant when an agent retries, replans, or changes the proposed action before the answer returns.
Human-in-the-loop workflows often look safe in a happy-path demo: the agent pauses, a reviewer clicks “approve,” and execution continues. The dangerous case is less visible. The agent pauses with question A, the run is retried or replanned, the agent now asks question B, and an old approval is accidentally accepted as if it answered B.
The central test target is therefore not merely “does the graph resume?” It is: does the graph reject a resume value whose identity does not match the pending approval request?
What “stale approval” means
LangGraph can pause execution with an interrupt and later continue with a resume value. A durable checkpointer preserves the graph state between those operations, commonly under a configurable thread ID.
The persistence mechanism answers “where is this paused execution?” It does not, by itself, prove that an approval belongs to the current request. That second property must be represented and checked by the application boundary.
A stale approval is any answer created for an earlier approval request but submitted after the pending request has changed. The change may come from:
- a retry that regenerated the proposed action;
- a planner revision after a tool failure;
- a user editing the task while an older approval screen remained open;
- a deployment or graph-version change;
- duplicate or delayed delivery from a client or queue.
The approval may still be semantically reasonable. That is not enough. A reviewer who approved “delete object X” did not necessarily approve “delete object Y,” even if both actions look similar.
The invariant to test
Model every approval request with a stable identity. At minimum, the resume payload should be tied to:
- the execution or thread being resumed;
- the specific pending interrupt;
- the question or action version shown to the reviewer;
- the action payload, preferably through a digest;
- the graph or policy version when deployments can change behavior.
A useful invariant is:
accept(resume) only if
resume.thread_id == pending.thread_id
and resume.interrupt_id == pending.interrupt_id
and resume.request_version == pending.request_version
and resume.action_digest == pending.action_digest
and resume.graph_version == pending.graph_version
You may choose different field names, but the test should make the binding explicit. Do not rely on a human-readable prompt, node name, timestamp, or array position as the only identity. Those values can repeat or change independently of the action.
A concrete failure scenario
Consider an agent that prepares a privileged file operation. The first plan asks:
Approve moving /incoming/report.csv to /archive/report.csv?
The reviewer approves that request. Before the response reaches the worker, the plan is rebuilt because the source file is no longer available. The new plan asks:
Approve moving /incoming/report.csv to /archive/2026/report.csv?
If the system accepts only a boolean such as {"approved": true}, the old
answer can be applied to the new action. The test must prove that the old approval is
rejected before the action executes.
Represent the approval request explicitly
The following dataclasses are deliberately small. They separate the request shown to a human from the response submitted by a client. The digest is calculated from a canonical representation of the action, not from display text.
from dataclasses import dataclass
from hashlib import sha256
import json
def action_digest(action: dict) -> str:
canonical = json.dumps(
action,
sort_keys=True,
separators=(",", ":"),
ensure_ascii=False,
).encode("utf-8")
return sha256(canonical).hexdigest()
@dataclass(frozen=True)
class ApprovalRequest:
thread_id: str
interrupt_id: str
request_version: int
graph_version: str
action: dict
action_digest: str
@classmethod
def create(
cls,
*,
thread_id: str,
interrupt_id: str,
request_version: int,
graph_version: str,
action: dict,
) -> "ApprovalRequest":
return cls(
thread_id=thread_id,
interrupt_id=interrupt_id,
request_version=request_version,
graph_version=graph_version,
action=action,
action_digest=action_digest(action),
)
@dataclass(frozen=True)
class ApprovalResponse:
thread_id: str
interrupt_id: str
request_version: int
graph_version: str
action_digest: str
approved: bool
In production, the request identity should be stored with the paused execution or in a transactionally related approval record. The client should return the identity it received, rather than reconstructing it from the visible text.
Validate before resuming the graph
Put the identity check at the boundary that accepts a resume. This is important because a graph node may be reached through more than one transport: an HTTP endpoint, a queue consumer, an administrative tool, or a test helper.
class StaleApprovalError(Exception):
"""The submitted answer does not identify the current approval request."""
def validate_resume(
pending: ApprovalRequest,
response: ApprovalResponse,
) -> None:
fields = (
("thread_id", pending.thread_id, response.thread_id),
("interrupt_id", pending.interrupt_id, response.interrupt_id),
("request_version", pending.request_version, response.request_version),
("graph_version", pending.graph_version, response.graph_version),
("action_digest", pending.action_digest, response.action_digest),
)
mismatches = [
name for name, expected, received in fields
if expected != received
]
if mismatches:
raise StaleApprovalError(
"approval does not match pending request: "
+ ", ".join(mismatches)
)
def resume_if_current(
pending: ApprovalRequest,
response: ApprovalResponse,
resume_graph,
graph_config: dict,
):
validate_resume(pending, response)
# The actual Command/import shape depends on the LangGraph version
# and the integration layer used by the application.
return resume_graph(response.approved, graph_config)
The validation function is intentionally independent from LangGraph. That makes it possible to test the safety property without requiring a model, network service, or particular checkpointer implementation.
The minimum regression test
The most valuable test has four phases:
- create approval request A;
- create a changed request B for the same thread;
- submit A’s response while B is pending;
- assert that validation fails and the side effect is not called.
With pytest, the core test can look like this:
import pytest
def test_old_approval_is_rejected_after_plan_change():
request_a = ApprovalRequest.create(
thread_id="thread-1",
interrupt_id="interrupt-1",
request_version=1,
graph_version="policy-v7",
action={
"operation": "move",
"source": "/incoming/report.csv",
"destination": "/archive/report.csv",
},
)
response_for_a = ApprovalResponse(
thread_id=request_a.thread_id,
interrupt_id=request_a.interrupt_id,
request_version=request_a.request_version,
graph_version=request_a.graph_version,
action_digest=request_a.action_digest,
approved=True,
)
request_b = ApprovalRequest.create(
thread_id="thread-1",
interrupt_id="interrupt-2",
request_version=2,
graph_version="policy-v7",
action={
"operation": "move",
"source": "/incoming/report.csv",
"destination": "/archive/2026/report.csv",
},
)
with pytest.raises(StaleApprovalError):
validate_resume(request_b, response_for_a)
This test does not claim that LangGraph itself creates or rejects these identifiers. It verifies the application invariant around the graph’s interrupt and resume flow.
Prove that no side effect occurs
Rejecting the response is necessary but not sufficient. A regression test should also prove that the protected operation was not called.
from unittest.mock import Mock
def test_stale_approval_cannot_execute_action():
execute_action = Mock()
request_a = ApprovalRequest.create(
thread_id="thread-1",
interrupt_id="interrupt-1",
request_version=1,
graph_version="policy-v7",
action={
"operation": "delete",
"resource_id": "object-a",
},
)
stale_response = ApprovalResponse(
thread_id=request_a.thread_id,
interrupt_id=request_a.interrupt_id,
request_version=request_a.request_version,
graph_version=request_a.graph_version,
action_digest=request_a.action_digest,
approved=True,
)
request_b = ApprovalRequest.create(
thread_id="thread-1",
interrupt_id="interrupt-2",
request_version=2,
graph_version="policy-v7",
action={
"operation": "delete",
"resource_id": "object-b",
},
)
with pytest.raises(StaleApprovalError):
validate_resume(request_b, stale_response)
execute_action.assert_not_called()
In an integrated test, replace the mock with the function that would send the side effect: a database mutation, tool call, message publication, or external API request. The assertion belongs as close as possible to the action boundary.
Test the digest mismatch separately
A request can retain the same interrupt identifier while its action changes if the application reuses identifiers incorrectly. That deserves a separate test.
def test_same_interrupt_id_does_not_hide_changed_action():
request_a = ApprovalRequest.create(
thread_id="thread-1",
interrupt_id="interrupt-reused-by-bug",
request_version=1,
graph_version="policy-v7",
action={"operation": "refund", "amount": "10.00"},
)
request_b = ApprovalRequest.create(
thread_id="thread-1",
interrupt_id="interrupt-reused-by-bug",
request_version=2,
graph_version="policy-v7",
action={"operation": "refund", "amount": "100.00"},
)
response_for_a = ApprovalResponse(
thread_id=request_a.thread_id,
interrupt_id=request_a.interrupt_id,
request_version=request_a.request_version,
graph_version=request_a.graph_version,
action_digest=request_a.action_digest,
approved=True,
)
with pytest.raises(StaleApprovalError):
validate_resume(request_b, response_for_a)
The version mismatch already protects this example. The digest is still valuable as a defense in depth and as an audit field: it lets operators compare the exact action approved with the exact action executed.
Test retry and replan behavior at the graph boundary
Unit tests validate the identity function. An integration test should exercise the lifecycle that caused the bug:
def test_retry_creates_new_pending_question_and_invalidates_old_answer(
graph_runner,
approval_store,
execute_action,
):
thread_id = "thread-retry-1"
first_pause = graph_runner.start(
thread_id=thread_id,
task="archive the report",
)
old_request = approval_store.get_pending(thread_id)
assert old_request.interrupt_id == first_pause.interrupt_id
# A transient failure causes the planner to produce a new action.
graph_runner.retry(thread_id=thread_id, reason="source changed")
new_request = approval_store.get_pending(thread_id)
assert new_request.request_version > old_request.request_version
assert new_request.action_digest != old_request.action_digest
old_response = ApprovalResponse(
thread_id=old_request.thread_id,
interrupt_id=old_request.interrupt_id,
request_version=old_request.request_version,
graph_version=old_request.graph_version,
action_digest=old_request.action_digest,
approved=True,
)
with pytest.raises(StaleApprovalError):
graph_runner.resume(
thread_id=thread_id,
response=old_response,
)
execute_action.assert_not_called()
This is a contract test for your integration layer, not a claim about a specific
LangGraph release. Adapt start, retry, and
resume to the functions your application actually exposes.
Make the test concurrency-aware
Two approval submissions can race. Both requests may pass an in-memory check before one of them marks the interrupt as consumed. Your persistence layer should enforce single-use semantics atomically.
A useful database-level rule is equivalent to:
UPDATE approval_requests
SET status = 'consumed',
consumed_at = :now
WHERE thread_id = :thread_id
AND interrupt_id = :interrupt_id
AND request_version = :request_version
AND action_digest = :action_digest
AND status = 'pending';
The application should require exactly one affected row. Zero rows means that the request is stale, already consumed, cancelled, or otherwise not eligible. The graph resume should happen only after this conditional update succeeds, preferably within a transaction strategy that matches your durability requirements.
A concurrency test can use a barrier so two workers submit the same response at nearly the same time:
def test_only_one_duplicate_approval_can_be_consumed(
approval_store,
concurrent_submit,
):
request = approval_store.create(
thread_id="thread-concurrent-1",
interrupt_id="interrupt-1",
request_version=1,
graph_version="policy-v7",
action={"operation": "publish", "document_id": "doc-1"},
)
response = ApprovalResponse(
thread_id=request.thread_id,
interrupt_id=request.interrupt_id,
request_version=request.request_version,
graph_version=request.graph_version,
action_digest=request.action_digest,
approved=True,
)
results = concurrent_submit(
lambda: approval_store.consume_if_current(request, response),
count=2,
)
assert sorted(results) == [False, True]
The helper in this example is a test fixture. Its implementation must use the real database transaction or compare-and-set mechanism used by the service.
Verify the actual interrupt payload
A common mistake is to test only the approval store while the graph emits a different identity at runtime. Add an inspection step around the interrupt payload and record the fields that the client must return.
def test_interrupt_payload_contains_resume_identity(graph_runner):
pause = graph_runner.start(
thread_id="thread-payload-1",
task="archive the report",
)
assert pause.payload["thread_id"] == "thread-payload-1"
assert pause.payload["interrupt_id"]
assert pause.payload["request_version"] == 1
assert pause.payload["graph_version"]
assert pause.payload["action_digest"]
assert pause.payload["action"]
If the framework exposes an interrupt object with a different shape, map it into your application’s approval DTO before it reaches the UI or API response. The important point is that the identity travels with the human-visible request.
Do not use the prompt as the identity
Text comparison is tempting:
if submitted_prompt == current_prompt:
resume()
It is fragile for several reasons:
- formatting, localization, and whitespace can change;
- the same text can describe different hidden parameters;
- the displayed prompt may omit fields that affect execution;
- model-generated wording is not a stable protocol identifier;
- two distinct requests can legitimately have identical text.
Use a structured action and a digest for binding. Keep the prompt for humans.
Failure cases the test suite should cover
A compact matrix helps prevent a test suite from proving only the happy path.
| Case | Expected result | Why it matters |
|---|---|---|
| All identity fields match | Resume accepted once | Confirms the normal workflow remains usable |
| Wrong thread ID | Reject | Prevents cross-thread approval reuse |
| Wrong interrupt ID | Reject | Prevents answers for another pause being applied |
| Older request version | Reject | Protects retries and replans |
| Changed action digest | Reject | Protects against hidden payload changes |
| Changed graph or policy version | Reject or require re-review | Prevents old policy assumptions from authorizing new behavior |
| Duplicate matching response | First accepted, second rejected | Enforces single-use approval semantics |
| Malformed or missing identity | Reject without graph resume | Prevents fail-open parsing behavior |
| Cancelled request | Reject | Ensures cancellation is a real state transition |
Example API contract
An approval response can be explicit without exposing secrets or personal data:
{
"thread_id": "thread-1",
"interrupt_id": "interrupt-2",
"request_version": 2,
"graph_version": "policy-v7",
"action_digest": "sha256:...",
"approved": true
}
Treat the digest as an opaque value at the API boundary. Do not let clients submit an arbitrary action alongside approval and then calculate the digest from that submitted action. The server should load the pending action, compare the returned digest, and resume only the stored pending execution.
Observability for rejected answers
A stale response is often a useful operational signal. Log enough to diagnose the mismatch without logging sensitive action contents:
- thread ID or an internal redacted correlation ID;
- pending interrupt ID and submitted interrupt ID;
- pending and submitted request versions;
- pending and submitted graph versions;
- whether the digest matched;
- the final rejection reason.
Avoid recording credentials, access tokens, raw personal data, or complete action payloads in ordinary application logs. If an audit record requires the action, apply the retention and access controls appropriate to that data.
Commands for a local verification loop
If the test is stored in tests/test_stale_approval.py, a typical local
command is:
python -m pytest -q tests/test_stale_approval.py
To focus on the identity and side-effect assertions:
python -m pytest -q \
tests/test_stale_approval.py \
-k "stale or digest or duplicate"
These commands are examples of a verification workflow; they were not run for this article. Record the actual command, environment, dependency versions, and result in your project’s test report when you execute them.
What a passing test should establish
A useful test report should let a reviewer answer all of these questions:
- Was an approval request created before the plan changed?
- Did the retry or replan create a distinct current request?
- Was the old response submitted against the new request?
- Did the validation reject it for an observable reason?
- Was the protected side effect not called?
- Does a current, matching response still resume successfully?
- Does a duplicate response fail after the first response is consumed?
If the test checks only for an exception, it may miss a dangerous implementation that raises after already performing the side effect. Place the side-effect assertion in the same test or in a closely coupled integration test.
Limitations and design choices
Identity binding reduces one class of approval bugs; it does not make every human-in- the-loop system safe.
- It does not prove that the reviewer understood the action or that the UI rendered it correctly.
- It does not replace authorization checks. The submitting principal still needs permission to approve the action.
- A digest is only as meaningful as the canonical action representation. Exclude no execution-relevant field from the digest.
- Graph-version binding can cause more re-review after deployments. That is usually a deliberate safety trade-off.
- Atomic consumption protects against duplicate submission, but external side effects may still require idempotency keys and reconciliation.
- A durable checkpointer preserves state; it does not automatically define your business rules for expiration, cancellation, or approval authority.
Recommended implementation checklist
- Give every pending approval a server-generated interrupt ID.
- Increment a request version whenever the proposed action changes.
- Canonicalize all execution-relevant action fields.
- Store a digest of that canonical action with the pending request.
- Return the identity fields to the approval client.
- Validate every field before invoking the graph resume.
- Consume the request atomically and only once.
- Assert that stale responses cannot trigger side effects.
- Exercise retries, replans, cancellations, duplicate delivery, and concurrency.
- Log mismatch reasons without exposing secrets or unnecessary personal data.
Conclusion
The safety property is simple to state: an approval must authorize one specific pending request, not merely a thread or a boolean decision. The practical way to protect it is to make request identity explicit, validate it before resuming the graph, consume it atomically, and test that an old answer is rejected after a retry or plan change.
Start with the focused regression test, then add the graph-boundary and concurrency tests. That progression gives you a small, framework-independent proof of the invariant and a larger test that protects the real execution path.
Continue with the site’s guides or look up related terms in the glossary.