NL2SQL represents failures as structured PipelineError objects and propagates them through state. Retries are managed at the subgraph level, and a single circuit breaker provides fast-fail safety for vector retrieval.
PipelineError includes:
node, message, severity, error_codeis_retryable derived from severity and error codeCommon error codes include MISSING_SQL, EXECUTION_FAILED, PIPELINE_TIMEOUT, SECURITY_VIOLATION.
create_breaker() configures pybreaker.CircuitBreaker instances with observability hooks. The system defines exactly one:
VECTOR_BREAKER (fail_max=5, reset_timeout=30)Retrieval calls in VectorStore are wrapped with VECTOR_BREAKER. It is the only breaker instance the system defines: LLM calls and SQL execution are not breaker-guarded, and their failures surface as PipelineError values in state.
flowchart TD
Node[Pipeline Node] --> Error[PipelineError]
Error --> State[GraphState.errors]
State --> Retry{is_retryable?}
Retry -->|yes| Refine[RefinerNode / retry loop]
Retry -->|no| Stop[Terminate branch]
See ../architecture/failure_recovery.md for failure domains, retry scope, and recovery limitations.
run_with_graph() enforces a global timeout (Settings.global_timeout_sec).nl2sql.common.cancellation.CancellationToken, passed to the graph via config["configurable"]["cancellation_token"], so cancelling one run never affects another.packages/nl2sql/src/nl2sql/common/errors.pypackages/nl2sql/src/nl2sql/common/resilience.pypackages/nl2sql/src/nl2sql/pipeline/subgraphs/sql_agent.pypackages/nl2sql/src/nl2sql/common/cancellation.py