LangChainRust
A LangChain-inspired Rust framework for building production LLM applications: agents, RAG, memory, MCP, A2A, guardrails, evaluation, and LangGraph workflows — all in pure Rust.
Start here
- Quick Start — a working agent in 5 minutes (Rust or Python)
- Installation — versions, feature flags, MSRV
- Agents — function calling, ReAct, planning, subagents
- MCP Protocol — client, server, 2026-07-28 compliance
- OpenAI-Compatible Server — call the runtime from any SDK
- Evaluation — evaluators, verifiers, regression gates
Why this framework
- Pure Rust — one binary, no Python runtime
- Production-first — durable execution, approvals, budgets, guardrails, audit
- Protocol-complete — MCP, A2A, and an OpenAI-compatible endpoint
- Verifiable — evaluators, verifiers, regression gates
Quick Start
From zero to a working agent in 5 minutes.
Prerequisites
- Rust 1.85+
- An API key for OpenAI or any OpenAI-compatible endpoint
1. Rust: an agent with a tool
export OPENAI_API_KEY=sk-...
export OPENAI_MODEL=gpt-4o-mini # optional
cargo run -p langchainrust --example quickstart_agent
The example (crates/lc/examples/quickstart_agent.rs) builds a
FunctionCallingAgent with Calculator, runs one tool-calling loop, and
prints the answer:
use std::sync::Arc;
use langchainrust::agents::{AgentExecutor, FunctionCallingAgent};
use langchainrust::core::tools::BaseTool;
use langchainrust::tools::Calculator;
use langchainrust::{OpenAIChat, OpenAIConfig};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let llm = OpenAIChat::new(OpenAIConfig::from_env_result()?);
let tools: Vec<Arc<dyn BaseTool>> = vec![Arc::new(Calculator::new())];
let agent = FunctionCallingAgent::new(
llm,
tools.clone(),
Some("You are a concise assistant. Use tools when needed.".to_string()),
);
let executor = AgentExecutor::new(Arc::new(agent), tools).with_max_iterations(5);
println!("{}", executor.invoke("What is 25 * 37?".to_string()).await?);
Ok(())
}
2. Python: call the runtime over HTTP
Terminal 1 — start the OpenAI-compatible server:
cargo run -p langchainrust --example openai_server --features openai-server
# OpenAI-compatible server on http://127.0.0.1:8080
Terminal 2 — install the thin client and call it:
pip install -e sdks/python
python - <<'PY'
from langchainrust import Client
c = Client("http://127.0.0.1:8080", model="gpt-4o-mini")
print(c.chat("What is Rust?").content)
PY
Any OpenAI SDK works against the same endpoint:
from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="unused")
print(client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "hi"}],
))
3. TypeScript: the same endpoint
npm install langchainrust
# or, from this repository: npm install ./sdks/typescript
import { Client } from "langchainrust";
const c = new Client("http://127.0.0.1:8080", "gpt-4o-mini");
console.log((await c.chat("What is Rust?")).content);
Troubleshooting
| Symptom | Fix |
|---|---|
OPENAI_API_KEY error | Export the key; the config fails fast when it is missing |
| Port already in use | LC_ADDR=127.0.0.1:9090 before starting the server |
unknown feature openai-server | Add --features openai-server |
Python ModuleNotFoundError | Run pip install -e sdks/python from the repo root |
Next
- Agents — hooks, budgets, approvals, subagents
- MCP — expose and consume tools
- A2A — agent-to-agent
- Evaluation — verifiers and regression gates
Installation
Cargo
[dependencies]
langchainrust = "0.30.0"
Feature Flags
LangChainRust uses feature flags to keep the default build lean:
[dependencies.langchainrust]
version = "0.30.0"
features = ["qdrant-integration"] # Optional features
Available Features
| Feature | Description |
|---|---|
qdrant-integration | Qdrant vector store |
mongodb-persistence | MongoDB document store |
redis-storage | Redis document store |
sqlite-storage | SQLite document store |
pgvector-storage | PGVector vector store |
local-embeddings | ONNX Runtime local embeddings |
fastembed | FastEmbed local embeddings (ONNX) |
openai-server | OpenAI-compatible HTTP server (/v1/models, /v1/chat/completions) |
Individual Crates
You can also use individual crates directly:
[dependencies]
lc-core = "0.30.0"
lc-providers = "0.30.0"
lc-agents = "0.30.0"
Environment Variables
Most providers require API keys set as environment variables:
export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...
export MISTRAL_API_KEY=...
export COHERE_API_KEY=...
export DEEPSEEK_API_KEY=...
export QWEN_API_KEY=...
See also
LLM Providers
LangChainRust supports 10+ LLM providers through a unified BaseChatModel trait.
Supported Providers
| Provider | Config | Auth |
|---|---|---|
| OpenAI | OpenAIChat | OPENAI_API_KEY |
| Anthropic | AnthropicChat | ANTHROPIC_API_KEY |
| Gemini | GeminiChat | GEMINI_API_KEY |
| Ollama | OllamaChat | Local (no key) |
| DeepSeek | DeepSeekChat | DEEPSEEK_API_KEY |
| Qwen | QwenChat | QWEN_API_KEY |
| Moonshot | MoonshotChat | MOONSHOT_API_KEY |
| Zhipu | ZhipuChat | ZHIPU_API_KEY |
| Mistral | MistralChat | MISTRAL_API_KEY |
| Azure OpenAI | AzureOpenAIChat | AZURE_OPENAI_API_KEY |
| Cohere | CohereChat | COHERE_API_KEY |
Auto-Detection
LLMClient::from_env() auto-detects the provider from environment variables:
#![allow(unused)]
fn main() {
use langchainrust::LLMClient;
let llm = LLMClient::from_env().expect("Set an API key");
}
Priority: OpenAI → Anthropic → Mistral → Ollama (local fallback).
OpenAI-Compatible Wrappers
DeepSeek, Qwen, Moonshot, Zhipu, and Mistral all wrap OpenAIChat internally
via into_openai_config() — they use the same chat format but with different
base URLs and default models.
Native Providers
Azure OpenAI and Cohere have native implementations because their URL/auth format differs from OpenAI’s.
Streaming
All providers support streaming via stream_chat():
#![allow(unused)]
fn main() {
let stream = llm.stream_chat(messages, None).await?;
while let Some(token) = stream.next().await {
print!("{}", token?.content);
}
}
Multimodal
OpenAI supports Whisper (STT), TTS, and DALL-E via the MultimodalModel trait:
#![allow(unused)]
fn main() {
use langchainrust::MultimodalModel;
let transcript = llm.transcribe(audio).await?;
let speech = llm.generate_speech("Hello!").await?;
let image = llm.generate_image("A cat wearing a hat").await?;
}
Retry & Fallback
#![allow(unused)]
fn main() {
use langchainrust::runnables::{RetryConfig, RunnableExt};
let llm_with_retry = llm.with_retry(RetryConfig::new(3));
}
See also
Agents
Agents are autonomous systems that use LLMs to decide which actions to take.
Agent Types
| Agent | Description | Use Case |
|---|---|---|
| ReActAgent | Reason + Act loop | General-purpose tasks |
| FunctionCallingAgent | Native tool calling | OpenAI/Anthropic tools |
| PlanExecuteAgent | Plan → Execute → Replan | Complex multi-step tasks |
| CRAG | Self-correcting RAG | Retrieval with quality control |
| AdaptiveRAG | Adaptive retrieval routing | Different retrieval strategies |
| DeepResearch | Multi-round research | Comprehensive reports |
Supervisor (orchestrator) | Runtime routing to named sub-agents | One objective, several specialist agents |
Basic Usage
#![allow(unused)]
fn main() {
use langchainrust::agents::{ReActAgent, AgentExecutor, BaseAgent};
let agent = ReActAgent::new(llm, prompt_template);
let executor = AgentExecutor::new(Arc::new(agent), tools)
.with_max_iterations(10);
let result = executor.invoke("What is 2+2?".to_string()).await?;
}
Supervisor: Dynamic Sub-Agent Routing
Where SequentialPipeline runs a fixed stage list and FanOutFanIn broadcasts to
everyone, the Supervisor asks a router model once per round which named worker
should handle the next subtask — or whether the job is done. Each worker is a complete
independent agent behind the Orchestrator trait (its own executor, budget, hooks);
the worker’s answer is fed back into the router’s scratchpad, so the next routing
decision is based on what earlier workers actually produced.
#![allow(unused)]
fn main() {
use langchainrust::{
Supervisor, TaskAdapter, Orchestrator, RunContext, AgentTask,
};
use std::sync::Arc;
// Any String -> String Orchestrator (an AgentExecutor, another pipeline, ...)
// becomes a worker through TaskAdapter (it receives an AgentTask, returns String).
let researcher: Arc<dyn Orchestrator<Input = AgentTask, Output = String>> =
Arc::new(TaskAdapter::new(Arc::new(research_executor)));
let writer = Arc::new(TaskAdapter::new(Arc::new(writer_executor)));
let supervisor = Supervisor::new(
router_llm, // String -> String; prompt it to return
vec![ // {"next":"<worker>","task":"..."} or
("researcher".to_string(), researcher), // {"next":"FINISH","answer":"..."}
("writer".to_string(), writer),
],
8, // max delegation rounds
);
let answer = supervisor
.run_with_context(
AgentTask::new("Write a one-page brief on reranking in RAG"),
&RunContext::new_random(),
)
.await?;
}
This is exactly one level of sub-agent recursion: workers are leaf orchestrators
and cannot themselves delegate. Routing is bounded by max_rounds (.with_max_rounds
adjusts it later; values below 1 clamp to 1) — if the model never emits
SUPERVISOR_FINISH, the run errors instead of looping forever. Decisions parse as JSON
first, with a <<<NEXT>>> delimiter fallback (parse_supervisor_decision), so weaker
models still work with an envelope prompt (supervisor_envelope).
Subagent: Context Isolation
The orchestrators above share one message history: a worker sees everything the
parent saw. For long tasks that is the bottleneck — every tool observation and
failed attempt stays in the parent context. SubagentTool takes the other
approach: each call builds a fresh child AgentExecutor with an empty history,
passes only the task text, and returns only {status, summary}.
#![allow(unused)]
fn main() {
use std::sync::Arc;
use langchainrust::{AgentExecutor, SubagentConfig, SubagentTool};
// The factory owns the child's tools, budget and approval handler.
let factory = Arc::new(|depth: usize| {
Ok(Arc::new(AgentExecutor::new(Arc::new(child_agent), child_tools)))
});
let tool = SubagentTool::new(
SubagentConfig::new("researcher", "Delegate a self-contained research sub-task")
.with_max_depth(2),
factory,
);
let parent = AgentExecutor::new(parent_agent, vec![Arc::new(tool)]);
let answer = parent
.invoke("Summarize the latest RAG reranking work".to_string())
.await?;
}
Invariants:
| Invariant | Meaning |
|---|---|
| Context isolation | The child starts empty and receives only task |
| Summary only | The return value is {status, summary}; no intermediate_steps |
| Depth cap | depth >= max_depth returns ToolError::ExecutionFailed |
| No privilege escalation | Tool subset / budget / approval are fixed by the factory |
The factory receives depth + 1, so a recursive subagent can wire
SubagentTool::with_depth(depth) itself; without that wiring max_depth cannot fire.
Parallel Tool Calls
When a model emits several tool calls in one turn, the executor runs them concurrently — bounded by a semaphore (default cap 8) rather than spawned unbounded:
#![allow(unused)]
fn main() {
let executor = AgentExecutor::new(agent, tools)
.with_max_concurrency(4); // at most 4 in-flight tool calls per turn
}
Observations are zipped back to their actions in the model’s call order (not completion order), so multi-tool turns are deterministic; the same guarantee holds under streaming.
Hooks
Add lifecycle hooks for approval, content filtering, and logging:
#![allow(unused)]
fn main() {
use langchainrust::hooks::{ApprovalHook, ContentFilterHook, LoggingHook};
let executor = AgentExecutor::new(agent, tools)
.hook(ApprovalHook::new()) // Require approval
.hook(ContentFilterHook::new(vec!["secret".into()])) // Filter words
.hook(LoggingHook::new()); // Log events
}
Streaming
#![allow(unused)]
fn main() {
let stream = executor.stream(input);
while let Some(event) = stream.next().await {
match event? {
AgentStreamEvent::ToolStart { name, input } => { /* ... */ }
AgentStreamEvent::ToolEnd { name, output } => { /* ... */ }
AgentStreamEvent::FinalAnswer { content } => { /* ... */ }
AgentStreamEvent::PipelineStep { step, detail } => { /* ... */ }
AgentStreamEvent::Error { message } => { /* ... */ }
}
}
}
Cancellation
#![allow(unused)]
fn main() {
use langchainrust::runnables::CancellationToken;
let token = CancellationToken::new();
tokio::spawn(async move {
tokio::time::sleep(Duration::from_secs(30)).await;
token.cancel();
});
let result = executor.invoke_with_config(input, Some(
RunnableConfig::new().with_cancellation_token(token)
)).await;
}
RAG & Retrieval
LangChainRust provides a full Retrieval-Augmented Generation stack: document loaders, splitters, BM25, hybrid search (RRF / weighted fusion), HyDE, multi-query, lexical and hosted neural reranking, MMR diversity selection, small-to-big retrieval (sentence-window / parent-document), token-level late chunking, and GraphRAG.
Feature Overview
| Feature | Type | Description |
|---|---|---|
RAGPipeline | Pipeline | End-to-end chunk + embed + store + retrieve + generate |
BM25Retriever | Sparse | Classic BM25 keyword search with English + Chinese tokenization |
ChunkedBM25Retriever | Sparse | BM25 with parent-child auto-merging (LlamaIndex-style) |
UnifiedHybridIndex | Hybrid | Dual-index with auto-splitting and detailed scores |
reciprocal_rank_fusion | Fusion | Combine BM25 + vector results via Reciprocal Rank Fusion |
HyDERetriever | Expansion | Generate hypothetical document, then retrieve with it |
MultiQueryRetriever | Expansion | Generate multiple query variants, merge results |
RerankingExecutor | Reranking | Post-retrieval scoring with KeywordReranker or BM25Reranker |
AsyncReranker / CohereRerank / JinaRerank | Neural rerank | Hosted cross-encoder reranking behind one async trait; rerank_async helper |
mmr / retrieve_mmr | Diversity | Maximal Marginal Relevance: trade relevance against result redundancy |
FusionMode::Weighted | Fusion | Linear w_bm25·bm25 + w_vec·vector alternative to RRF |
SentenceWindowRetriever | Small-to-big | Retrieve against single sentences, return a ±N-sentence window |
ParentDocumentRetriever | Small-to-big | Small leaf chunks for recall; any hit returns the whole parent document |
late_chunk / add_late_chunked_document | Late chunking | One token-level embedding pass, mean-pool per chunk, inject both legs |
GraphRAG | Graph | Entity/relation extraction, community detection, multi-mode query |
SemanticSplitter | Splitter | Embedding-based semantic chunking |
RecursiveCharacterSplitter | Splitter | Character-level recursive splitting |
RAGPipeline
#![allow(unused)]
fn main() {
use langchainrust::{
RAGPipelineBuilder, OpenAIChat, OpenAIConfig,
OpenAIEmbeddings, OpenAIEmbeddingsConfig, InMemoryVectorStore,
};
let rag = RAGPipelineBuilder::new()
.llm(OpenAIChat::new(OpenAIConfig::new("sk-...")))
.embeddings(OpenAIEmbeddings::new(OpenAIEmbeddingsConfig::new("sk-...")))
.vector_store(InMemoryVectorStore::new())
.retrieve_k(5)
.system("Answer based on the provided context.")
.build()?;
// Or inject any `RetrieverTrait` implementation directly (BM25, hybrid, ...):
// let rag = RAGPipelineBuilder::new()
// .llm(OpenAIChat::new(OpenAIConfig::new("sk-...")))
// .retriever(BM25Retriever::new())
// .build()?;
rag.index_documents(docs).await?;
let answer = rag.query("What is Rust?").await?;
let result = rag.query_with_sources("What is Rust?").await?;
// result.answer, result.sources
}
Hybrid Retrieval with Reranking
#![allow(unused)]
fn main() {
use langchainrust::{
BM25Retriever, RerankingExecutor, KeywordReranker,
reciprocal_rank_fusion,
};
// BM25 sparse retrieval
let bm25 = BM25Retriever::new();
bm25.add_documents_sync(docs.clone());
// Vector retrieval (via SimilarityRetriever)
let vector_results: Vec<Document> = retriever.retrieve("query", 10).await?;
// Fuse with Reciprocal Rank Fusion
let fused = reciprocal_rank_fusion(bm25_results, vector_results, 60);
// Rerank
let executor = RerankingExecutor::new(Box::new(KeywordReranker::new()))
.with_top_n(5)
.with_min_score(0.1);
let reranked = executor.rerank_documents("query", fused_docs)?;
}
Neural Cross-Encoder Reranking
Lexical rerankers are cheap and offline; when recall quality matters more, a hosted cross-encoder scores each candidate against the query jointly. Both providers sit behind one async trait, so swapping providers does not touch call-site code.
#![allow(unused)]
fn main() {
use langchainrust::{CohereRerank, JinaRerank, AsyncReranker, rerank_async};
// Empty key falls back to COHERE_API_KEY; defaults to rerank-multilingual-v3.0.
// JinaRerank::new("") reads JINA_API_KEY (jina-reranker-v2-base-multilingual).
let cohere = CohereRerank::new("")
.with_model("rerank-v3.5"); // optional
// .with_base_url("https://api.cohere.com") (also how tests point at a mock)
// Overshoot on recall, then let the cross-encoder pick the final top_n.
let pool = retriever.retrieve_with_scores("query", 20).await?;
let top: Vec<SearchResult> = rerank_async(&cohere, "query", pool, 5).await?;
}
Contract details worth knowing: the helper maps the API’s results[].index back to
your input positions (provider order is not guaranteed), a malformed response is an
error rather than silently unsorted input, and both clients bypass ambient proxy
settings (.no_proxy()) because rerank endpoints are usually called server-to-server.
MMR Diversity & Weighted Fusion
RRF sorts purely by rank, so one narrow topic can fill the whole top-k. MMR greedily
picks each next item for relevance minus similarity to what is already selected,
with a λ dial: λ=1 is plain relevance, λ=0 maximizes diversity (0.5–0.7 is the
usual range).
#![allow(unused)]
fn main() {
use langchainrust::{UnifiedHybridIndex, HybridIndexConfig, FusionMode, mmr};
// Weighted linear fusion instead of the default RRF.
let config = HybridIndexConfig::new()
.with_fusion(FusionMode::Weighted { bm25_weight: 0.3, vector_weight: 0.7 });
let index = UnifiedHybridIndex::with_config(embeddings, store, 1536, config);
// MMR as a retriever method: pool 20 fused candidates, re-embed just those once,
// return a diversified 5.
let picks = index.retrieve_mmr("query", 20, 5, 0.6).await?;
// Or use the pure algorithm over your own (id, relevance, embedding) triples.
let ids: Vec<String> = mmr(&candidates, 0.6, 5);
}
Fusion scores are min-max normalized inside the candidate pool before MMR, so the λ trade-off means the same thing under RRF and weighted fusion.
Small-to-Big Retrieval
Embedding one sentence gives a precise hit; answering usually needs the surrounding context. Two retrievers implement the small-to-big pattern.
#![allow(unused)]
fn main() {
use langchainrust::{SentenceWindowRetriever, ParentDocumentRetriever,
InMemoryChunkedDocumentStore};
use std::sync::Arc;
// Sentence window: index single sentences, return ±N sentences around each hit.
let sw = SentenceWindowRetriever::from_documents(docs)
.with_window(2) // two sentences on each side (default)
.with_top_k(3);
let windows = sw.retrieve("query", 3).await?; // one deduplicated window per source
// Parent document: tiny leaf chunks for recall, the ENTIRE parent doc for the LLM.
let store = Arc::new(InMemoryChunkedDocumentStore::new());
store.add_parent_with_chunks(parent_doc, vec!["leaf 1".into(), "leaf 2".into()]).await?;
let pdr = ParentDocumentRetriever::new(store);
let parents = pdr.retrieve("query", 4).await?;
}
Unlike ChunkedBM25Retriever’s hit-ratio-gated auto-merging, ParentDocumentRetriever
always returns the parent — leaves exist only for matching.
Late Chunking
Normal chunking embeds each chunk in isolation, so cross-chunk references lose their antecedent. Late chunking runs one token-level embedding pass over the full document, then mean-pools token vectors inside each chunk window — every chunk vector carries document-wide context.
#![allow(unused)]
fn main() {
use langchainrust::{late_chunk, LateChunkConfig, UnifiedHybridIndex};
let config = LateChunkConfig::new().with_chunk_size(256).with_chunk_overlap(32);
let chunks = late_chunk(&token_embedder, &document.content, &config).await?;
// Inject one document into BOTH legs at once: pooled vectors into the vector
// index, chunk texts into the BM25/parent store. Deterministic ids make
// re-registering the same parent idempotent.
index.add_late_chunked_document(document, &chunks).await?;
}
GraphRAG
#![allow(unused)]
fn main() {
use langchainrust::{GraphRAG, GraphRAGConfig, QueryMode as GraphQueryMode};
let graph_rag = GraphRAG::new(llm).with_config(
GraphRAGConfig::new()
.with_max_entities_per_doc(10)
.with_max_relations_per_doc(10),
);
graph_rag.add_documents(&docs).await?;
graph_rag.build_communities().await?;
let result = graph_rag.query("How does X relate to Y?", GraphQueryMode::Local).await?;
// result.answer, result.sources, result.mode
}
Document Loaders
| Loader | Format |
|---|---|
TextLoader | Plain text |
CSVLoader | CSV |
MarkdownLoader | Markdown |
HTMLLoader | HTML |
JSONLoader | JSON |
PDFLoader | |
DocxLoader | Word DOCX |
WebScraperLoader | Web pages |
SitemapLoader | Sitemap XML |
All loaders implement DocumentLoader with async fn load() -> Result<Vec<Document>, LoaderError>.
LangGraph
LangGraph provides graph-based orchestration for stateful, long-running LLM workflows with conditional routing, subgraphs, parallel execution, dynamic in-node interrupt/resume human-in-the-loop, and checkpoint state history with fork-based time travel.
Core Concepts
| Concept | Type | Description |
|---|---|---|
StateGraph<S> | Builder | Define nodes, edges, and reducers on state S |
GraphBuilder<S> | Builder | Fluent (consuming) variant of StateGraph |
CompiledGraph<S> | Runtime | Compiled, executable graph |
GraphNode<S> | Trait | Node execution: async fn execute(&self, state, config) -> NodeResult<S> |
ConditionalEdge<S> | Trait | Dynamic routing: async fn route(&self, state) -> String |
Reducer<S> | Trait | State merge strategy: fn reduce(&self, current, update) -> S |
Checkpointer<S> | Trait | State persistence: save, load, list, delete, snapshots |
SubgraphNode<S, SubS> | Node | Embed a CompiledGraph<SubS> inside a parent graph |
InterruptibleNode<S> | Node | Resume-aware node: the closure receives resume: Option<&Value> (None on first pass, Some(decision) after a human answers) |
CheckpointInfo<S> | Record | One history entry: id, timestamp, seq, recursion_count, state |
StreamEvent<S> | Enum | Start, EnterNode, NodeComplete, StateUpdate, NodeInterrupt, Resumed, End |
Basic Graph
#![allow(unused)]
fn main() {
use langchainrust::{
StateGraph, GraphBuilder, StateSchema, StateUpdate,
AgentState, START, END,
};
use serde::{Deserialize, Serialize};
#[derive(Serialize, Deserialize, Clone, Debug)]
struct MyState { messages: Vec<String>, count: usize }
impl StateSchema for MyState {}
let mut graph = StateGraph::<MyState>::new();
graph.add_node_fn("process", |state: MyState| {
Ok(StateUpdate::full(MyState {
messages: state.messages.clone(),
count: state.count + 1,
}))
});
graph.add_edge(START, "process");
graph.add_edge("process", END);
let compiled = graph.compile()?;
let result = compiled.invoke(MyState { messages: vec![], count: 0 }).await?;
}
Conditional Routing & Human-in-the-Loop
#![allow(unused)]
fn main() {
use langchainrust::{GraphBuilder, FunctionRouter, AgentState, StateUpdate, START, END};
use std::collections::HashMap;
let compiled = GraphBuilder::<AgentState>::new()
.add_node_fn("entry", |s| Ok(StateUpdate::full(s.clone())))
.add_node_fn("short", |s| { /* ... */ })
.add_node_fn("long", |s| { /* ... */ })
.add_node_fn("review", |s| { /* ... */ })
.add_edge(START, "entry")
.add_conditional_edges("entry", "router",
HashMap::from([("short".into(), "short".into()), ("long".into(), "long".into())]),
None)
.add_edge("short", "review")
.add_edge("long", "review")
.add_edge("review", END)
.set_conditional_router("router", FunctionRouter::new(|s: &AgentState| {
if s.input.len() < 10 { "short" } else { "long" }.to_string()
}))
.compile()?
.with_interrupt_before(vec!["review".to_string()]); // HITL
// First invoke returns ExecutionInterrupted
// Resume with: compiled.create_resume_execution("review") ... compiled.resume(execution).await?
}
The static list fixes stopping points before launch. When the decision depends on runtime data (“this action would spend money — ask a human”), use a dynamic in-node interrupt instead.
Dynamic interrupt() + resume (human-in-the-loop)
InterruptibleNode wraps a closure that is called twice per interrupt. On the
first pass resume is None; the node does its pre-side-effect work and suspends
itself by returning GraphError::InterruptRequest { payload }. After a human
answers, resume_with_value re-enters the same node with the answer and the closure
runs again with resume = Some(decision) — branch on it so the side effect that was
approved happens once, on the resumed pass only.
#![allow(unused)]
fn main() {
use langchainrust::{
GraphBuilder, InterruptibleNode, AgentState, StateUpdate,
GraphError, GraphNode, START, END, ThreadSafeMemoryCheckpointer,
};
use serde_json::json;
let approve = InterruptibleNode::new("charge", |state, resume| {
let output = state.output.clone().unwrap_or_default();
Box::pin(async move {
match resume {
// First pass: suspend, hand the approval context to the outside world.
None => Err(GraphError::InterruptRequest {
payload: json!({ "kind": "tool_approval", "command": output }),
}),
// Resumed with the human's decision: perform the side effect once.
Some(decision) => {
let mut next = state.clone();
next.set_output(format!("decision={decision}, charged"));
Ok(StateUpdate::full(next))
}
}
})
});
let compiled = GraphBuilder::<AgentState>::new()
.add_node(approve) // InterruptibleNode implements GraphNode
.add_edge(START, "charge")
.add_edge("charge", END)
.compile()?
// A checkpointer is mandatory: the interrupt payload is persisted into the
// checkpoint, so even a process restart can resume instead of replaying.
.with_checkpointer(ThreadSafeMemoryCheckpointer::new());
// First run suspends. The error carries the node name + payload.
let err = compiled.invoke(AgentState::new("charge $99")).await.unwrap_err();
// GraphError::DynamicInterrupt { node, payload } surfaces the suspended node.
// A human answers; the SAME node is re-entered with the value. Nothing before the
// interrupt is replayed, and the recursion budget carries over.
let invocation = compiled
.resume_with_value("charge", json!({ "approved": true }))
.await?;
println!("{}", invocation.final_state.output.as_deref().unwrap_or(""));
}
Key semantics:
- One persistence system. The interrupt payload lives in the graph checkpoint
(memory / file / SQLite / Postgres / Redis). A brand-new process over the same
checkpoint directory can call
resume_with_value. - Side effects run once. Keep any first-pass work that must not repeat inside
the
Nonebranch; theSome(decision)branch is the only place effects fire. - Cascading approvals. Resuming one node can immediately suspend at the next
interrupt — each resume returns the next
DynamicInterruptuntil the graph reachesEND. - Streaming. A suspended node emits
StreamEvent::NodeInterrupt(name, payload)and the resumed pass emitsStreamEvent::Resumed(name, decision); pre-interrupt events are not replayed.
When agents run on the graph, lc_agents::graph_approval::ApprovalGate builds
exactly this pattern: an agent tool approval is an interrupt payload, and
ApprovalDecision (Allow / Deny { reason } / Modify { arguments, note }) is
the resume value — Deny skips the tool with no side effect.
Checkpointer & Subgraphs
#![allow(unused)]
fn main() {
use langchainrust::{ThreadSafeMemoryCheckpointer, SubgraphBuilder, START, END};
// Checkpointer for state persistence
let checkpointer = ThreadSafeMemoryCheckpointer::<AgentState>::new();
let compiled = graph.compile()?.with_checkpointer(checkpointer);
// Subgraph composition
let subgraph = GraphBuilder::<AgentState>::new()
.add_node_fn("sub_process", |s| Ok(StateUpdate::full(s.clone())))
.add_edge(START, "sub_process")
.add_edge("sub_process", END)
.compile()?;
let parent = GraphBuilder::<AgentState>::new()
.add_subgraph_same_state("subworkflow", subgraph)
.add_edge(START, "subworkflow")
.add_edge("subworkflow", END)
.compile()?;
}
State History & Time Travel
Every checkpointed step is also a snapshot. get_state_history lists them, oldest
first, and fork_from branches the run off any one of them.
#![allow(unused)]
fn main() {
use langchainrust::CheckpointInfo;
// Run a graph with a checkpointer, then inspect where it has been.
let history: Vec<CheckpointInfo<AgentState>> = compiled.get_state_history()?;
for cp in &history {
println!("seq={} ts={} recursion={} id={}", cp.seq, cp.timestamp, cp.recursion_count, cp.id);
let state_at_that_point: &AgentState = &cp.state;
}
// Replay a "what if": branch off an old snapshot and re-run forward from a
// chosen node. Pass Some(state) to edit history (e.g. fix a bad tool output);
// None replays the snapshot's state as-is.
let forked = compiled
.fork_from(&history[2].id, "analyze", Some(corrected_state))
.await?;
println!("forked answer: {}", forked.final_state.output.as_deref().unwrap_or(""));
}
The fork is a new checkpoint lineage — it never mutates the original run, so you
can keep the production timeline and explore alternatives side by side. Execution is
forward-only from the chosen node: nodes before the fork point do not run again
(their side effects are assumed already applied). If you need a true re-execution
including those effects, fork from an earlier snapshot. Forks compose with interrupts:
if the branched path suspends, resume it with resume_with_value exactly like any
other run.
Parallel & Streaming
#![allow(unused)]
fn main() {
// Fan-out / fan-in
graph.add_fan_out("entry", vec!["branch_a".into(), "branch_b".into()]);
graph.add_fan_in(vec!["branch_a".into(), "branch_b".into()], "merge");
// Streaming
let stream = compiled.stream(state);
while let Some(event) = stream.next().await {
match event? {
StreamEvent::EnterNode(name, state) => { /* ... */ }
StreamEvent::NodeComplete(name, update) => { /* ... */ }
StreamEvent::End(final_state) => { /* ... */ }
_ => {}
}
}
// Visualization
println!("{}", compiled.visualize_ascii());
println!("{}", compiled.visualize_mermaid());
}
MCP Protocol
The Model Context Protocol (MCP) is Anthropic’s open standard for connecting LLM applications to external tools and data sources. LangChainRust provides both a client and server implementation. As of 0.22.0 the crate is stateless single-track: every request is a self-contained JSON-RPC HTTP POST tagged with Mcp-Method / Mcp-Name routing headers and a _meta block; there is no handshake and no session. The legacy handshake-based MCPClient (SSE/stdio transports) was removed in 0.22.0.
Feature Overview
| Feature | Type | Description |
|---|---|---|
StatelessMcpClient | Client | Connect to any stateless MCP server, list/call tools |
MCPServer | Server | Expose BaseTool implementations via MCP (handle_request / serve_http / serve_stdio) |
MCPToolAdapter | Adapter | Wrap MCP tools as BaseTool for agent use |
ConnectionManager | Client | 100+ Server lazy startup / idle reaping / managed registry |
ToolNamespace | Client | server:tool name uniquification + conflict policy |
ToolDiscovery | Client | Static (pinned) + dynamic (query top-k) tool selection |
ToolSpec | Client | Per-tool timeout with hard cap |
ServerHealth / CircuitBreaker | Client | Per-server health probe (list_tools) + circuit breaker |
ServerSandbox | Client | Per-server security isolation (param least-privilege + egress whitelist + audit) |
SamplingGuard | Client | Sampling recursion protection (depth + token budget + timeout) |
MCPGateway | Client | Unified registry + on-demand dispatch (rate limiting + audit) |
TenantGateway | Client | Multi-tenant isolation (per-tenant registry + audit) |
ToolOrchestrator / ToolStep | Client | Tool orchestration (dependency DAG + parallel/serial + ${id} template args) |
VersionPolicy / ProtocolInfo | Protocol | Protocol version handling (degrade or reject on unsupported) |
StatelessTransport | Transport | Pure HTTP POST JSON-RPC, no session |
MethodRateLimiter | Client | Client-side per-method rate limiting (fails fast, no network round trip) |
TokenValidator / AuthScheme | Auth | Bearer/JWT token validation (server side) + auth headers (client side) |
McpTaskHandle | Client | MCP Tasks placeholder (persistence deferred to 0.22.1) |
MCP Client
#![allow(unused)]
fn main() {
use langchainrust::mcp::{MCPToolAdapter, StatelessMcpClient};
// Connect to a stateless MCP endpoint — no handshake, this never fails
let client = StatelessMcpClient::connect("http://localhost:3001/mcp");
// List available tools
let tools = client.list_tools().await?;
for tool in &tools {
println!("{}: {}", tool.name, tool.description);
}
// Call a tool
let result = client.call_tool("read_file", serde_json::json!({"path": "/tmp/hello.txt"})).await?;
println!("{}", result.text());
// Convert all MCP tools to BaseTool for agent use
let base_tools: Vec<Arc<dyn BaseTool>> = client
.list_tools()
.await?
.into_iter()
.map(|def| Arc::new(MCPToolAdapter::new(client.clone(), def)) as Arc<dyn BaseTool>)
.collect();
}
Client-side extras:
.with_mrtr(MrtrConfig { max_round_trips })— bound theinput_requiredmulti-round request loop; the client resends the original request with the continuation token (MRTR), up to the limit (-32003when exceeded);.with_answer_provider(provider)— collects answers forinput_requiredquestions (required for servers that use MRTR);.with_method_rate_limiter(MethodRateLimiter)— per-method client-side rate limit; a hit returns-32002without a network round trip;client.meta()— the pinned self-contained_meta(protocol version2026-07-28+ client identity + optionalrequestState).
MCP Server
#![allow(unused)]
fn main() {
use langchainrust::{MCPServer, BaseTool, Calculator};
use std::sync::Arc;
let server = MCPServer::new()
.with_tool(Arc::new(Calculator::new()) as Arc<dyn BaseTool>)
.with_tool(Arc::new(DateTimeTool::new()) as Arc<dyn BaseTool>)
.with_server_info("my-mcp-server", "1.0.0");
// Serve via stdio (for external hosts like Claude Desktop)
server.serve_stdio().await?;
}
Or serve it as a deployable stateless HTTP service (see
examples/mcp_http_server.rs):
#![allow(unused)]
fn main() {
use std::sync::Arc;
use tokio::net::TcpListener;
let listener = TcpListener::bind("0.0.0.0:8788").await?;
let url = Arc::new(server).serve_http(listener); // "http://0.0.0.0:8788/mcp"
println!("clients connect to: {url}");
}
server.handle_request(req) is also available for custom transports: one-shot
JSON-RPC handling with no network at all.
Stateless HTTP Track
Every request is self-contained: the client POSTs a JSON-RPC envelope plus
_meta (protocol version / client identity / optional requestState) and
tags it with the Mcp-Method / Mcp-Name headers so gateways can route and
throttle without parsing the body. There is no handshake, no session id, and
no sticky routing — any HTTP client that can POST JSON can talk to the server.
Auth is per-request bearer/JWT (connect_with_auth on the client,
TokenValidator implementations on the server/gateway side).
Multi-Server Management (100+ Servers)
Holding one dedicated client per server still costs hundreds of long-lived
handles. Use ConnectionManager to host a managed registry, and
ToolNamespace to keep each server’s tools uniquely named.
Connection Manager (lazy start / idle reaping / pooling)
#![allow(unused)]
fn main() {
use langchainrust::{ConnectionManager, ServerSpec};
let manager = ConnectionManager::new();
// register is lazy: nothing is built yet
manager.register(
ServerSpec::new("fs", "http://mcp-fs.internal:8080/mcp"),
).await?;
// stateful servers marked keep_alive are never reaped while idle
manager.register(
ServerSpec::new("db", "http://mcp-db.internal:8080/mcp").keep_alive(),
).await?;
// first client() call builds the client; later calls reuse it
let client = manager.client("fs").await?;
}
Idle servers (non-keep_alive, idle beyond max_idle) are reaped in the
background. manager.reap_idle() triggers a sweep manually;
manager.shutdown() closes everything and stops the reaper task.
Tool Namespace (name uniquification + conflict policy)
Different servers often expose same-named tools (several with read_file). The
ToolNamespace registry uniquifies each tool as server_name:tool_name, with an
explicit conflict strategy:
ToolConflict::Prefix— colliding tools are all exposed, each under its ownserver:prefix;ToolConflict::Reject— a same-named tool from another server is rejected at registration.
#![allow(unused)]
fn main() {
use langchainrust::{ToolNamespace, ToolConflict};
let mut ns = ToolNamespace::new();
// both servers expose "read_file" → distinct names fs:read_file / db:read_file
ns.register("fs", fs_tools, ToolConflict::Prefix)?;
ns.register("db", db_tools, ToolConflict::Prefix)?;
// route a call back to the owning server / raw tool name
let (server, raw) = ns.resolve("fs:read_file").expect("registered");
}
The namespaced adapter exposes server:tool to the LLM while calling the raw
tool name on the server side:
#![allow(unused)]
fn main() {
use langchainrust::MCPToolAdapter;
let client = manager.client("fs").await?;
let adapter = MCPToolAdapter::namespaced(client, "fs", read_file_def);
assert_eq!(adapter.name(), "fs:read_file"); // what the LLM sees
}
Static + Dynamic Tool Discovery
100+ servers can declare hundreds of thousands of tokens of tool schemas —
far beyond any context window. ToolDiscovery avoids injecting everything at
once by splitting tools into two layers:
- Static layer: 20-50 high-frequency tools pinned as always-on;
- Dynamic layer: tools retrieved per-query by relevance (top-k), like a RAG
step over the tool registry. Relevance is scored by
[
KeywordScorer] (token overlap, zero dependencies) by default; implement [ToolScorer] for vector-based scoring and inject it withwith_scorer.
#![allow(unused)]
fn main() {
use langchainrust::{ToolDiscovery, ToolScorer};
let mut discovery = ToolDiscovery::new();
for def in all_tools { discovery.register(def); } // full registry
discovery.pin("get_time"); // static layer: always injected
discovery.pin("search_db");
// per-query injection: pinned tools + top-2 query-relevant tools (deduped)
let injected = discovery.select("find files modified today", /*top_k*/ 2, /*static_limit*/ 50);
}
Per-Tool Timeout with Hard Cap
Long-running tools must not hold callers hostage. ToolSpec gives each tool a
default timeout plus a hard cap that bounds total time regardless of anything
else.
#![allow(unused)]
fn main() {
use langchainrust::{MCPToolAdapter, ToolSpec};
use std::time::Duration;
let adapter = MCPToolAdapter::new(client, def)
.with_timeout(ToolSpec::new("read_file", Duration::from_secs(30)));
// default timeout 30s; hard cap defaults to 3× (90s), or override with .with_max_timeout(..)
}
Health Probe + Circuit Breaker
With 100+ servers a single one can go down at any time. Each registered server
carries a [CircuitBreaker]: list_tools doubles as the health probe, N
consecutive failures trip the breaker open (the server is removed — incoming
client() calls fail fast instead of hammering a dead server), and after an
exponential backoff the breaker opens a half-open probe window so a recovered
server can reconnect.
#![allow(unused)]
fn main() {
use langchainrust::{
ConnectionManager, ServerSpec, ServerHealth, HealthStatus,
};
let manager = ConnectionManager::new();
manager.register(
ServerSpec::new("fs", "http://localhost:8080/mcp")
.with_max_failures(3), // 3 consecutive failures → trip
).await?;
// proactive probe: list_tools is the probe; returns a health snapshot
let health: ServerHealth = manager.health("fs").await?;
match health.status {
HealthStatus::Healthy => { /* normal */ }
HealthStatus::Degraded => { /* failures below threshold */ }
HealthStatus::Down => { /* breaker open */ }
}
// during the open state client() fails fast
// manually remove all tripped servers (returns their names)
let removed = manager.reap_unhealthy().await;
}
Default max_failures is 3. The breaker backs off 0.5s → 1s → 2s → … (cap 30s)
between reconnect attempts. Health is the liveness gate only — tool calls still
fail fast through [ToolSpec] timeouts.
Per-Server Security Sandbox
100+ servers come from very different origins; each one must be narrowed to its
own least-privilege boundary. [ServerSandbox] bundles:
- Parameter-level least privilege — [
ParamRule] constrains tool-call arguments: a filesystem server only allowsfile:///tmp/prefixes, formats only allow enum values, and path-traversal substrings (..) are rejected. Violating calls are blocked before the request is sent to the server. - Outbound network whitelist — [
EgressPolicy] declares which hosts the server may contact; an empty whitelist denies all egress (fail-closed). - Full audit log — every allowed/blocked call is recorded (server, tool, arguments, decision, reason), ring-buffered with a cap.
#![allow(unused)]
fn main() {
use langchainrust::{MCPToolAdapter, ServerSandbox, ParamRule, EgressPolicy};
use std::sync::Arc;
let sandbox = Arc::new(
ServerSandbox::new("fs")
.with_param_rule(ParamRule::Prefix {
field: "path".to_string(),
prefix: "file:///tmp/".to_string(), // only allow tmp prefixes
})
.with_param_rule(ParamRule::RejectContains {
field: "path".to_string(),
forbidden: vec!["..".to_string()], // reject path traversal
})
.allow_host("example.com"), // egress whitelist
);
let adapter = MCPToolAdapter::new(client, def).with_sandbox(sandbox);
// run() checks the sandbox first; a block returns InvalidInput and records audit
// let out = adapter.run(r#"{"path": "file:///etc/passwd"}"#).await?; // Err
}
Same-server tools share one sandbox (cheap Arc clone), so they all write into
the same audit log. Read it back with sandbox.audit_log().
Sampling Recursion Protection
MCP Sampling (“Agent calls tool → tool requests Sampling → LLM calls tool →
another Sampling request”) can recurse unboundedly. [SamplingGuard] bounds
the whole chain on the Host side with three constraints:
- Depth limit — nested Sampling may not exceed
max_depth(default 3); - Cumulative token budget — each request’s
max_tokensaccumulates toward a chain-wide budget; - Timeout / deadline — the whole chain must finish within
total_timeoutor an explicitdeadline.
Call enter(request.max_tokens) before every sampling/createMessage. The
returned [SamplingLease] holds one nesting level for the duration of the call
(atomic counters, safe across await) and releases it on Drop.
#![allow(unused)]
fn main() {
use langchainrust::{SamplingGuard};
use std::time::Duration;
let guard = SamplingGuard::new(/*max_depth*/ 3, /*token_budget*/ 10_000)
.with_timeout(Duration::from_secs(60));
// guard.enter(req.max_tokens)?; // depth/budget/timeout: any breach → Err
// ... run LLM inference ...
// lease Drop releases the nesting depth
}
The counter is AtomicUsize (not a Mutex) so re-entrant nested sampling across
await cannot deadlock, and a rejected enter occupies neither depth nor budget.
MCP Gateway (unified registry + on-demand dispatch)
MCPGateway is the single entry point that composes P2-1~P2-6 into one registry:
it hosts the [ConnectionManager] (lazy connect / idle reaping / circuit breaker),
the [ToolNamespace] (per-server name isolation), the [ToolDiscovery] (static +
dynamic selection), per-server sandboxes and timeouts, plus rate limiting and a
unified audit log. Registering a server is lazy — nothing is built until the
first sync / call.
#![allow(unused)]
fn main() {
use langchainrust::{MCPGateway, GatewayServerSpec};
use std::time::Duration;
let gateway = MCPGateway::new();
// register is lazy: nothing connects yet
gateway.register(
GatewayServerSpec::new("fs", "http://localhost:8080/mcp")
.with_rate_limit(10, Duration::from_secs(60)) // max 10 calls / minute
.with_timeout(Duration::from_secs(30)),
).await?;
gateway.register(
GatewayServerSpec::new("db", "http://localhost:8081/mcp"),
).await?;
// sync connects + pulls tools + populates namespace/discovery (idempotent)
gateway.sync("fs").await?;
gateway.sync_all().await?;
// per-query injection: pinned static tools + top-k relevant dynamic tools
let selected = gateway.select("find files modified today", /*top_k*/ 2, /*static_limit*/ 50);
// on-demand dispatch: "server:tool" resolves → rate-limits → breaker-gated
// client → sandbox → (timeout) → call → audit. Auto-syncs on a first-time miss.
let out = gateway.call("fs:read_file", serde_json::json!({"path": "/tmp/a.txt"})).await?;
// hang the whole registry on an agent as BaseTools (namespaced `server:tool`)
let base_tools = gateway.as_base_tools().await?;
}
Every call passes through a unified audit log (server, tool, decision, reason),
ring-buffered with a cap (with_max_audit); read it with gateway.audit_log().
Ops helpers mirror the underlying manager: health, reap_unhealthy,
reap_idle, release, and shutdown.
Multi-Tenancy (per-tenant isolation)
One process serving many customers (SaaS) must never leak a tenant’s tools,
connections, or audit trail to another. [TenantGateway] wraps an [MCPGateway]
per tenant: register is lazy (nothing is built until a tenant’s first
sync_all / call), and every operation is routed by tenant_id.
#![allow(unused)]
fn main() {
use langchainrust::{TenantGateway, GatewayServerSpec};
let tenants = TenantGateway::new();
// register is scoped to the tenant — tenant "b" never sees tenant "a"'s tools
tenants.register("a", GatewayServerSpec::new("fs", "http://localhost:8080/mcp")).await?;
tenants.sync_all("a").await?;
// on-demand dispatch: auto-syncs on a first-time miss, then calls
let out = tenants.call("a", "fs:read_file", serde_json::json!({"path": "/tmp/a.txt"})).await?;
// audit logs are per-tenant — no cross-tenant visibility
let audit = tenants.audit_log("a");
// teardown: removing a tenant drops its registry and connections
tenants.remove_tenant("a");
}
TenantGateway::tenant_ids() lists live tenants; a missing tenant is created
lazily on first use, so there is no setup step to forget.
Tool Orchestration (dependency DAG)
A multi-step task is often a DAG of tool calls — the output of one tool feeds
the next. [ToolOrchestrator] declares steps with dependencies, then executes:
- Validation first — duplicate ids, unknown dependencies, and dependency cycles are rejected before any tool runs (Kahn topological sort);
- Round-based parallelism — every step whose dependencies are satisfied
runs in the same round, with concurrency capped by
with_max_concurrency(default 4); - Argument templating —
${id}substitutes a previous step’s whole JSON output,${id.field}extracts one field, so downstream args are computed from upstream results.
#![allow(unused)]
fn main() {
use langchainrust::{ToolOrchestrator, ToolStep, MCPGateway};
use serde_json::json;
let orch = ToolOrchestrator::new()
.with_max_concurrency(4)
.add_step(ToolStep::new("a", "fs:read_file", json!({"path": "/tmp/orders.csv"})))
.add_step(ToolStep::new("b", "db:query", json!({"sql": "SELECT ..."})).after("a"))
.add_step(ToolStep::new("sum", "calc:total", json!({
"rows": "${b.sum}",
"file": "${a.content}"
})).after("b"));
// MCPGateway implements ToolCaller — plug the whole DAG into the registry
let results = orch.execute(&gateway).await?;
let total = &results["sum"]["total"];
}
Any [ToolCaller] implementation drives the orchestrator; steps reference
server:tool full names. Failure of one step fails the round — later steps
that depended on it are skipped and the error propagates as OrchestrateError.
Protocol Version Handling (vNext)
MCP protocol versions evolve. On the stateless track there is no handshake —
the client pins its _meta.protocol_version (default 2026-07-28) and the
server can inspect it per request. langchainrust records the negotiation
result and lets you choose how to treat mismatches:
SUPPORTED_PROTOCOL_VERSIONS— the versions this library implements;- [
VersionPolicy::Degrade] (default) — unknown version: keep going on the library’s own version, recordingsupported = false; - [
VersionPolicy::Reject] — unknown version: the request is refused; - [
ProtocolInfo] — the negotiation result (requested,server_version,negotiated,supported).
#![allow(unused)]
fn main() {
use langchainrust::mcp::StatelessMcpClient;
let client = StatelessMcpClient::connect("http://localhost:8080/mcp");
assert_eq!(client.meta().protocol_version, "2026-07-28");
}
2026-07-28 Compliance Fields
The latest revision makes three result-envelope fields mandatory; v0.30.0 implements them:
| Field | Where | Behavior |
|---|---|---|
resultType | Every successful result | "complete" for ordinary results; "input_required" for MRTR interim results |
ttlMs | tools/list, resources/list, resources/read, prompts/list | Defaults to 60_000 ms |
cacheScope | Same as above | Defaults to "public" (process-wide static registries) |
Older peers that omit the fields are still accepted as complete. tools/list and
server/discover sort tools by name before pagination, keeping client caches and
prompt-cache hit rates stable.
Protocol Details
- Version:
2026-07-28(stateless single-track) - Format: JSON-RPC 2.0 over HTTP POST
- Transport: stateless, self-contained requests; no handshake, no session
- Methods:
initialize,tools/list,tools/call,server/discover,resources/list,resources/read,prompts/list,prompts/get,completion/complete,sampling/createMessage
Sub-Protocol Support
| Sub-protocol | Client | Server |
|---|---|---|
| Tools | list_tools, call_tool | serve_stdio, serve_http |
| Resources | list_resources, read_resource | – |
| Prompts | list_prompts, get_prompt | – |
| Sampling | create_message | – |
| Completion | complete | – |
| Roots | list_roots | – |
| Elicitation | elicit | – |
OpenAI-Compatible Server
MCP and A2A cover agent ↔ agent and agent ↔ tool. The openai-server
feature adds the entry point most existing applications speak: OpenAI’s
/v1/chat/completions.
# `lc-agents` exposes the feature today:
lc-agents = { version = "0.30.0", features = ["openai-server"] }
The
langchainrustfacade forwards it asopenai-server. If you build frommain, enablefeatures = ["openai-server"]on the facade as well.
Serving a chat model
#![allow(unused)]
fn main() {
use std::sync::Arc;
use langchainrust::agents::{openai_compat_router, ModelBackend};
let backend = Arc::new(ModelBackend::new(Arc::new(llm)));
let app = openai_compat_router(backend, "my-agent");
// axum::serve(listener, app).await?;
}
Endpoints
| Method | Path | Behavior |
|---|---|---|
| GET | /v1/models | Returns the configured model id |
| POST | /v1/chat/completions | stream: false → JSON; stream: true → SSE followed by [DONE] |
Tool calling
The server maps the OpenAI tool protocol end to end:
| Direction | Shape |
|---|---|
Request role: "tool" | requires tool_call_id; mapped to a tool message |
Request tool_calls on an assistant message | preserved for the next turn |
Response message.tool_calls[] | {id, type: "function", function: {name, arguments}} |
Response finish_reason | "tool_calls" when the model requested tools |
Streaming delta.tool_calls[] | each entry carries index |
{
"model": "my-agent",
"messages": [
{ "role": "user", "content": "weather in beijing?" },
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_1",
"type": "function",
"function": { "name": "get_weather", "arguments": "{}" }
}
]
},
{ "role": "tool", "tool_call_id": "call_1", "content": "sunny" }
]
}
Custom backend
ChatBackend is the seam between the wire protocol and your agent/chain:
#![allow(unused)]
fn main() {
#[async_trait::async_trait]
impl ChatBackend for MyAgentBackend {
async fn complete(&self, messages: Vec<Message>) -> Result<LLMResult, String> { ... }
async fn stream_chat(&self, messages: Vec<Message>)
-> Result<Pin<Box<dyn Stream<Item = Result<StreamChunk, String>> + Send>>, String> { ... }
}
}
Scope
toolrole,tool_callsresponses and streaming tool deltas are mapped.temperature/max_tokensare accepted for compatibility but do not override the backend per request; sampling belongs to the backend configuration.- Client SDKs: Language Clients (Python / TypeScript).
Language Clients (Python / TypeScript)
The Rust core is the runtime. Other languages talk to it over the OpenAI-compatible HTTP endpoint, so they never need to write Rust.
Start the server once:
cargo run -p langchainrust --example openai_server --features openai-server
# OpenAI-compatible server on http://127.0.0.1:8080
Python
pip install -e sdks/python
from langchainrust import Client
c = Client("http://127.0.0.1:8080", model="gpt-4o-mini")
print(c.chat("What is Rust?").content)
for chunk in c.stream("hello"):
print(chunk, end="")
tools = [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the weather for a city",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
},
},
}]
result = c.chat_with_tools("weather in beijing?", tools)
for call in result.tool_calls:
print(call.name, call.arguments)
Methods
| Method | Returns |
|---|---|
chat(input, tools=None) | ChatResult(content, finish_reason, tool_calls, usage) |
stream(input) | iterator of text deltas |
chat_with_tools(input, tools) | same as chat(..., tools=...) |
models() | the server’s model list |
Errors raise LangChainRustError with an optional status.
TypeScript
npm install langchainrust
# or, from this repository: npm install ./sdks/typescript
import { Client } from "langchainrust";
const c = new Client("http://127.0.0.1:8080", "gpt-4o-mini");
console.log((await c.chat("What is Rust?")).content);
for await (const chunk of c.stream("hello")) {
process.stdout.write(chunk);
}
const result = await c.chatWithTools("weather in beijing?", tools);
for (const call of result.toolCalls) {
console.log(call.name, call.arguments);
}
Client exposes chat, chatWithTools, stream and models; errors are
thrown as LangChainRustError with an optional status.
Any OpenAI SDK
Because the endpoint is OpenAI-compatible, existing SDKs work unchanged:
from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="unused")
print(client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "hi"}],
))
Scope
- These clients are thin: they only speak HTTP and do not re-implement agent logic.
- Sampling parameters belong to the backend configuration; the request-level
temperature/max_tokensare accepted for compatibility but do not override it.
Vector Stores
LangChainRust provides a unified VectorStore trait with 10+ backends for storing and retrieving document embeddings.
Supported Backends
| Store | Struct | Feature Gate | Auth |
|---|---|---|---|
| InMemory | InMemoryVectorStore | always | None |
| File | FileVectorStore | always | None |
| ChromaDB | ChromaDBVectorStore | always | http://localhost:8000 |
| Pinecone | PineconeStore | always | API key + host |
| LanceDB | LanceDBVectorStore | always | Optional API key |
| Neo4j | Neo4jVectorStore | always | bolt:// + password |
| Qdrant | QdrantVectorStore | qdrant-integration | http://localhost:6334 |
| PGVector | pgvector (helpers) | pgvector-storage | sqlx connection |
| Redis | RedisDocumentStore | redis-storage | Redis URL |
| SQLite | SQLiteDocumentStore | sqlite-storage | File path |
| MongoDB | MongoChunkedDocumentStore | mongodb-persistence | MongoDB URI |
VectorStore Trait
#![allow(unused)]
fn main() {
#[async_trait]
pub trait VectorStore: Send + Sync {
async fn add_documents(&self, documents: Vec<Document>, embeddings: Vec<Vec<f32>>) -> Result<Vec<String>, VectorStoreError>;
async fn similarity_search(&self, query_embedding: &[f32], k: usize) -> Result<Vec<SearchResult>, VectorStoreError>;
async fn get_document(&self, id: &str) -> Result<Option<Document>, VectorStoreError>;
async fn get_embedding(&self, id: &str) -> Result<Option<Vec<f32>>, VectorStoreError>;
async fn delete_document(&self, id: &str) -> Result<(), VectorStoreError>;
async fn count(&self) -> usize;
async fn clear(&self) -> Result<(), VectorStoreError>;
}
}
InMemory & File
#![allow(unused)]
fn main() {
use langchainrust::{InMemoryVectorStore, FileVectorStore, Document};
// In-memory (no persistence)
let store = InMemoryVectorStore::new();
let ids = store.add_documents(docs, embeddings).await?;
let results = store.similarity_search(&query_vec, 5).await?;
// File-backed (JSON persistence)
let store = FileVectorStore::new(PathBuf::from("./vectors.json"), 1536)?;
}
ChromaDB & Qdrant
#![allow(unused)]
fn main() {
use langchainrust::{ChromaDBVectorStore, ChromaDBConfig};
// ChromaDB
let store = ChromaDBVectorStore::new(ChromaDBConfig::new(
"http://localhost:8000", "my_collection", 1536,
))?;
// Qdrant (feature: qdrant-integration)
use langchainrust::{QdrantVectorStore, QdrantConfig, QdrantDistance};
let store = QdrantVectorStore::new(QdrantConfig::new("http://localhost:6334", "my_collection")
.with_vector_size(1536)
.with_distance(QdrantDistance::Cosine))?;
}
Document & Builder Pattern
#![allow(unused)]
fn main() {
use langchainrust::{Document, VectorStoreBuilder, VectorStoreType};
let doc = Document::new("Hello, world!")
.with_metadata("source", "test")
.with_id("doc-1");
// Builder pattern
let store = VectorStoreBuilder::in_memory().build().await?;
let store = VectorStoreBuilder::file_backed("./vectors.json", 1536).build().await?;
}
See also
Embeddings
LangChainRust provides a unified Embeddings trait with multiple backends for generating text embedding vectors.
Supported Providers
| Provider | Struct | Default Model | Dimension | Feature Gate |
|---|---|---|---|---|
| OpenAI | OpenAIEmbeddings | text-embedding-ada-002 | 1536 | always |
| DeepSeek | DeepSeekEmbeddings | deepseek-embedding | 1536 | always |
| Qwen | QwenEmbeddings | text-embedding-v1 | 1536 | always |
| Cohere | CohereEmbeddings | embed-english-v3.0 | 1024 | always |
| FastEmbed | FastEmbedEmbeddings | BGESmallENV15 | 384 | fastembed |
| BagOfWords | BagOfWordsEmbeddings | local-bow | configurable | always |
| Mock | MockEmbeddings | mock-embeddings | configurable | always |
Embeddings Trait
#![allow(unused)]
fn main() {
#[async_trait]
pub trait Embeddings: Send + Sync {
async fn embed_query(&self, text: &str) -> Result<Vec<f32>, EmbeddingError>;
async fn embed_documents(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>, EmbeddingError>;
fn dimension(&self) -> usize;
fn model_name(&self) -> &str;
}
}
OpenAI Embeddings
#![allow(unused)]
fn main() {
use langchainrust::{OpenAIEmbeddings, OpenAIEmbeddingsConfig};
// From environment (OPENAI_API_KEY)
let embedder = OpenAIEmbeddings::from_env_result()?;
// Explicit config
let embedder = OpenAIEmbeddings::new(
OpenAIEmbeddingsConfig::new("sk-...")
.with_model("text-embedding-3-small")
.with_base_url("https://api.openai.com/v1"),
);
let vec = embedder.embed_query("What is Rust?").await?;
let vecs = embedder.embed_documents(&["doc1 text", "doc2 text"]).await?;
}
FastEmbed (Local ONNX)
#![allow(unused)]
fn main() {
use langchainrust::FastEmbedEmbeddings;
use fastembed::EmbeddingModel;
// Default model (BGESmallENV15, 384-dim)
let embedder = FastEmbedEmbeddings::default_model()?;
// Specific model
let embedder = FastEmbedEmbeddings::with_model(EmbeddingModel::BGESmallENV15)?;
let vec = embedder.embed_query("hello world").await?;
}
BagOfWords & Mock
#![allow(unused)]
fn main() {
use langchainrust::{BagOfWordsEmbeddings, MockEmbeddings};
// BagOfWords (no API needed, FNV-1a hash based)
let embedder = BagOfWordsEmbeddings::default_dim(); // 256-dim
let embedder = BagOfWordsEmbeddings::new(128);
// Mock (deterministic pseudo-random for testing)
let embedder = MockEmbeddings::new(1536);
}
Cosine Similarity
#![allow(unused)]
fn main() {
use langchainrust::cosine_similarity;
let sim = cosine_similarity(&vec_a, &vec_b)?;
// Returns f64 in range [-1.0, 1.0]
}
Memory
LangChainRust provides conversation memory management with multiple strategies: buffer, window, summary, summary-buffer, persistent, and context window.
Memory Types
| Type | Struct | Description | Use Case |
|---|---|---|---|
| Buffer | ConversationBufferMemory | Stores all messages | Short conversations |
| Window | ConversationBufferWindowMemory | Keeps last k rounds | Bounded history |
| Summary | ConversationSummaryMemory | LLM-compressed summary | Long conversations |
| SummaryBuffer | ConversationSummaryBufferMemory | Summary + recent messages | Balanced approach |
| Persistent | PersistentMemory trait | Load/save to storage | Cross-session |
| ContextWindow | ContextWindow | Token-budget fitting | LLM context limits |
| VectorStore | VectorStoreRetrieverMemory | Semantic retrieval | Large history |
Buffer & Window Memory
#![allow(unused)]
fn main() {
use langchainrust::{ConversationBufferMemory, ConversationBufferWindowMemory};
use std::collections::HashMap;
// Buffer -- stores everything
let mut memory = ConversationBufferMemory::new()
.with_input_key("input".to_string())
.with_output_key("output".to_string());
memory.save_context(
&HashMap::from([("input".into(), "What is Rust?".into())]),
&HashMap::from([("output".into(), "A systems language.".into())]),
).await?;
let loaded = memory.load_memory_variables(&HashMap::new()).await?;
// Window -- keeps last k rounds
let mut memory = ConversationBufferWindowMemory::new(5); // last 5 rounds
}
Summary & SummaryBuffer Memory
#![allow(unused)]
fn main() {
use langchainrust::{ConversationSummaryMemory, ConversationSummaryBufferMemory, OpenAIChat};
// Summary -- LLM compresses history
let mut memory = ConversationSummaryMemory::new(llm)
.with_max_recent_turns(2);
// SummaryBuffer -- hybrid with token limit
let mut memory = ConversationSummaryBufferMemory::new(llm, 4000)
.with_memory_key("history");
}
ContextWindow
#![allow(unused)]
fn main() {
use langchainrust::{ContextWindow, Strategy};
// Truncate strategy (default) -- drops oldest, preserves system messages
let cw: ContextWindow<OpenAIChat> = ContextWindow::new(4096)?;
let fitted = cw.fit(messages).await?;
// Summarize strategy -- LLM compresses older messages
let cw = ContextWindow::with_strategy(4096, Strategy::summarize(llm))?;
let fitted = cw.fit(messages).await?;
}
Budget semantics (Truncate): System 消息恒保留且不占预算;若 System 自身超预算,原样返回(可能超预算)。
Persistent Memory
#![allow(unused)]
fn main() {
use langchainrust::{PersistentMemory, PersistenceConfig};
let config = PersistenceConfig::new()
.with_auto_save(true)
.with_token_limit(4000);
// Implement PersistentMemory for your storage backend
// Methods: load_from_store, save_to_store, delete_session, session_exists
}
ChatMessageHistory
#![allow(unused)]
fn main() {
use langchainrust::ChatMessageHistory;
let mut history = ChatMessageHistory::new();
history.add_user_message("Hello");
history.add_ai_message("Hi there!");
history.add_system_message("You are helpful.");
let messages = history.messages();
}
Chains
Chains compose operations into pipelines. LangChainRust provides LLMChain, SequentialChain, ConversationChain, RouterChain, RetrievalQA, and document chains.
Chain Types
| Chain | Struct | Description |
|---|---|---|
| LLMChain | LLMChain<M> | Prompt + LLM (simplest chain) |
| SequentialChain | SequentialChain | Execute chains in sequence |
| ConversationChain | ConversationChain<M> | LLM with conversation memory |
| RouterChain | RouterChain | Keyword-based routing |
| LLMRouterChain | LLMRouterChain<M> | LLM-based routing |
| RetrievalQA | RetrievalQA<M> | RAG question answering |
| ConversationRetrievalChain | ConversationRetrievalChain<M> | RAG with memory |
| StuffDocumentsChain | StuffDocumentsChain<M> | All docs in one prompt |
| RefineDocumentsChain | RefineDocumentsChain<M> | Iterative refinement |
| MapReduceDocumentsChain | MapReduceDocumentsChain<M> | Parallel map + reduce |
| MapRerankDocumentsChain | MapRerankDocumentsChain<M> | Parallel map + rank |
LLMChain
#![allow(unused)]
fn main() {
use langchainrust::{LLMChain, LLMChainBuilder};
use std::collections::HashMap;
use serde_json::Value;
let chain = LLMChain::new(llm, "Explain this topic: {topic}")
.with_input_key("topic")
.with_output_key("text");
let mut inputs = HashMap::new();
inputs.insert("topic".to_string(), Value::String("Rust ownership".to_string()));
let result = chain.invoke(inputs).await?;
// Builder pattern
let chain = LLMChainBuilder::new(llm, "Summarize: {text}")
.input_key("text")
.output_key("summary")
.build();
}
SequentialChain
#![allow(unused)]
fn main() {
use langchainrust::SequentialChain;
let pipeline = SequentialChain::new()
.add_chain(Arc::new(chain1), vec!["topic"], vec!["features"])
.add_chain(Arc::new(chain2), vec!["features"], vec!["summary"]);
let result = pipeline.invoke(inputs).await?;
}
ConversationChain
#![allow(unused)]
fn main() {
use langchainrust::{ConversationChain, ConversationBufferMemory};
let chain = ConversationChain::builder(llm)
.memory(ConversationBufferMemory::new())
.system_prompt("You are a helpful assistant.")
.build();
let reply = chain.predict("What is Rust?").await?;
let reply2 = chain.predict("Tell me more about ownership.").await?; // remembers context
}
RetrievalQA
#![allow(unused)]
fn main() {
use langchainrust::RetrievalQA;
let qa = RetrievalQA::new(llm, retriever)
.with_k(4)
.with_return_source_documents(true);
let answer = qa.query("What is LangChain?").await?;
let (answer, sources) = qa.query_with_sources("What is LangChain?").await?;
}
Document Chains
#![allow(unused)]
fn main() {
use langchainrust::{StuffDocumentsChain, RefineDocumentsChain, MapReduceDocumentsChain};
// Stuff -- all documents in one prompt
let chain = StuffDocumentsChain::new(llm)
.with_prompt_template("Summarize: {context}\nQuestion: {input}");
let result = chain.invoke_with_documents(docs, "What is the main idea?").await?;
// Refine -- iterative refinement
let chain = RefineDocumentsChain::new(llm)
.with_initial_prompt("Answer based on: {context}\nQuestion: {input}")
.with_refine_prompt("Refine this answer: {existing_answer}\nNew context: {context}");
// MapReduce -- parallel map then reduce
let chain = MapReduceDocumentsChain::new(llm)
.with_map_prompt("Summarize: {context}")
.with_reduce_prompt("Combine summaries: {context}");
}
Tools
LangChainRust ships with 12+ built-in tools and a #[tool] macro for custom tools.
Built-in Tools
| Tool | Struct | Name | Key Input |
|---|---|---|---|
| Calculator | Calculator | calculator | expression: String |
| DateTime | DateTimeTool | datetime | operation, datetime, unit |
| Math | SimpleMathTool | math | operation, value, value2 |
| Python REPL | PythonREPLTool | python_repl | code: String |
| DuckDuckGo Search | DuckDuckGoSearchTool | web_search | query: String |
| URL Fetch | URLFetchTool | url_fetch | url, operation |
| Wikipedia | WikipediaTool | wikipedia | query, lang |
| File | FileTool | file_operation | op, path, content |
| HTTP | HTTPTool | http_request | url, method, body |
| SQL | SQLTool | sql_query | SQL string |
| Computer Use | ComputerUseTool | computer_use | action, coordinate |
| Sandbox | SandboxTool<S> | code_interpreter | code: String |
Using Tools with Agents
#![allow(unused)]
fn main() {
use langchainrust::tools::{Calculator, DateTimeTool, SimpleMathTool};
use langchainrust::{FunctionCallingAgent, AgentExecutor, BaseAgent};
use std::sync::Arc;
let tools: Vec<Arc<dyn BaseTool>> = vec![
Arc::new(Calculator::new()),
Arc::new(DateTimeTool::new()),
Arc::new(SimpleMathTool::new()),
];
let agent = FunctionCallingAgent::new(llm, tools.clone(), None);
let executor = AgentExecutor::new(Arc::new(agent) as Arc<dyn BaseAgent>, tools)
.with_max_iterations(5);
let result = executor.invoke("Calculate 15 * 4 and tell me the time".to_string()).await?;
}
Security Features
HTTPTool: SSRF protection by default – blocks private IPs (127.0.0.1, 10.x, 192.168.x, cloud metadata).
FileTool: Path traversal prevention, extension whitelist, size limits.
SQLTool: SELECT-only, blocks SQL comments and dangerous patterns (sleep, benchmark, exec).
PythonREPLTool: Disabled by default; blocks dangerous imports (os, subprocess, socket); configurable timeout.
Sandbox
#![allow(unused)]
fn main() {
use langchainrust::tools::sandbox::{LocalSandbox, CodeSandbox, Language, SandboxTool};
let sandbox = LocalSandbox::new();
let result = sandbox.run("print(2 + 2)", Language::Python, 10_000).await?;
// result.stdout, result.stderr, result.exit_code
let tool = SandboxTool::new(LocalSandbox::new(), Language::Python).with_timeout(10_000);
}
Computer Use
#![allow(unused)]
fn main() {
use langchainrust::tools::ComputerUseTool;
let tool = ComputerUseTool::new_anthropic("sk-...", 1920, 1080);
// Actions: screenshot, click, type, scroll, key_press, wait
}
See also
- Tool macro — derive a typed tool in one attribute
- Full usage guide
Guardrails
Guardrails provide input/output validation to protect production agents from malicious input and sensitive information leakage.
Core Concepts
| Concept | Type | Description |
|---|---|---|
InputGuardrail | Trait | Validates agent input before execution |
OutputGuardrail | Trait | Validates agent output after execution |
GuardrailsConfig | Struct | Configures input + output guardrails |
GuardrailRunner | Struct | Executes guardrail validation pipeline |
GuardedAgent | Struct | Agent wrapper with automatic guardrail enforcement |
Guardrail Result
Input and output guardrails return different result types, so the type system
enforces that Modify can only be produced on the output side.
#![allow(unused)]
fn main() {
// Input side: no Modify variant — input guardrails can only Pass or Block
pub enum InputGuardrailResult {
Pass,
Block { reason: String },
}
// Output side: Modify is the only place rewriting is allowed
pub enum OutputGuardrailResult {
Pass,
Block { reason: String },
Modify { new_value: String },
}
}
Built-in Validators
| Validator | Side | Description |
|---|---|---|
MaxLengthGuardrail | Input | Blocks if input exceeds character limit |
ForbiddenWordsGuardrail | Input | Blocks on forbidden word match (case-insensitive) |
SensitiveInfoGuardrail | Output | Detects API keys, emails, credit cards, sensitive keywords |
Usage
#![allow(unused)]
fn main() {
use langchainrust::{
GuardrailsConfig, GuardedAgent, GuardrailRunner,
MaxLengthGuardrail, ForbiddenWordsGuardrail, SensitiveInfoGuardrail,
};
use std::sync::Arc;
// Configure guardrails
let config = GuardrailsConfig::new()
.with_input(Arc::new(MaxLengthGuardrail::new(1000)))
.with_input(Arc::new(ForbiddenWordsGuardrail::new(vec![
"hack".into(), "exploit".into(),
])))
.with_output(Arc::new(SensitiveInfoGuardrail::new()));
// Wrap agent with guardrails
let mut guarded = GuardedAgent::new(
Arc::new(executor),
config,
);
let result = guarded.invoke("What is my password?".to_string()).await?;
// Returns Err(GuardrailError::Blocked { .. }) if guardrail triggers
// Check violations
let violations = guarded.violations();
}
SensitiveInfoGuardrail Details
Detects:
- Sensitive keywords: “password”, “token”, “secret”, “api_key”, “credential” (plus Chinese equivalents)
- OpenAI-style API keys (regex
sk-...) - Email addresses
- Credit card numbers (with Luhn validation)
#![allow(unused)]
fn main() {
let guardrail = SensitiveInfoGuardrail::new()
.with_keywords(vec!["internal_key".into()]);
}
Custom Guardrails
#![allow(unused)]
fn main() {
use langchainrust::{InputGuardrail, InputGuardrailResult};
struct NoSQLInjection;
#[async_trait]
impl InputGuardrail for NoSQLInjection {
fn name(&self) -> &str { "no_sql_injection" }
async fn validate(&self, input: &str) -> InputGuardrailResult {
if input.contains("{$") || input.contains("{$gt:") {
InputGuardrailResult::Block { reason: "Potential NoSQL injection".into() }
} else {
InputGuardrailResult::Pass
}
}
}
}
输入护栏返回 [
InputGuardrailResult](没有Modify变体),输出护栏返回 [OutputGuardrailResult](含Modify)。类型系统强制“修改结果仅输出侧可产生“。
Evaluation
LangChainRust provides 10 built-in evaluators and an LLM-as-judge for quantifying prompt and model changes.
Evaluators
| Evaluator | Type | Description |
|---|---|---|
ExactMatch | String | Returns 1.0 for exact match, 0.0 otherwise |
StringDistance | String | Levenshtein distance normalized to 0.0-1.0 |
ContainsKeyword | Rule | Checks for keyword presence (any or all) |
RegexMatch | Rule | Matches against a regex pattern |
LengthCheck | Rule | Validates output length within min/max bounds |
Bleu | NLP | BLEU score with character-level Chinese support |
EmbeddingSimilarity | Semantic | Cosine similarity of embeddings mapped to 0.0-1.0 |
LLMAsJudge | LLM | LLM scores predictions on a configurable rubric |
Faithfulness | LLM | Verifies claims against reference context |
PairwiseJudge | LLM | Compares two predictions with position-bias mitigation |
Core Types
#![allow(unused)]
fn main() {
pub struct Score { pub value: f64, pub label: Option<String> } // 0.0-1.0
pub struct Example {
pub input: String,
pub reference: String,
pub contexts: Vec<String>, // retrieved RAG contexts, rank order; [] for non-RAG tasks
}
pub struct Dataset { pub examples: Vec<Example> }
// Report 携带原文 + 汇总(均值/标准差)+ 失败清单,可反序列化(落盘二次分析)
pub struct Report {
pub per_example: Vec<ExampleReport>, // 每条含 input/reference/prediction 原文
pub summary: HashMap<String, ScoreSummary>, // 每个评测器: mean / std / count
pub failures: Vec<FailureRecord>, // 逐条容错记录(为空 = 全部成功)
}
pub struct ExampleReport {
pub index: usize,
pub input: String,
pub reference: String,
pub prediction: String,
pub scores: HashMap<String, Score>,
}
pub struct ScoreSummary { pub mean: f64, pub std: f64, pub count: usize }
}
Batch Evaluation
#![allow(unused)]
fn main() {
use langchainrust::{EvalRunner, ExactMatch, StringDistance, Dataset, Example, Evaluator};
let dataset = Dataset::new(vec![
Example::new("2+2?", "4"),
Example::new("Capital of France?", "Paris"),
]);
let runner = EvalRunner::new(vec![
Box::new(ExactMatch),
Box::new(StringDistance),
]);
let report = runner.run(&dataset, &predictor).await?;
// report.summary: {"exact_match": ScoreSummary { mean: 0.5, .. }, "string_distance": { .. }}
// 单条 predict / 打分失败不中断整批,记入 report.failures(P1-3)
}
Individual Evaluators
#![allow(unused)]
fn main() {
use langchainrust::{ContainsKeyword, RegexMatch, LengthCheck, Bleu, LLMAsJudge, Faithfulness};
// Rule-based
let eval = ContainsKeyword::new(vec!["Rust".into()]).all_required(true);
let score = eval.eval("What language?", "Rust is great", "").await?;
let eval = RegexMatch::new(r"\d+")?;
let score = eval.eval("Calculate", "The answer is 42", "").await?;
let eval = LengthCheck::new().min(10).max(100);
// BLEU
let eval = Bleu::new().with_char_level(true); // for Chinese
// LLM-as-judge
let eval = LLMAsJudge::new(judge_llm)
.with_rubric("Score accuracy on a scale of 0-10")
.with_max_score(10);
// Faithfulness (hallucination detection)
let eval = Faithfulness::new(judge_llm)
.with_llm_split(true) // LLM-based claim splitting
.with_empty_score(0.0);
}
Pairwise Comparison
#![allow(unused)]
fn main() {
use langchainrust::{PairwiseJudge, Verdict};
let judge = PairwiseJudge::new(llm).with_rubric("Which answer is more accurate?");
let verdict = judge.compare("What is X?", "Answer A", "Answer B").await?;
// Verdict::AWins, BWins, or Tie
}
PairwiseJudge 也实现了 PairwiseEvaluator trait(P1-1),可以直接进 EvalRunner
统一报告:成对评测器以 (prediction, reference) 作为 A/B 两个候选(竞技场用法把待比
答案放进 reference 槽),得分 1.0 = prediction 优、0.5 = 平局、0.0 = reference 优:
#![allow(unused)]
fn main() {
let runner = EvalRunner::new(vec![/* 单点评测器 */])
.with_pairwise(vec![Box::new(judge)]);
let report = runner.run(&dataset, &predictor).await?;
}
Trace → Golden → Regression Gate
Offline datasets go stale; the companion lc-testkit crate closes the loop from
production traces to a checked-in golden set to a CI regression gate. It is a
direct/dev dependency (lc-testkit = "0.24") rather than part of the facade.
The mapping from a recorded model exchange to one Example is fixed:
input← the last human message in the recorded request history (agent-loop re-requests still carry the real user turn there);reference← the recorded assistant text;contexts← non-empty tool-result messages in order, so RAGAS-style context precision/recall evaluators can score retrieval too.
Agent loops record several exchanges per user turn, including pure tool-call responses
with empty text — is_scoring_candidate filters those out (GoldenError names the row
that has no human message or empty reference).
// One recording file (from RecordingProvider) -> golden JSONL checked into the repo.
let exchanges = read_exchanges("traces/recorded.jsonl")?;
let scored: Vec<_> = exchanges.iter().filter(|e| is_scoring_candidate(e)).cloned().collect();
let dataset = golden_dataset(&scored)?;
write_golden_jsonl(&dataset, "eval/golden.jsonl")?;
// Fully offline smoke/baseline run: a FIFO ReplayPredictor over the same file
// reproduces every recorded reference exactly — zero network, no API key.
let (dataset, replay) = replay_golden_from_file("traces/recorded.jsonl", ReplayStrategy::Fifo)?;
let report = EvalRunner::new(vec![Box::new(ExactMatch)])
.run(&dataset, &replay)
.await?;
For the real regression run, load the checked-in JSONL (Dataset::from_jsonl) with a
predictor wrapping the new prompt/model, then compare the two reports:
#![allow(unused)]
fn main() {
use langchainrust::{compare_reports, Report};
// baseline = serialized Report from the known-good build (Report round-trips JSON).
let cmp = compare_reports(&baseline, &candidate, 0.02);
assert!(!cmp.is_regressed(), "regressed:\n{}", cmp.to_table());
}
A metric regresses when its mean drops by strictly more than the tolerance —
delta < -tolerance, so a boundary move of exactly -0.02 still passes. ReportComparison
also lists per-evaluator MetricDeltas plus added / dropped evaluator names, so a
candidate that silently stops running an evaluator is visible in the same table.
Verifier / Best-of-N
A judge answers “is this good?”; a verifier answers “how strongly does this satisfy the criteria?” with a score fine-grained enough to compare candidates.
#![allow(unused)]
fn main() {
use langchainrust::evaluation::{best_of_n, LogprobVerifier};
let verifier = LogprobVerifier::new(openai_chat) // requires a logprob-capable provider
.with_max_score(9)
.with_top_logprobs(5);
let candidates = vec![candidate_a, candidate_b, candidate_c];
let result = best_of_n(&verifier, "task description", &candidates).await?;
}
LogprobVerifier expands the first score-bearing token into its top_logprobs
distribution and takes the expectation instead of the argmax. For providers without
logprobs (Anthropic, most local servers), JudgeVerifier parses a single integer and
reports confidence = 0.25 so callers do not over-trust it.
Judge Reliability
Judge output needs position-bias and chance correction before it can gate a release:
#![allow(unused)]
fn main() {
use langchainrust::evaluation::{
cohen_kappa, order_swap_consistent, reliability_report, same_family, Verdict,
};
// A winner must survive both A/B and B/A.
let winner = order_swap_consistent(Verdict::AWins, Verdict::BWins); // AWins
// Chance-corrected agreement.
let kappa = cohen_kappa(&[true, false, true], &[true, false, false]);
// Same-family self-preference risk.
assert!(same_family("gpt-5", "gpt-4o"));
assert!(!same_family("gpt-5", "claude-3"));
let report = reliability_report(&original, &swapped);
}
cohen_kappa returns None for empty, mismatched or degenerate (single-label)
samples. reliability_report aggregates agreement, kappa and ties.
Tracing & Callbacks
LangChainRust provides a callback system for observability, tracing, and monitoring with handlers for console, file, LangSmith, and OpenTelemetry.
Callback System
| Component | Description |
|---|---|
CallbackHandler | Trait with lifecycle hooks (LLM, chain, tool, retriever) |
CallbackManager | Manages multiple handlers, dispatches events |
RunTree | Structured run tracking with parent-child relationships |
RunType | Llm, Chain, Tool, Retriever, Embedding, Prompt, Parser |
Callback Handlers
| Handler | Struct | Output |
|---|---|---|
| Console | StdOutHandler | Prints to stdout |
| File | FileCallbackHandler | Writes to file (Plain/JSON/JSONLines) |
| LangSmith | LangSmithHandler | Sends to LangSmith platform |
| OpenTelemetry | OtelHandler | Emits OTel spans with GenAI SemConv |
Setup
#![allow(unused)]
fn main() {
use langchainrust::{
CallbackManager, StdOutHandler, FileCallbackHandler,
LangSmithHandler, LogFormat,
};
use std::sync::Arc;
let manager = CallbackManager::new()
.add_handler(Arc::new(StdOutHandler::new()))
.add_handler(Arc::new(FileCallbackHandler::new("./traces.jsonl")?
.with_format(LogFormat::JsonLines)))
.add_handler(Arc::new(LangSmithHandler::from_env()?));
}
LangSmith Configuration
Environment variables:
| Variable | Description | Default |
|---|---|---|
LANGSMITH_API_KEY | API key (starts with ls_) | Required |
LANGSMITH_TRACING | Enable tracing | true |
LANGSMITH_PROJECT | Project name | default |
LANGSMITH_ENDPOINT | API endpoint | LangSmith official |
LANGSMITH_WORKSPACE_ID | Workspace ID | For org accounts |
OpenTelemetry (GenAI SemConv)
#![allow(unused)]
fn main() {
// Feature: opentelemetry
use langchainrust::OtelHandler;
let handler = OtelHandler::from_global("langchainrust");
// Sets GenAI Semantic Convention attributes:
// gen_ai.system, gen_ai.request.model, gen_ai.response.model,
// gen_ai.response.finish_reason, gen_ai.client.token.usage.*,
// gen_ai.request.max_tokens, gen_ai.request.temperature,
// gen_ai.operation.name, gen_ai.tool.name
}
Tracing Subsystem
#![allow(unused)]
fn main() {
use langchainrust::{Tracer, SpanKind, SpanGuard, InMemoryTracingBackend};
let backend = Arc::new(InMemoryTracingBackend::new());
let tracer = Tracer::new(backend.clone());
let mut span = tracer.start("llm_call", SpanKind::Llm);
span.with_tokens(SpanTokenUsage { prompt_tokens: 10, completion_tokens: 20, total_tokens: 30 });
span.end(); // RAII: ends on drop if not called
// Inspect traces
let spans = backend.spans();
}
RunTree
#![allow(unused)]
fn main() {
use langchainrust::{RunTree, RunType};
let run = RunTree::new("my_chain", RunType::Chain, serde_json::json!({"input": "hello"}))
.with_tag("production")
.with_project("my-project");
let child = run.create_child("llm_step", RunType::Llm, serde_json::json!({"prompt": "..."}));
}
LCEL Pipelines
The LangChain Expression Language (LCEL) provides pipe-based composition for Runnable components, inspired by Python LangChain’s prompt | llm | parser syntax.
Core Types
| Type | Description |
|---|---|
Runnable<I, O> | Base execution trait (invoke, batch, stream, transform) |
RunnableExt | Extension providing pipe() composition |
RunnableSequence<I, O> | Ordered pipeline of chained steps |
RunnableLambda<I, O> | Closure wrapper (sync/async) |
RunnablePassthrough<I> | Identity pass-through with true streaming |
RunnableParallel<I> | Fan-out/fan-in, results in HashMap<String, Value> |
RunnableBranch<I, O> | Conditional routing (first match wins) |
RunnableBinding<I, O> | Pre-bind config/kwargs to a Runnable |
RunnableWithFallbacks<I, O> | Primary + fallback chain |
RunnableAssign | Inject key-value pairs into HashMap mid-pipeline |
RunnableRetry<I, O> | Retry with exponential backoff |
Pipe Composition
#![allow(unused)]
fn main() {
use langchainrust::{RunnableExt, RunnableLambda, RunnablePassthrough};
// Chain runnables with pipe()
let chain = prompt.pipe(llm).pipe(parser);
// Lambda steps
let step1 = RunnableLambda::new_sync(|input: String| input.to_uppercase());
let step2 = RunnableLambda::new_sync(|input: String| format!("Result: {}", input));
let pipeline = step1.pipe(step2);
let result = pipeline.invoke("hello".to_string(), None).await?;
// "Result: HELLO"
}
Parallel & Branch
#![allow(unused)]
fn main() {
use langchainrust::{RunnableParallel, RunnableBranch, RunnablePassthrough};
// Parallel fan-out
let parallel = RunnableParallel::<String>::new()
.with("upper", RunnableLambda::new_sync(|s: String| s.to_uppercase()))
.with("len", RunnableLambda::new_sync(|s: String| s.len().to_string()));
let result = parallel.invoke("hello".to_string(), None).await?;
// HashMap {"upper": "HELLO", "len": "5"}
// Branch (conditional routing)
let branch = RunnableBranch::new(default_step)
.when(condition_runnable, branch_a)
.when(condition_runnable, branch_b);
}
Fallbacks & Retry
#![allow(unused)]
fn main() {
use langchainrust::{RetryConfig, RetryOn};
// Fallbacks
let chain = primary_llm.with_fallbacks(vec![fallback_llm1, fallback_llm2]);
// Retry with exponential backoff
let retry_config = RetryConfig::new(3)
.with_initial_delay(Duration::from_millis(500))
.with_max_delay(Duration::from_secs(10))
.with_backoff_multiplier(2.0)
.with_retry_on(RetryOn::TransientErrors); // 429, 500, 502, 503, 504
let chain = llm.with_retry(retry_config);
}
Assign & Binding
#![allow(unused)]
fn main() {
use langchainrust::RunnableAssign;
// Inject context into HashMap pipeline
let assign = RunnableAssign::new()
.with("context", retriever_step);
// Bind config
let bound = llm.pipe(RunnableConfig::new().with_tag("production"));
}
Stream Events
#![allow(unused)]
fn main() {
use langchainrust::LcelStreamEvent;
// LCEL streaming via transform()
let stream = chain.transform(input_stream, config).await?;
while let Some(event) = stream.next().await {
match event? {
LcelStreamEvent::OnLlmStart { .. } => { /* ... */ }
LcelStreamEvent::OnLlmStream { token } => print!("{}", token),
LcelStreamEvent::OnLlmEnd { .. } => { /* ... */ }
LcelStreamEvent::OnToolEnd { output } => { /* ... */ }
LcelStreamEvent::OnChainEnd { output } => { /* ... */ }
}
}
}
Streaming
LangChainRust supports streaming at multiple levels: LLM token streaming, agent event streaming, chain streaming, and LCEL pipeline streaming.
Streaming Levels
| Level | Method | Output |
|---|---|---|
| LLM | stream_chat() | Token-by-token text |
| Agent | AgentExecutor::stream() | AgentStreamEvent stream |
| Chain | BaseChain::stream() | StreamToken stream |
| LCEL | Runnable::stream() / transform() | LcelStreamEvent stream |
| Structured Output | stream_structured_output() | Partial T values |
LLM Streaming
#![allow(unused)]
fn main() {
use langchainrust::openai::OpenAIChat;
let llm = OpenAIChat::new(OpenAIConfig { streaming: true, ..Default::default() });
let messages = vec![
Message::system("You are a helpful assistant."),
Message::human("Explain Rust ownership."),
];
let mut stream = llm.stream_chat(messages, None).await?;
while let Some(chunk) = stream.next().await {
let token = chunk?;
print!("{}", token.content);
}
}
Agent Streaming
#![allow(unused)]
fn main() {
use langchainrust::AgentExecutor;
let stream = executor.stream(input);
while let Some(event) = stream.next().await {
match event? {
AgentStreamEvent::Text { content } => print!("{}", content),
AgentStreamEvent::ToolStart { name, input } => {
println!("[Tool: {}] Input: {}", name, input);
}
AgentStreamEvent::ToolEnd { name, output } => {
println!("[Tool: {}] Output: {}", name, output);
}
AgentStreamEvent::FinalAnswer { content } => {
println!("Answer: {}", content);
}
AgentStreamEvent::PipelineStep { step, detail } => { /* ... */ }
AgentStreamEvent::Error { message } => eprintln!("Error: {}", message),
}
}
}
Chain Streaming
#![allow(unused)]
fn main() {
use langchainrust::{LLMChain, StreamToken};
let chain = LLMChain::new(llm, "Explain: {topic}");
let mut stream = chain.stream(inputs).await?;
while let Some(token) = stream.next().await {
match token? {
StreamToken { token, is_final: false } => print!("{}", token),
StreamToken { token, is_final: true } => println!("\n[Done]"),
}
}
}
LCEL Streaming
#![allow(unused)]
fn main() {
use langchainrust::LcelStreamEvent;
let stream = pipeline.stream(input, None).await?;
while let Some(event) = stream.next().await {
match event? {
LcelStreamEvent::OnLlmStream { token } => print!("{}", token),
LcelStreamEvent::OnToolEnd { output } => println!("Tool: {}", output),
LcelStreamEvent::OnChainEnd { output } => println!("Chain: {:?}", output),
_ => {}
}
}
}
Structured Output Streaming
#![allow(unused)]
fn main() {
use langchainrust::StreamingStructuredOutputExt;
let stream = llm.stream_structured_output::<MyStruct>(schema, "prompt").await?;
while let Some(partial) = stream.next().await {
let partial: MyStruct = partial?;
// Fields populate incrementally as tokens arrive
}
}
Tool Macro
The #[tool] procedural macro auto-generates BaseTool + Tool implementations from a simple function, reducing ~50 lines of boilerplate to 3 lines.
Basic Usage
#![allow(unused)]
fn main() {
use langchainrust::{tool, BaseTool, Tool, ToolError};
#[tool(description = "Greets a person by name")]
fn greet(
#[param(desc = "The name of the person to greet")]
name: String,
) -> Result<String, ToolError> {
Ok(format!("Hello, {}!", name))
}
}
This generates:
GreetToolstruct (withnew()andDefault)GreetInputstruct (withDeserialize+JsonSchema)impl Tool for GreetTool(type-safeinvoke)impl BaseTool for GreetTool(string-basedrun,name,description,args_schema)- The original
greetfunction is preserved
Parameters
#![allow(unused)]
fn main() {
#[tool(description = "Adds two numbers")]
fn add_numbers(
#[param(desc = "The first number")]
a: i64,
#[param(desc = "The second number")]
b: i64,
) -> Result<i64, ToolError> {
Ok(a + b)
}
// Generates: AddNumbersTool, AddNumbersInput
}
Parameter type rules:
| Type | Schema | Required |
|---|---|---|
String, i64, f64, bool | Primitive | Yes |
Option<T> | Nullable | No |
Vec<T> | Array | Yes |
Optional Parameters
#![allow(unused)]
fn main() {
#[tool(description = "Says hello with optional title")]
fn hello_optional(
#[param(desc = "The person's name")]
name: String,
#[param(desc = "Optional title (e.g., Dr., Mr.)")]
title: Option<String>,
) -> Result<String, ToolError> {
match title {
Some(t) => Ok(format!("Hello, {} {}!", t, name)),
None => Ok(format!("Hello, {}!", name)),
}
}
// Generates: HelloOptionalTool, HelloOptionalInput
}
Using Generated Tools
#![allow(unused)]
fn main() {
// Type-safe invocation via Tool trait
let tool = GreetTool::new();
let input = GreetInput { name: "Alice".into() };
let result = tool.invoke(input).await?;
assert_eq!(result, "Hello, Alice!");
// String-based invocation via BaseTool trait (for agents)
let tool = GreetTool::new();
assert_eq!(tool.name(), "greet");
assert_eq!(tool.description(), "Greets a person by name");
let result = tool.run(r#"{"name":"Bob"}"#.to_string()).await?;
assert_eq!(result, "\"Hello, Bob!\"");
// Get JSON schema for LLM function calling
let schema = tool.args_schema(); // Auto-generated from JsonSchema
}
Attributes Reference
| Attribute | Target | Required | Description |
|---|---|---|---|
#[tool(description = "...")] | Function | Yes | Tool description shown to the LLM |
#[param(desc = "...")] | Parameter | No | Parameter description in JSON schema |
The struct and input types are named by converting the function name to PascalCase and appending Tool or Input (e.g., add_numbers becomes AddNumbersTool / AddNumbersInput).
Hooks
Hooks provide a composable middleware mechanism for intercepting agent lifecycle events: completions, tool calls, streaming, and errors.
AgentHook Trait
#![allow(unused)]
fn main() {
#[async_trait]
pub trait AgentHook: Send + Sync {
fn on_before_completion(&self, ctx: &mut CompletionContext) -> CompletionAction;
fn on_after_completion(&self, ctx: &mut CompletionResult) -> Result<(), HookError>;
fn on_before_tool_call(&self, ctx: &mut ToolCallContext) -> ToolCallAction;
fn on_after_tool_call(&self, ctx: &mut ToolResultContext) -> Result<(), HookError>;
fn on_stream_chunk(&self, chunk: &str) -> StreamAction;
fn on_agent_start(&self, input: &str) -> Result<(), HookError>;
fn on_agent_end(&self, output: &str) -> Result<(), HookError>;
fn on_error(&self, error: &HookError) -> ErrorAction;
}
}
All methods have default no-op implementations – implement only what you need.
Built-in Hooks
| Hook | Struct | Purpose |
|---|---|---|
| Approval | ApprovalHook | Require manual approval before tool calls |
| Content Filter | ContentFilterHook | Filter sensitive words from stream output |
| Logging | LoggingHook | Log all lifecycle events |
| Prompt Injection | PromptInjectionHook | Detect & sanitize prompt injection in tool outputs |
| Token Budget | TokenBudgetHook | Enforce token budget / LLM call quota (rate limiting) |
ApprovalHook
#![allow(unused)]
fn main() {
use langchainrust::hooks::ApprovalHook;
// Manual approval (default) -- rejects tool calls
let hook = ApprovalHook::new();
// Auto-approve all tool calls
let hook = ApprovalHook::auto_approve();
// Configurable
let hook = ApprovalHook::new().with_auto_approve(false);
}
ContentFilterHook
#![allow(unused)]
fn main() {
use langchainrust::hooks::ContentFilterHook;
let hook = ContentFilterHook::new(vec![
"password".into(), "secret".into(), "internal".into(),
])
.with_placeholder("[REDACTED]") // Replace sensitive words
.with_drop_token(false); // If true, filter entire chunk instead of replacing
}
PromptInjectionHook
Detects and sanitizes prompt-injection text in tool outputs. A malicious page /
retrieved document that says “ignore previous instructions” is replaced with a
safe marker before it can reach the next plan() prompt, breaking cross-round
pollution.
#![allow(unused)]
fn main() {
use langchainrust::hooks::PromptInjectionHook;
let hook = PromptInjectionHook::new(); // Default injection patterns
// Custom patterns and marker
let hook = PromptInjectionHook::new()
.with_patterns(vec!["evil-text".into()])
.with_marker("[BLOCKED:{}]"); // `{}` is replaced with the matched pattern
let detected = hook.detected_count(); // Total injections sanitized so far
}
TokenBudgetHook
Enforces a token budget and/or an LLM call quota before each completion.
on_before_completion rejects (aborting the run) when the estimated usage would
exceed the budget; on_after_completion accumulates real token usage.
#![allow(unused)]
fn main() {
use langchainrust::hooks::TokenBudgetHook;
let hook = TokenBudgetHook::new(10_000) // Total token budget for the run
.with_max_calls(20); // Optional: max LLM calls
}
LoggingHook
#![allow(unused)]
fn main() {
use langchainrust::hooks::LoggingHook;
let hook = LoggingHook::new(); // Basic logging
let hook = LoggingHook::with_tokens(); // Include token usage logging
}
Adding Hooks to Agent
#![allow(unused)]
fn main() {
use langchainrust::{AgentExecutor, BaseAgent};
use langchainrust::hooks::{ApprovalHook, ContentFilterHook, LoggingHook};
let executor = AgentExecutor::new(agent, tools)
.hook(ApprovalHook::new()) // Require approval
.hook(ContentFilterHook::new(vec!["secret".into()])) // Filter words
.hook(LoggingHook::with_tokens()); // Log events
}
Tool Permission Policy
Beyond hooks, AgentExecutor can enforce a tool permission policy before every
tool execution — two gates:
- Permission tier (
ToolRisk):Safe<Standard<Dangerous. A tool whose risk exceedsmax_permittedis rejected. - Sandbox gate: a
Dangeroustool must be declared as having been moved into a sandboxed environment (ToolPolicy::sandboxed) or it is rejected, preventing dangerous tools from running un-sandboxed. (Actual sandbox backends live inlc-tools—SandboxTool/CodeSandbox— which the operator wires onto the tool.)
#![allow(unused)]
fn main() {
use langchainrust::{AgentExecutor, ToolPolicy, ToolRisk};
let policy = ToolPolicy::new()
.risk("code_interpreter", ToolRisk::Dangerous) // Declare risk tier
.sandboxed("code_interpreter") // Declare it runs in a sandbox
.with_max_permitted(ToolRisk::Standard); // Gate: reject riskier tools
let executor = AgentExecutor::new(agent, tools).with_tool_policy(policy);
}
A rejected tool call aborts the run with an AgentError. When no policy is
configured, tool execution is unrestricted.
Action Types
| Action | Hook | Variants |
|---|---|---|
CompletionAction | on_before_completion | Continue, Modify { messages }, Reject { reason } |
ToolCallAction | on_before_tool_call | Continue, Modify { name, arguments }, Reject { reason }, Skip |
StreamAction | on_stream_chunk | Forward(String), Filter, Replace(String) |
ErrorAction | on_error | Propagate, Retry, Ignore |
Custom Hook
#![allow(unused)]
fn main() {
use langchainrust::hooks::{AgentHook, ToolCallAction, ToolCallContext, StreamAction};
struct RateLimitHook { max_calls: usize, calls: AtomicUsize }
#[async_trait]
impl AgentHook for RateLimitHook {
fn on_before_tool_call(&self, _ctx: &mut ToolCallContext) -> ToolCallAction {
if self.calls.fetch_add(1, Ordering::SeqCst) >= self.max_calls {
ToolCallAction::Reject { reason: "Rate limit exceeded".into() }
} else {
ToolCallAction::Continue
}
}
}
}
Multimodal
LangChainRust supports multimodal interactions including audio transcription (Whisper), text-to-speech (TTS), image generation (DALL-E), and vision with images.
MultimodalModel Trait
#![allow(unused)]
fn main() {
#[async_trait]
pub trait MultimodalModel: BaseChatModel + Send + Sync {
async fn transcribe(&self, audio: AudioContent) -> Result<String, MultimodalError>;
async fn generate_speech(&self, text: &str) -> Result<Vec<u8>, MultimodalError>;
async fn generate_image(&self, prompt: &str) -> Result<ImageContent, MultimodalError>;
}
}
All methods default to returning MultimodalError::Unsupported. Only OpenAI implements them currently.
Content Types
| Type | Construction | Description |
|---|---|---|
AudioContent | from_url(), from_base64() | Audio input for transcription |
ImageContent | from_url(), from_base64() | Image URL or base64 data |
FileContent | from_url(), from_base64(), with_name() | Generic file with MIME type |
#![allow(unused)]
fn main() {
use langchainrust::{AudioContent, ImageContent, FileContent};
let audio = AudioContent::from_url("https://example.com/audio.wav");
let audio = AudioContent::from_base64("base64data...");
let image = ImageContent::from_url("https://example.com/photo.jpg");
let image = ImageContent::from_base64("iVBORw0KGgo...");
let file = FileContent::from_url("https://example.com/doc.pdf")
.with_name("report.pdf");
}
Whisper (Speech-to-Text)
#![allow(unused)]
fn main() {
use langchainrust::MultimodalModel;
let audio = AudioContent::from_base64(audio_base64);
let transcript = llm.transcribe(audio).await?;
}
OpenAI-specific with voice options:
#![allow(unused)]
fn main() {
use langchainrust::openai::OpenAIChat;
// Provider-specific method with more control
let transcript = llm.whisper_transcribe(audio).await?;
}
TTS (Text-to-Speech)
#![allow(unused)]
fn main() {
use langchainrust::MultimodalModel;
let speech = llm.generate_speech("Hello, world!").await?;
// Provider-specific with voice selection
use langchainrust::openai::TtsVoice;
let speech = llm.tts_generate("Hello!", TtsVoice::Alloy).await?;
// Voices: Alloy, Echo, Fable, Onyx, Nova, Shimmer
}
DALL-E (Image Generation)
#![allow(unused)]
fn main() {
use langchainrust::MultimodalModel;
let image = llm.generate_image("A cat wearing a hat").await?;
// Provider-specific with size control
use langchainrust::openai::DallEImageSize;
let image = llm.dalle_generate("A cat wearing a hat", DallEImageSize::S1024).await?;
// Sizes: S256, S512, S1024, S1792x1024, S1024x1792
}
Vision (Image Understanding)
#![allow(unused)]
fn main() {
use langchainrust::Message;
let messages = vec![
Message::system("Describe what you see."),
Message::human_with_image("What is in this image?", "https://example.com/cat.jpg"),
];
let response = llm.chat(messages, None).await?;
}
Anthropic Images
Anthropic supports image inputs via the same Message::human_with_image pattern. The image content is formatted according to the Anthropic API specification.
Sessions
Sessions provide multi-turn conversation lifecycle management: create, retrieve, archive, and chat with automatic history maintenance.
⚠️ v0.22.0: the event-sourcing path (
EventSessionManager) is the recommended API. The legacySessionManageris#[deprecated]— removal was planned for 0.23.0, but as of 0.24.0 it still ships and compiles;#[allow(deprecated)]silences the warnings. See Event Sourcing (v0.22.0) below.
Core Types
| Type | Description |
|---|---|
SessionManager | Legacy API (deprecated in 0.22.0; removal was planned for 0.23.0 but it still ships in 0.24.0) |
EventSessionManager | Recommended event-sourced manager (v0.22.0) |
Session | A conversation with messages, metadata, and status |
SessionStore | Trait for pluggable storage backends (legacy) |
EventStore | Append-only event log trait (v0.22.0) |
MemorySessionStore / MemoryEventStore | In-memory implementations |
SessionStatus | Active, Archived, Deleted |
Basic Usage
#![allow(unused)]
fn main() {
use langchainrust::{SessionManager, MemorySessionStore};
use std::sync::Arc;
let manager = SessionManager::new(Arc::new(MemorySessionStore::new()));
// Create a new session
let session_id = manager.create_session().await?;
// Create a session for a specific user
let session_id = manager.create_session_for("user_123").await?;
// Chat (auto-maintains history)
let reply = manager.chat(&session_id, &llm, "What is Rust?".to_string()).await?;
let reply2 = manager.chat(&session_id, &llm, "Tell me more about ownership.".to_string()).await?;
// Get conversation history
let messages = manager.history(&session_id).await?;
// Archive or clear
manager.archive(&session_id).await?;
manager.clear(&session_id).await?;
// List sessions for a user
let sessions = manager.list_by_user("user_123").await?;
}
Session Struct
#![allow(unused)]
fn main() {
pub struct Session {
pub id: String,
pub user_id: Option<String>,
pub messages: Vec<Message>,
pub metadata: HashMap<String, Value>,
pub created_at: DateTime<Utc>,
pub updated_at: DateTime<Utc>,
pub status: SessionStatus,
}
}
Methods: add_message(), recent_messages(n), clear(), archive()
Custom SessionStore
#![allow(unused)]
fn main() {
use langchainrust::{SessionStore, Session, SessionError};
struct RedisSessionStore { /* ... */ }
#[async_trait]
impl SessionStore for RedisSessionStore {
async fn create(&self, session: Session) -> Result<String, SessionError>;
async fn get(&self, id: &str) -> Result<Option<Session>, SessionError>;
async fn update(&self, session: &Session) -> Result<(), SessionError>;
async fn delete(&self, id: &str) -> Result<(), SessionError>;
async fn list_by_user(&self, user_id: &str) -> Result<Vec<Session>, SessionError>;
}
}
SessionManager Methods
| Method | Description |
|---|---|
create_session() | Create an anonymous session |
create_session_for(user_id) | Create a session for a user |
get_session(id) | Get session by ID |
chat(id, llm, message) | Send message and get reply (auto-persists) |
history(id) | Get conversation messages |
clear(id) | Clear messages |
archive(id) | Archive session |
list_by_user(user_id) | List sessions for a user |
Event Sourcing (v0.22.0)
The legacy manager is “rewrite-style”: each turn reads the whole Session, mutates it, and writes it back — a crash mid-way loses messages, and there is no way to fork or time-travel. 0.22.0 rewrites sessions as event sourcing: the session is an append-only event log; history is a projection (replay) of the log.
#![allow(unused)]
fn main() {
use langchainrust::sessions::{EventSessionManager, MemoryEventStore, AutoCompaction};
use std::sync::Arc;
let manager = EventSessionManager::new(Arc::new(MemoryEventStore::new()))
.with_max_context_turns(10) // turn window (default: full history)
.with_auto_compaction(AutoCompaction::new(20)?); // snapshot beyond 20 turns
let id = manager.create_session().await?; // uuid v7
let reply = manager.chat(&id, &llm, "What is Rust?".to_string()).await?;
let history = manager.history(&id).await?; // projected Vec<Message>
// clear / archive / delete: append Metadata events to the EventStore directly
}
Old API → New API Mapping
| Old (0.21.x) | New (0.22.0) | Note |
|---|---|---|
SessionManager::new(Arc<dyn SessionStore>) | EventSessionManager::new(Arc<dyn EventStore>) | storage becomes an append-only log |
create_session_for(user) | create_session() (uuid v7) | ownership via Metadata events |
chat(&id, &llm, msg) | same signature | persistence changes from rewrite to append — crash-safe |
history(&id) | history(&id) / replay_session(&id) | replay_session yields the old mutable Session view |
| — | fork_session(&id, branch, until) | new: branch from any point in the log |
max_context_messages(n) | with_max_context_turns(n) | turn window; user/ai always paired |
| — | with_auto_compaction(AutoCompaction) | new: deterministic snapshot past N turns (no LLM call) |
clear / archive / delete_session | append Metadata events to the EventStore directly (convenience wrappers planned for 0.23.0 had not landed as of 0.24.0) | the log is immutable; cleanup is a state event |
SessionStore | EventStore | append / append_batch / read / fork; idempotency key (session, branch, id) |
Fork
#![allow(unused)]
fn main() {
// Copy the prefix up to some event sequence number into a new branch;
// the trunk is unaffected. Returns a manager pinned to the branch.
let mut experiment = manager.fork_session(&id, "experiment", None).await?;
experiment.chat(&id, &llm, "branch message".to_string()).await?;
// Re-forking the same (session, branch) is idempotent
}
Crash Safety
appendis idempotent per(session_id, branch, id)— replaying a half-written batch after a crash never duplicates events.project()/to_session()rebuild state from the log; orphan tool results are detected and rejected, so the history fed to the LLM is always properly paired.SessionCheckpointtrait +NoopCheckpointare the placeholder for durable checkpoints: durable projections were planned for 0.23.0, but as of 0.24.0 the no-op implementation is still the only one — recovery replays the full event log.
Event Types
EventPayload variant | Meaning |
|---|---|
SessionCreated | session established (uuid v7) |
UserMessage / AssistantMessage | one Turn per round |
ToolCall / ToolResult | appended as a pair |
Snapshot | deterministic compaction snapshot |
Metadata | lifecycle / ownership key-value state |
A2A Protocol
The A2A (Agent-to-Agent) protocol is Google’s open standard for agent interoperability — letting agents built by different teams and vendors discover and call each other. LangChainRust provides both a server and client implementation over JSON-RPC 2.0.
Layering: A2A handles “Agent ↔ Agent” communication; MCP handles “Agent ↔ tools / data sources.” They compose: MCP supplies tools, agents collaborate over A2A.
Feature Overview
| Feature | Type | Description |
|---|---|---|
A2AServer | Server | Expose your agent as an A2A service (handler functions, pluggable into any HTTP framework) |
A2AClient | Client | Discover and call remote agents (send_task / get_task / cancel_task) |
AgentCard | Protocol | Agent self-description card; v1.0.1 supportedInterfaces[] |
AgentInterface / A2ATransport | Protocol | Per-interface (protocolVersion, transport, url) + optional tenant tag |
negotiate / is_v101 | Protocol | Client/server transport + version negotiation |
sign_agent_card / verify_card_signature | Security | Card signing (RFC 8785-lite canonicalization, HMAC) stored in card.signature |
sign_card_jws / verify_card_jws | Security | Compact JWS HS256 card signing (header-friendly) |
AgentRegistry / RegistryClient | Discovery | Registry-based multi-agent discovery |
FederationGateway | Gateway | Cross-org federation with call policies and data contracts |
ResilientA2AClient | Client | Retries / timeouts / circuit breaking wrapper |
SkillRouter / SkillMapRouter | Client | Route tasks to agents by skill |
Agent Card (v1.0.1)
A v1.0.1 card declares multiple interfaces instead of a single protocol binding:
#![allow(unused)]
fn main() {
use langchainrust::a2a::protocol::{AgentCard, AgentInterface, A2ATransport, A2A_VERSION_V101};
let card = AgentCard::new("agent-a", "A helpful agent", "https://a.example")
.with_supported_interface(AgentInterface::new(
A2A_VERSION_V101, // "1.0.1", a separate protocolVersion per interface
A2ATransport::HttpJson, // JsonRpc | HttpJson | Grpc
"https://a.example/a2a",
))
.with_supported_interface(
AgentInterface::new("1.0.1", A2ATransport::JsonRpc, "https://a.example/a2a/jsonrpc")
.with_tenant("acme"), // enterprise multi-tenancy (optional)
);
assert!(card.is_v101());
}
The old v0.3 fields are kept for reader compatibility; new cards should fill supportedInterfaces and treat the old fields as fallback.
Negotiation
The client picks a mutually supported interface given the transport it wants and the versions it speaks; a miss is a clear error:
#![allow(unused)]
fn main() {
let picked = card.negotiate(A2ATransport::HttpJson, &["1.0.1", "1.0"])?;
}
Card Signing
The Agent Card drives discovery — a tampered card (swapped endpoint, downgraded protocol) is a supply-chain attack. Signing canonicalizes the card JSON (RFC 8785-lite: recursive key ordering) and applies HMAC-SHA256.
#![allow(unused)]
fn main() {
use langchainrust::a2a::client::{sign_agent_card, verify_card_signature, sign_card_jws, verify_card_jws};
use langchainrust::a2a::protocol::AgentCard;
let secret = b"shared-secret-between-registrar-and-clients";
// In-place signing, hex-encoded into card.signature
let mut card = AgentCard::new("agent-a", "A helpful agent", "https://a.example");
sign_agent_card(&mut card, secret)?;
// Verify — tampering with any field fails
verify_card_signature(&card, secret)?;
// Compact JWS token (header-friendly)
let jws = sign_card_jws(&card, secret)?;
verify_card_jws(&card, &jws, secret)?;
}
Boundaries: HS256 symmetric only (ES256 planned 0.22.1); canonicalization is RFC 8785-lite, not full JCS; expiry rides the card’s expiresAt field.
A2A Server (Expose Your Agent)
A2AServer provides handler functions you plug into any HTTP framework (axum, actix, warp) — it does NOT start its own listener.
#![allow(unused)]
fn main() {
use langchainrust::a2a::A2AServer;
use langchainrust::LLMChain;
use std::sync::Arc;
let chain = Arc::new(LLMChain::new(llm, "You are a helpful assistant"));
let server = A2AServer::new(chain)
.with_card(card);
// In your HTTP handlers:
// GET /.well-known/agent-card.json → server.get_agent_card()
// POST / → server.handle_a2a_request(body).await
}
Endpoints: GET /.well-known/agent-card.json (discovery) and POST / (JSON-RPC requests). Tasks from tasks/send are stored in an in-memory RwLock<HashMap> — wrap with a database-backed TaskStore for production.
A2A Client (Call Remote Agent)
#![allow(unused)]
fn main() {
use langchainrust::a2a::{A2AClient, A2AMessage};
let client = A2AClient::new("http://remote-agent:8080".to_string()).unwrap();
// Discover
let card = client.get_agent_card().await?;
// Dispatch / poll / cancel
let task = client.send_task(A2AMessage::user("hello")).await?;
let task = client.get_task(&task.id).await?;
let task = client.cancel_task(&task.id).await?;
}
Protocol Flow
| Step | Operation | Role |
|---|---|---|
| 1. Discover | get_agent_card fetches the remote Agent Card | Client |
| 2. Dispatch | send_task submits a task (A2AMessage) | Client → Server |
| 3. Progress | get_task queries status and result by task ID | Client |
| 4. Cancel | cancel_task cancels an unfinished task | Client |
Status Matrix
| Capability | Status |
|---|---|
tasks/send / tasks/get / tasks/cancel | Implemented |
| Agent Card discovery | Implemented |
v1.0.1 supportedInterfaces + negotiation | Implemented (0.22.0) |
| Card signing JWS HS256 | Implemented (0.22.0; ES256 / full JCS → 0.22.1) |
| API key / Static Bearer auth | Implemented |
| gRPC binding | → 0.22.1 (tonic codegen needs protoc) |
| OAuth / OIDC | Planned (0.22.1) |
| Task state machine / streaming push / TLS | Planned |
A deployment walkthrough lives at docs/internal/deployment/a2a-http-server-deployment.md; the runnable example is crates/lc/examples/a2a_http_server.rs.
Migration: v0.7 to v0.8
Version 0.8.0 introduced a major workspace restructure, migrating from a single-crate architecture to a multi-crate workspace. This guide covers what changed and how to migrate.
What Changed
Before (v0.7): Single Crate
All code lived in src/ of a single langchainrust crate.
After (v0.8): Workspace Architecture
The project is now organized as a Cargo workspace with a facade crate:
langchainrust/
├── crates/
│ ├── lc/ # Facade crate (package: langchainrust)
│ ├── lc-core/ # Runnable, BaseTool, output parsers, structured output
│ ├── lc-schema/ # Message types
│ ├── lc-shared/ # Shared utilities, Document type
│ ├── lc-providers/ # OpenAI, Ollama, DeepSeek, Anthropic, Gemini, etc.
│ ├── lc-agents/ # ReAct, FunctionCalling, PlanExecute, CRAG, etc.
│ ├── lc-chains/ # LLMChain, SequentialChain, RouterChain, etc.
│ ├── lc-memory/ # Buffer, Window, Summary, Persistent
│ ├── lc-embeddings/ # OpenAI, DeepSeek, Qwen, Local, Mock
│ ├── lc-vector-stores/ # InMemory, Qdrant, ChromaDB, Pinecone, etc.
│ ├── lc-rag/ # RAGPipeline, BM25, Hybrid, HyDE, GraphRAG
│ ├── lc-prompts/ # PromptTemplate, ChatPromptTemplate
│ ├── lc-callbacks/ # StdOut, LangSmith, File, OTel
│ ├── lc-tools/ # Calculator, DateTime, PythonREPL, etc.
│ ├── lc-tools-derive/ # #[tool] procedural macro
│ ├── lc-evaluation/ # 10 evaluators
│ ├── lc-guardrails/ # Input/output safety
│ ├── lc-sessions/ # Session management
│ ├── lc-langgraph/ # StateGraph, CompiledGraph
│ ├── lc-mcp/ # MCP client/server
│ └── lc-a2a/ # Agent-to-Agent protocol
Import Compatibility
The facade crate (lc) re-exports all public types under langchainrust::, so most import paths are unchanged:
#![allow(unused)]
fn main() {
// These still work exactly as before:
use langchainrust::OpenAIChat;
use langchainrust::agents::ReActAgent;
use langchainrust::memory::ConversationBufferMemory;
use langchainrust::chains::LLMChain;
}
Breaking Changes
Feature Flags
Some features moved or were renamed. Update your Cargo.toml:
| Old (v0.7) | New (v0.8) |
|---|---|
features = ["qdrant"] | features = ["qdrant-integration"] |
features = ["mongo"] | features = ["mongodb-persistence"] |
features = ["redis"] | features = ["redis-storage"] |
features = ["sqlite"] | features = ["sqlite-storage"] |
Error Types
Error types are now defined in their respective sub-crates. The facade re-exports them, so most code compiles unchanged. If you referenced internal error paths directly, update to the re-exported names.
Internal Module Paths
If you used langchainrust::internal_module::... paths, they may have moved. Prefer the top-level re-exports:
#![allow(unused)]
fn main() {
// PREFER:
use langchainrust::{RAGPipeline, BM25Retriever, GraphRAG};
// AVOID (internal path may change):
use langchainrust::retrieval::pipeline::RAGPipeline;
}
New in v0.8+
- LCEL (v0.9):
pipe(),RunnableSequence,RunnableLambda,RunnableParallel,RunnableBranch,RunnableBinding #[tool]macro (v0.10): Auto-generate tool implementations from functions- Fallbacks (v0.10):
RunnableWithFallbacksfor resilient pipelines - Assign (v0.10):
RunnableAssignfor injecting context into HashMap pipelines - Document chain streaming (v0.10): Stuff, Refine, MapReduce chains support streaming
Dependency Update
# Before (v0.7)
[dependencies]
langchainrust = "0.7"
# After (v0.8+)
[dependencies]
langchainrust = "0.11" # Facade crate, all features available via feature flags