Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

LangChainRust

A LangChain-inspired Rust framework for building production LLM applications: agents, RAG, memory, MCP, A2A, guardrails, evaluation, and LangGraph workflows — all in pure Rust.

Start here

Why this framework

  • Pure Rust — one binary, no Python runtime
  • Production-first — durable execution, approvals, budgets, guardrails, audit
  • Protocol-complete — MCP, A2A, and an OpenAI-compatible endpoint
  • Verifiable — evaluators, verifiers, regression gates

Quick Start

From zero to a working agent in 5 minutes.

Prerequisites

  • Rust 1.85+
  • An API key for OpenAI or any OpenAI-compatible endpoint

1. Rust: an agent with a tool

export OPENAI_API_KEY=sk-...
export OPENAI_MODEL=gpt-4o-mini   # optional
cargo run -p langchainrust --example quickstart_agent

The example (crates/lc/examples/quickstart_agent.rs) builds a FunctionCallingAgent with Calculator, runs one tool-calling loop, and prints the answer:

use std::sync::Arc;

use langchainrust::agents::{AgentExecutor, FunctionCallingAgent};
use langchainrust::core::tools::BaseTool;
use langchainrust::tools::Calculator;
use langchainrust::{OpenAIChat, OpenAIConfig};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let llm = OpenAIChat::new(OpenAIConfig::from_env_result()?);
    let tools: Vec<Arc<dyn BaseTool>> = vec![Arc::new(Calculator::new())];

    let agent = FunctionCallingAgent::new(
        llm,
        tools.clone(),
        Some("You are a concise assistant. Use tools when needed.".to_string()),
    );

    let executor = AgentExecutor::new(Arc::new(agent), tools).with_max_iterations(5);
    println!("{}", executor.invoke("What is 25 * 37?".to_string()).await?);
    Ok(())
}

2. Python: call the runtime over HTTP

Terminal 1 — start the OpenAI-compatible server:

cargo run -p langchainrust --example openai_server --features openai-server
# OpenAI-compatible server on http://127.0.0.1:8080

Terminal 2 — install the thin client and call it:

pip install -e sdks/python

python - <<'PY'
from langchainrust import Client

c = Client("http://127.0.0.1:8080", model="gpt-4o-mini")
print(c.chat("What is Rust?").content)
PY

Any OpenAI SDK works against the same endpoint:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="unused")
print(client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "hi"}],
))

3. TypeScript: the same endpoint

npm install langchainrust
# or, from this repository: npm install ./sdks/typescript
import { Client } from "langchainrust";

const c = new Client("http://127.0.0.1:8080", "gpt-4o-mini");
console.log((await c.chat("What is Rust?")).content);

Troubleshooting

SymptomFix
OPENAI_API_KEY errorExport the key; the config fails fast when it is missing
Port already in useLC_ADDR=127.0.0.1:9090 before starting the server
unknown feature openai-serverAdd --features openai-server
Python ModuleNotFoundErrorRun pip install -e sdks/python from the repo root

Next

  • Agents — hooks, budgets, approvals, subagents
  • MCP — expose and consume tools
  • A2A — agent-to-agent
  • Evaluation — verifiers and regression gates

Installation

Cargo

[dependencies]
langchainrust = "0.30.0"

Feature Flags

LangChainRust uses feature flags to keep the default build lean:

[dependencies.langchainrust]
version = "0.30.0"
features = ["qdrant-integration"]  # Optional features

Available Features

FeatureDescription
qdrant-integrationQdrant vector store
mongodb-persistenceMongoDB document store
redis-storageRedis document store
sqlite-storageSQLite document store
pgvector-storagePGVector vector store
local-embeddingsONNX Runtime local embeddings
fastembedFastEmbed local embeddings (ONNX)
openai-serverOpenAI-compatible HTTP server (/v1/models, /v1/chat/completions)

Individual Crates

You can also use individual crates directly:

[dependencies]
lc-core = "0.30.0"
lc-providers = "0.30.0"
lc-agents = "0.30.0"

Environment Variables

Most providers require API keys set as environment variables:

export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...
export MISTRAL_API_KEY=...
export COHERE_API_KEY=...
export DEEPSEEK_API_KEY=...
export QWEN_API_KEY=...

See also

LLM Providers

LangChainRust supports 10+ LLM providers through a unified BaseChatModel trait.

Supported Providers

ProviderConfigAuth
OpenAIOpenAIChatOPENAI_API_KEY
AnthropicAnthropicChatANTHROPIC_API_KEY
GeminiGeminiChatGEMINI_API_KEY
OllamaOllamaChatLocal (no key)
DeepSeekDeepSeekChatDEEPSEEK_API_KEY
QwenQwenChatQWEN_API_KEY
MoonshotMoonshotChatMOONSHOT_API_KEY
ZhipuZhipuChatZHIPU_API_KEY
MistralMistralChatMISTRAL_API_KEY
Azure OpenAIAzureOpenAIChatAZURE_OPENAI_API_KEY
CohereCohereChatCOHERE_API_KEY

Auto-Detection

LLMClient::from_env() auto-detects the provider from environment variables:

#![allow(unused)]
fn main() {
use langchainrust::LLMClient;

let llm = LLMClient::from_env().expect("Set an API key");
}

Priority: OpenAI → Anthropic → Mistral → Ollama (local fallback).

OpenAI-Compatible Wrappers

DeepSeek, Qwen, Moonshot, Zhipu, and Mistral all wrap OpenAIChat internally via into_openai_config() — they use the same chat format but with different base URLs and default models.

Native Providers

Azure OpenAI and Cohere have native implementations because their URL/auth format differs from OpenAI’s.

Streaming

All providers support streaming via stream_chat():

#![allow(unused)]
fn main() {
let stream = llm.stream_chat(messages, None).await?;
while let Some(token) = stream.next().await {
    print!("{}", token?.content);
}
}

Multimodal

OpenAI supports Whisper (STT), TTS, and DALL-E via the MultimodalModel trait:

#![allow(unused)]
fn main() {
use langchainrust::MultimodalModel;

let transcript = llm.transcribe(audio).await?;
let speech = llm.generate_speech("Hello!").await?;
let image = llm.generate_image("A cat wearing a hat").await?;
}

Retry & Fallback

#![allow(unused)]
fn main() {
use langchainrust::runnables::{RetryConfig, RunnableExt};

let llm_with_retry = llm.with_retry(RetryConfig::new(3));
}

See also

Agents

Agents are autonomous systems that use LLMs to decide which actions to take.

Agent Types

AgentDescriptionUse Case
ReActAgentReason + Act loopGeneral-purpose tasks
FunctionCallingAgentNative tool callingOpenAI/Anthropic tools
PlanExecuteAgentPlan → Execute → ReplanComplex multi-step tasks
CRAGSelf-correcting RAGRetrieval with quality control
AdaptiveRAGAdaptive retrieval routingDifferent retrieval strategies
DeepResearchMulti-round researchComprehensive reports
Supervisor (orchestrator)Runtime routing to named sub-agentsOne objective, several specialist agents

Basic Usage

#![allow(unused)]
fn main() {
use langchainrust::agents::{ReActAgent, AgentExecutor, BaseAgent};

let agent = ReActAgent::new(llm, prompt_template);
let executor = AgentExecutor::new(Arc::new(agent), tools)
    .with_max_iterations(10);

let result = executor.invoke("What is 2+2?".to_string()).await?;
}

Supervisor: Dynamic Sub-Agent Routing

Where SequentialPipeline runs a fixed stage list and FanOutFanIn broadcasts to everyone, the Supervisor asks a router model once per round which named worker should handle the next subtask — or whether the job is done. Each worker is a complete independent agent behind the Orchestrator trait (its own executor, budget, hooks); the worker’s answer is fed back into the router’s scratchpad, so the next routing decision is based on what earlier workers actually produced.

#![allow(unused)]
fn main() {
use langchainrust::{
    Supervisor, TaskAdapter, Orchestrator, RunContext, AgentTask,
};
use std::sync::Arc;

// Any String -> String Orchestrator (an AgentExecutor, another pipeline, ...)
// becomes a worker through TaskAdapter (it receives an AgentTask, returns String).
let researcher: Arc<dyn Orchestrator<Input = AgentTask, Output = String>> =
    Arc::new(TaskAdapter::new(Arc::new(research_executor)));
let writer = Arc::new(TaskAdapter::new(Arc::new(writer_executor)));

let supervisor = Supervisor::new(
    router_llm,                                   // String -> String; prompt it to return
    vec![                                         // {"next":"<worker>","task":"..."} or
        ("researcher".to_string(), researcher),   // {"next":"FINISH","answer":"..."}
        ("writer".to_string(), writer),
    ],
    8,                                            // max delegation rounds
);

let answer = supervisor
    .run_with_context(
        AgentTask::new("Write a one-page brief on reranking in RAG"),
        &RunContext::new_random(),
    )
    .await?;
}

This is exactly one level of sub-agent recursion: workers are leaf orchestrators and cannot themselves delegate. Routing is bounded by max_rounds (.with_max_rounds adjusts it later; values below 1 clamp to 1) — if the model never emits SUPERVISOR_FINISH, the run errors instead of looping forever. Decisions parse as JSON first, with a <<<NEXT>>> delimiter fallback (parse_supervisor_decision), so weaker models still work with an envelope prompt (supervisor_envelope).

Subagent: Context Isolation

The orchestrators above share one message history: a worker sees everything the parent saw. For long tasks that is the bottleneck — every tool observation and failed attempt stays in the parent context. SubagentTool takes the other approach: each call builds a fresh child AgentExecutor with an empty history, passes only the task text, and returns only {status, summary}.

#![allow(unused)]
fn main() {
use std::sync::Arc;
use langchainrust::{AgentExecutor, SubagentConfig, SubagentTool};

// The factory owns the child's tools, budget and approval handler.
let factory = Arc::new(|depth: usize| {
    Ok(Arc::new(AgentExecutor::new(Arc::new(child_agent), child_tools)))
});

let tool = SubagentTool::new(
    SubagentConfig::new("researcher", "Delegate a self-contained research sub-task")
        .with_max_depth(2),
    factory,
);

let parent = AgentExecutor::new(parent_agent, vec![Arc::new(tool)]);
let answer = parent
    .invoke("Summarize the latest RAG reranking work".to_string())
    .await?;
}

Invariants:

InvariantMeaning
Context isolationThe child starts empty and receives only task
Summary onlyThe return value is {status, summary}; no intermediate_steps
Depth capdepth >= max_depth returns ToolError::ExecutionFailed
No privilege escalationTool subset / budget / approval are fixed by the factory

The factory receives depth + 1, so a recursive subagent can wire SubagentTool::with_depth(depth) itself; without that wiring max_depth cannot fire.

Parallel Tool Calls

When a model emits several tool calls in one turn, the executor runs them concurrently — bounded by a semaphore (default cap 8) rather than spawned unbounded:

#![allow(unused)]
fn main() {
let executor = AgentExecutor::new(agent, tools)
    .with_max_concurrency(4);   // at most 4 in-flight tool calls per turn
}

Observations are zipped back to their actions in the model’s call order (not completion order), so multi-tool turns are deterministic; the same guarantee holds under streaming.

Hooks

Add lifecycle hooks for approval, content filtering, and logging:

#![allow(unused)]
fn main() {
use langchainrust::hooks::{ApprovalHook, ContentFilterHook, LoggingHook};

let executor = AgentExecutor::new(agent, tools)
    .hook(ApprovalHook::new())                              // Require approval
    .hook(ContentFilterHook::new(vec!["secret".into()]))    // Filter words
    .hook(LoggingHook::new());                               // Log events
}

Streaming

#![allow(unused)]
fn main() {
let stream = executor.stream(input);
while let Some(event) = stream.next().await {
    match event? {
        AgentStreamEvent::ToolStart { name, input } => { /* ... */ }
        AgentStreamEvent::ToolEnd { name, output } => { /* ... */ }
        AgentStreamEvent::FinalAnswer { content } => { /* ... */ }
        AgentStreamEvent::PipelineStep { step, detail } => { /* ... */ }
        AgentStreamEvent::Error { message } => { /* ... */ }
    }
}
}

Cancellation

#![allow(unused)]
fn main() {
use langchainrust::runnables::CancellationToken;

let token = CancellationToken::new();
tokio::spawn(async move {
    tokio::time::sleep(Duration::from_secs(30)).await;
    token.cancel();
});

let result = executor.invoke_with_config(input, Some(
    RunnableConfig::new().with_cancellation_token(token)
)).await;
}

RAG & Retrieval

LangChainRust provides a full Retrieval-Augmented Generation stack: document loaders, splitters, BM25, hybrid search (RRF / weighted fusion), HyDE, multi-query, lexical and hosted neural reranking, MMR diversity selection, small-to-big retrieval (sentence-window / parent-document), token-level late chunking, and GraphRAG.

Feature Overview

FeatureTypeDescription
RAGPipelinePipelineEnd-to-end chunk + embed + store + retrieve + generate
BM25RetrieverSparseClassic BM25 keyword search with English + Chinese tokenization
ChunkedBM25RetrieverSparseBM25 with parent-child auto-merging (LlamaIndex-style)
UnifiedHybridIndexHybridDual-index with auto-splitting and detailed scores
reciprocal_rank_fusionFusionCombine BM25 + vector results via Reciprocal Rank Fusion
HyDERetrieverExpansionGenerate hypothetical document, then retrieve with it
MultiQueryRetrieverExpansionGenerate multiple query variants, merge results
RerankingExecutorRerankingPost-retrieval scoring with KeywordReranker or BM25Reranker
AsyncReranker / CohereRerank / JinaRerankNeural rerankHosted cross-encoder reranking behind one async trait; rerank_async helper
mmr / retrieve_mmrDiversityMaximal Marginal Relevance: trade relevance against result redundancy
FusionMode::WeightedFusionLinear w_bm25·bm25 + w_vec·vector alternative to RRF
SentenceWindowRetrieverSmall-to-bigRetrieve against single sentences, return a ±N-sentence window
ParentDocumentRetrieverSmall-to-bigSmall leaf chunks for recall; any hit returns the whole parent document
late_chunk / add_late_chunked_documentLate chunkingOne token-level embedding pass, mean-pool per chunk, inject both legs
GraphRAGGraphEntity/relation extraction, community detection, multi-mode query
SemanticSplitterSplitterEmbedding-based semantic chunking
RecursiveCharacterSplitterSplitterCharacter-level recursive splitting

RAGPipeline

#![allow(unused)]
fn main() {
use langchainrust::{
    RAGPipelineBuilder, OpenAIChat, OpenAIConfig,
    OpenAIEmbeddings, OpenAIEmbeddingsConfig, InMemoryVectorStore,
};

let rag = RAGPipelineBuilder::new()
    .llm(OpenAIChat::new(OpenAIConfig::new("sk-...")))
    .embeddings(OpenAIEmbeddings::new(OpenAIEmbeddingsConfig::new("sk-...")))
    .vector_store(InMemoryVectorStore::new())
    .retrieve_k(5)
    .system("Answer based on the provided context.")
    .build()?;

// Or inject any `RetrieverTrait` implementation directly (BM25, hybrid, ...):
// let rag = RAGPipelineBuilder::new()
//     .llm(OpenAIChat::new(OpenAIConfig::new("sk-...")))
//     .retriever(BM25Retriever::new())
//     .build()?;

rag.index_documents(docs).await?;
let answer = rag.query("What is Rust?").await?;
let result = rag.query_with_sources("What is Rust?").await?;
// result.answer, result.sources
}

Hybrid Retrieval with Reranking

#![allow(unused)]
fn main() {
use langchainrust::{
    BM25Retriever, RerankingExecutor, KeywordReranker,
    reciprocal_rank_fusion,
};

// BM25 sparse retrieval
let bm25 = BM25Retriever::new();
bm25.add_documents_sync(docs.clone());

// Vector retrieval (via SimilarityRetriever)
let vector_results: Vec<Document> = retriever.retrieve("query", 10).await?;

// Fuse with Reciprocal Rank Fusion
let fused = reciprocal_rank_fusion(bm25_results, vector_results, 60);

// Rerank
let executor = RerankingExecutor::new(Box::new(KeywordReranker::new()))
    .with_top_n(5)
    .with_min_score(0.1);
let reranked = executor.rerank_documents("query", fused_docs)?;
}

Neural Cross-Encoder Reranking

Lexical rerankers are cheap and offline; when recall quality matters more, a hosted cross-encoder scores each candidate against the query jointly. Both providers sit behind one async trait, so swapping providers does not touch call-site code.

#![allow(unused)]
fn main() {
use langchainrust::{CohereRerank, JinaRerank, AsyncReranker, rerank_async};

// Empty key falls back to COHERE_API_KEY; defaults to rerank-multilingual-v3.0.
// JinaRerank::new("") reads JINA_API_KEY (jina-reranker-v2-base-multilingual).
let cohere = CohereRerank::new("")
    .with_model("rerank-v3.5");             // optional
    // .with_base_url("https://api.cohere.com") (also how tests point at a mock)

// Overshoot on recall, then let the cross-encoder pick the final top_n.
let pool = retriever.retrieve_with_scores("query", 20).await?;
let top: Vec<SearchResult> = rerank_async(&cohere, "query", pool, 5).await?;
}

Contract details worth knowing: the helper maps the API’s results[].index back to your input positions (provider order is not guaranteed), a malformed response is an error rather than silently unsorted input, and both clients bypass ambient proxy settings (.no_proxy()) because rerank endpoints are usually called server-to-server.

MMR Diversity & Weighted Fusion

RRF sorts purely by rank, so one narrow topic can fill the whole top-k. MMR greedily picks each next item for relevance minus similarity to what is already selected, with a λ dial: λ=1 is plain relevance, λ=0 maximizes diversity (0.5–0.7 is the usual range).

#![allow(unused)]
fn main() {
use langchainrust::{UnifiedHybridIndex, HybridIndexConfig, FusionMode, mmr};

// Weighted linear fusion instead of the default RRF.
let config = HybridIndexConfig::new()
    .with_fusion(FusionMode::Weighted { bm25_weight: 0.3, vector_weight: 0.7 });
let index = UnifiedHybridIndex::with_config(embeddings, store, 1536, config);

// MMR as a retriever method: pool 20 fused candidates, re-embed just those once,
// return a diversified 5.
let picks = index.retrieve_mmr("query", 20, 5, 0.6).await?;

// Or use the pure algorithm over your own (id, relevance, embedding) triples.
let ids: Vec<String> = mmr(&candidates, 0.6, 5);
}

Fusion scores are min-max normalized inside the candidate pool before MMR, so the λ trade-off means the same thing under RRF and weighted fusion.

Small-to-Big Retrieval

Embedding one sentence gives a precise hit; answering usually needs the surrounding context. Two retrievers implement the small-to-big pattern.

#![allow(unused)]
fn main() {
use langchainrust::{SentenceWindowRetriever, ParentDocumentRetriever,
                    InMemoryChunkedDocumentStore};
use std::sync::Arc;

// Sentence window: index single sentences, return ±N sentences around each hit.
let sw = SentenceWindowRetriever::from_documents(docs)
    .with_window(2)      // two sentences on each side (default)
    .with_top_k(3);
let windows = sw.retrieve("query", 3).await?;   // one deduplicated window per source

// Parent document: tiny leaf chunks for recall, the ENTIRE parent doc for the LLM.
let store = Arc::new(InMemoryChunkedDocumentStore::new());
store.add_parent_with_chunks(parent_doc, vec!["leaf 1".into(), "leaf 2".into()]).await?;
let pdr = ParentDocumentRetriever::new(store);
let parents = pdr.retrieve("query", 4).await?;
}

Unlike ChunkedBM25Retriever’s hit-ratio-gated auto-merging, ParentDocumentRetriever always returns the parent — leaves exist only for matching.

Late Chunking

Normal chunking embeds each chunk in isolation, so cross-chunk references lose their antecedent. Late chunking runs one token-level embedding pass over the full document, then mean-pools token vectors inside each chunk window — every chunk vector carries document-wide context.

#![allow(unused)]
fn main() {
use langchainrust::{late_chunk, LateChunkConfig, UnifiedHybridIndex};

let config = LateChunkConfig::new().with_chunk_size(256).with_chunk_overlap(32);
let chunks = late_chunk(&token_embedder, &document.content, &config).await?;

// Inject one document into BOTH legs at once: pooled vectors into the vector
// index, chunk texts into the BM25/parent store. Deterministic ids make
// re-registering the same parent idempotent.
index.add_late_chunked_document(document, &chunks).await?;
}

GraphRAG

#![allow(unused)]
fn main() {
use langchainrust::{GraphRAG, GraphRAGConfig, QueryMode as GraphQueryMode};

let graph_rag = GraphRAG::new(llm).with_config(
    GraphRAGConfig::new()
        .with_max_entities_per_doc(10)
        .with_max_relations_per_doc(10),
);

graph_rag.add_documents(&docs).await?;
graph_rag.build_communities().await?;

let result = graph_rag.query("How does X relate to Y?", GraphQueryMode::Local).await?;
// result.answer, result.sources, result.mode
}

Document Loaders

LoaderFormat
TextLoaderPlain text
CSVLoaderCSV
MarkdownLoaderMarkdown
HTMLLoaderHTML
JSONLoaderJSON
PDFLoaderPDF
DocxLoaderWord DOCX
WebScraperLoaderWeb pages
SitemapLoaderSitemap XML

All loaders implement DocumentLoader with async fn load() -> Result<Vec<Document>, LoaderError>.

LangGraph

LangGraph provides graph-based orchestration for stateful, long-running LLM workflows with conditional routing, subgraphs, parallel execution, dynamic in-node interrupt/resume human-in-the-loop, and checkpoint state history with fork-based time travel.

Core Concepts

ConceptTypeDescription
StateGraph<S>BuilderDefine nodes, edges, and reducers on state S
GraphBuilder<S>BuilderFluent (consuming) variant of StateGraph
CompiledGraph<S>RuntimeCompiled, executable graph
GraphNode<S>TraitNode execution: async fn execute(&self, state, config) -> NodeResult<S>
ConditionalEdge<S>TraitDynamic routing: async fn route(&self, state) -> String
Reducer<S>TraitState merge strategy: fn reduce(&self, current, update) -> S
Checkpointer<S>TraitState persistence: save, load, list, delete, snapshots
SubgraphNode<S, SubS>NodeEmbed a CompiledGraph<SubS> inside a parent graph
InterruptibleNode<S>NodeResume-aware node: the closure receives resume: Option<&Value> (None on first pass, Some(decision) after a human answers)
CheckpointInfo<S>RecordOne history entry: id, timestamp, seq, recursion_count, state
StreamEvent<S>EnumStart, EnterNode, NodeComplete, StateUpdate, NodeInterrupt, Resumed, End

Basic Graph

#![allow(unused)]
fn main() {
use langchainrust::{
    StateGraph, GraphBuilder, StateSchema, StateUpdate,
    AgentState, START, END,
};
use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize, Clone, Debug)]
struct MyState { messages: Vec<String>, count: usize }
impl StateSchema for MyState {}

let mut graph = StateGraph::<MyState>::new();
graph.add_node_fn("process", |state: MyState| {
    Ok(StateUpdate::full(MyState {
        messages: state.messages.clone(),
        count: state.count + 1,
    }))
});
graph.add_edge(START, "process");
graph.add_edge("process", END);

let compiled = graph.compile()?;
let result = compiled.invoke(MyState { messages: vec![], count: 0 }).await?;
}

Conditional Routing & Human-in-the-Loop

#![allow(unused)]
fn main() {
use langchainrust::{GraphBuilder, FunctionRouter, AgentState, StateUpdate, START, END};
use std::collections::HashMap;

let compiled = GraphBuilder::<AgentState>::new()
    .add_node_fn("entry", |s| Ok(StateUpdate::full(s.clone())))
    .add_node_fn("short", |s| { /* ... */ })
    .add_node_fn("long", |s| { /* ... */ })
    .add_node_fn("review", |s| { /* ... */ })
    .add_edge(START, "entry")
    .add_conditional_edges("entry", "router",
        HashMap::from([("short".into(), "short".into()), ("long".into(), "long".into())]),
        None)
    .add_edge("short", "review")
    .add_edge("long", "review")
    .add_edge("review", END)
    .set_conditional_router("router", FunctionRouter::new(|s: &AgentState| {
        if s.input.len() < 10 { "short" } else { "long" }.to_string()
    }))
    .compile()?
    .with_interrupt_before(vec!["review".to_string()]);  // HITL

// First invoke returns ExecutionInterrupted
// Resume with: compiled.create_resume_execution("review") ... compiled.resume(execution).await?
}

The static list fixes stopping points before launch. When the decision depends on runtime data (“this action would spend money — ask a human”), use a dynamic in-node interrupt instead.

Dynamic interrupt() + resume (human-in-the-loop)

InterruptibleNode wraps a closure that is called twice per interrupt. On the first pass resume is None; the node does its pre-side-effect work and suspends itself by returning GraphError::InterruptRequest { payload }. After a human answers, resume_with_value re-enters the same node with the answer and the closure runs again with resume = Some(decision) — branch on it so the side effect that was approved happens once, on the resumed pass only.

#![allow(unused)]
fn main() {
use langchainrust::{
    GraphBuilder, InterruptibleNode, AgentState, StateUpdate,
    GraphError, GraphNode, START, END, ThreadSafeMemoryCheckpointer,
};
use serde_json::json;

let approve = InterruptibleNode::new("charge", |state, resume| {
    let output = state.output.clone().unwrap_or_default();
    Box::pin(async move {
        match resume {
            // First pass: suspend, hand the approval context to the outside world.
            None => Err(GraphError::InterruptRequest {
                payload: json!({ "kind": "tool_approval", "command": output }),
            }),
            // Resumed with the human's decision: perform the side effect once.
            Some(decision) => {
                let mut next = state.clone();
                next.set_output(format!("decision={decision}, charged"));
                Ok(StateUpdate::full(next))
            }
        }
    })
});

let compiled = GraphBuilder::<AgentState>::new()
    .add_node(approve)                       // InterruptibleNode implements GraphNode
    .add_edge(START, "charge")
    .add_edge("charge", END)
    .compile()?
    // A checkpointer is mandatory: the interrupt payload is persisted into the
    // checkpoint, so even a process restart can resume instead of replaying.
    .with_checkpointer(ThreadSafeMemoryCheckpointer::new());

// First run suspends. The error carries the node name + payload.
let err = compiled.invoke(AgentState::new("charge $99")).await.unwrap_err();
// GraphError::DynamicInterrupt { node, payload } surfaces the suspended node.

// A human answers; the SAME node is re-entered with the value. Nothing before the
// interrupt is replayed, and the recursion budget carries over.
let invocation = compiled
    .resume_with_value("charge", json!({ "approved": true }))
    .await?;
println!("{}", invocation.final_state.output.as_deref().unwrap_or(""));
}

Key semantics:

  • One persistence system. The interrupt payload lives in the graph checkpoint (memory / file / SQLite / Postgres / Redis). A brand-new process over the same checkpoint directory can call resume_with_value.
  • Side effects run once. Keep any first-pass work that must not repeat inside the None branch; the Some(decision) branch is the only place effects fire.
  • Cascading approvals. Resuming one node can immediately suspend at the next interrupt — each resume returns the next DynamicInterrupt until the graph reaches END.
  • Streaming. A suspended node emits StreamEvent::NodeInterrupt(name, payload) and the resumed pass emits StreamEvent::Resumed(name, decision); pre-interrupt events are not replayed.

When agents run on the graph, lc_agents::graph_approval::ApprovalGate builds exactly this pattern: an agent tool approval is an interrupt payload, and ApprovalDecision (Allow / Deny { reason } / Modify { arguments, note }) is the resume value — Deny skips the tool with no side effect.

Checkpointer & Subgraphs

#![allow(unused)]
fn main() {
use langchainrust::{ThreadSafeMemoryCheckpointer, SubgraphBuilder, START, END};

// Checkpointer for state persistence
let checkpointer = ThreadSafeMemoryCheckpointer::<AgentState>::new();
let compiled = graph.compile()?.with_checkpointer(checkpointer);

// Subgraph composition
let subgraph = GraphBuilder::<AgentState>::new()
    .add_node_fn("sub_process", |s| Ok(StateUpdate::full(s.clone())))
    .add_edge(START, "sub_process")
    .add_edge("sub_process", END)
    .compile()?;

let parent = GraphBuilder::<AgentState>::new()
    .add_subgraph_same_state("subworkflow", subgraph)
    .add_edge(START, "subworkflow")
    .add_edge("subworkflow", END)
    .compile()?;
}

State History & Time Travel

Every checkpointed step is also a snapshot. get_state_history lists them, oldest first, and fork_from branches the run off any one of them.

#![allow(unused)]
fn main() {
use langchainrust::CheckpointInfo;

// Run a graph with a checkpointer, then inspect where it has been.
let history: Vec<CheckpointInfo<AgentState>> = compiled.get_state_history()?;
for cp in &history {
    println!("seq={} ts={} recursion={} id={}", cp.seq, cp.timestamp, cp.recursion_count, cp.id);
    let state_at_that_point: &AgentState = &cp.state;
}

// Replay a "what if": branch off an old snapshot and re-run forward from a
// chosen node. Pass Some(state) to edit history (e.g. fix a bad tool output);
// None replays the snapshot's state as-is.
let forked = compiled
    .fork_from(&history[2].id, "analyze", Some(corrected_state))
    .await?;
println!("forked answer: {}", forked.final_state.output.as_deref().unwrap_or(""));
}

The fork is a new checkpoint lineage — it never mutates the original run, so you can keep the production timeline and explore alternatives side by side. Execution is forward-only from the chosen node: nodes before the fork point do not run again (their side effects are assumed already applied). If you need a true re-execution including those effects, fork from an earlier snapshot. Forks compose with interrupts: if the branched path suspends, resume it with resume_with_value exactly like any other run.

Parallel & Streaming

#![allow(unused)]
fn main() {
// Fan-out / fan-in
graph.add_fan_out("entry", vec!["branch_a".into(), "branch_b".into()]);
graph.add_fan_in(vec!["branch_a".into(), "branch_b".into()], "merge");

// Streaming
let stream = compiled.stream(state);
while let Some(event) = stream.next().await {
    match event? {
        StreamEvent::EnterNode(name, state) => { /* ... */ }
        StreamEvent::NodeComplete(name, update) => { /* ... */ }
        StreamEvent::End(final_state) => { /* ... */ }
        _ => {}
    }
}

// Visualization
println!("{}", compiled.visualize_ascii());
println!("{}", compiled.visualize_mermaid());
}

MCP Protocol

The Model Context Protocol (MCP) is Anthropic’s open standard for connecting LLM applications to external tools and data sources. LangChainRust provides both a client and server implementation. As of 0.22.0 the crate is stateless single-track: every request is a self-contained JSON-RPC HTTP POST tagged with Mcp-Method / Mcp-Name routing headers and a _meta block; there is no handshake and no session. The legacy handshake-based MCPClient (SSE/stdio transports) was removed in 0.22.0.

Feature Overview

FeatureTypeDescription
StatelessMcpClientClientConnect to any stateless MCP server, list/call tools
MCPServerServerExpose BaseTool implementations via MCP (handle_request / serve_http / serve_stdio)
MCPToolAdapterAdapterWrap MCP tools as BaseTool for agent use
ConnectionManagerClient100+ Server lazy startup / idle reaping / managed registry
ToolNamespaceClientserver:tool name uniquification + conflict policy
ToolDiscoveryClientStatic (pinned) + dynamic (query top-k) tool selection
ToolSpecClientPer-tool timeout with hard cap
ServerHealth / CircuitBreakerClientPer-server health probe (list_tools) + circuit breaker
ServerSandboxClientPer-server security isolation (param least-privilege + egress whitelist + audit)
SamplingGuardClientSampling recursion protection (depth + token budget + timeout)
MCPGatewayClientUnified registry + on-demand dispatch (rate limiting + audit)
TenantGatewayClientMulti-tenant isolation (per-tenant registry + audit)
ToolOrchestrator / ToolStepClientTool orchestration (dependency DAG + parallel/serial + ${id} template args)
VersionPolicy / ProtocolInfoProtocolProtocol version handling (degrade or reject on unsupported)
StatelessTransportTransportPure HTTP POST JSON-RPC, no session
MethodRateLimiterClientClient-side per-method rate limiting (fails fast, no network round trip)
TokenValidator / AuthSchemeAuthBearer/JWT token validation (server side) + auth headers (client side)
McpTaskHandleClientMCP Tasks placeholder (persistence deferred to 0.22.1)

MCP Client

#![allow(unused)]
fn main() {
use langchainrust::mcp::{MCPToolAdapter, StatelessMcpClient};

// Connect to a stateless MCP endpoint — no handshake, this never fails
let client = StatelessMcpClient::connect("http://localhost:3001/mcp");

// List available tools
let tools = client.list_tools().await?;
for tool in &tools {
    println!("{}: {}", tool.name, tool.description);
}

// Call a tool
let result = client.call_tool("read_file", serde_json::json!({"path": "/tmp/hello.txt"})).await?;
println!("{}", result.text());

// Convert all MCP tools to BaseTool for agent use
let base_tools: Vec<Arc<dyn BaseTool>> = client
    .list_tools()
    .await?
    .into_iter()
    .map(|def| Arc::new(MCPToolAdapter::new(client.clone(), def)) as Arc<dyn BaseTool>)
    .collect();
}

Client-side extras:

  • .with_mrtr(MrtrConfig { max_round_trips }) — bound the input_required multi-round request loop; the client resends the original request with the continuation token (MRTR), up to the limit (-32003 when exceeded);
  • .with_answer_provider(provider) — collects answers for input_required questions (required for servers that use MRTR);
  • .with_method_rate_limiter(MethodRateLimiter) — per-method client-side rate limit; a hit returns -32002 without a network round trip;
  • client.meta() — the pinned self-contained _meta (protocol version 2026-07-28 + client identity + optional requestState).

MCP Server

#![allow(unused)]
fn main() {
use langchainrust::{MCPServer, BaseTool, Calculator};
use std::sync::Arc;

let server = MCPServer::new()
    .with_tool(Arc::new(Calculator::new()) as Arc<dyn BaseTool>)
    .with_tool(Arc::new(DateTimeTool::new()) as Arc<dyn BaseTool>)
    .with_server_info("my-mcp-server", "1.0.0");

// Serve via stdio (for external hosts like Claude Desktop)
server.serve_stdio().await?;
}

Or serve it as a deployable stateless HTTP service (see examples/mcp_http_server.rs):

#![allow(unused)]
fn main() {
use std::sync::Arc;
use tokio::net::TcpListener;

let listener = TcpListener::bind("0.0.0.0:8788").await?;
let url = Arc::new(server).serve_http(listener); // "http://0.0.0.0:8788/mcp"
println!("clients connect to: {url}");
}

server.handle_request(req) is also available for custom transports: one-shot JSON-RPC handling with no network at all.

Stateless HTTP Track

Every request is self-contained: the client POSTs a JSON-RPC envelope plus _meta (protocol version / client identity / optional requestState) and tags it with the Mcp-Method / Mcp-Name headers so gateways can route and throttle without parsing the body. There is no handshake, no session id, and no sticky routing — any HTTP client that can POST JSON can talk to the server. Auth is per-request bearer/JWT (connect_with_auth on the client, TokenValidator implementations on the server/gateway side).

Multi-Server Management (100+ Servers)

Holding one dedicated client per server still costs hundreds of long-lived handles. Use ConnectionManager to host a managed registry, and ToolNamespace to keep each server’s tools uniquely named.

Connection Manager (lazy start / idle reaping / pooling)

#![allow(unused)]
fn main() {
use langchainrust::{ConnectionManager, ServerSpec};

let manager = ConnectionManager::new();
// register is lazy: nothing is built yet
manager.register(
    ServerSpec::new("fs", "http://mcp-fs.internal:8080/mcp"),
).await?;
// stateful servers marked keep_alive are never reaped while idle
manager.register(
    ServerSpec::new("db", "http://mcp-db.internal:8080/mcp").keep_alive(),
).await?;

// first client() call builds the client; later calls reuse it
let client = manager.client("fs").await?;
}

Idle servers (non-keep_alive, idle beyond max_idle) are reaped in the background. manager.reap_idle() triggers a sweep manually; manager.shutdown() closes everything and stops the reaper task.

Tool Namespace (name uniquification + conflict policy)

Different servers often expose same-named tools (several with read_file). The ToolNamespace registry uniquifies each tool as server_name:tool_name, with an explicit conflict strategy:

  • ToolConflict::Prefix — colliding tools are all exposed, each under its own server: prefix;
  • ToolConflict::Reject — a same-named tool from another server is rejected at registration.
#![allow(unused)]
fn main() {
use langchainrust::{ToolNamespace, ToolConflict};

let mut ns = ToolNamespace::new();
// both servers expose "read_file" → distinct names fs:read_file / db:read_file
ns.register("fs", fs_tools, ToolConflict::Prefix)?;
ns.register("db", db_tools, ToolConflict::Prefix)?;

// route a call back to the owning server / raw tool name
let (server, raw) = ns.resolve("fs:read_file").expect("registered");
}

The namespaced adapter exposes server:tool to the LLM while calling the raw tool name on the server side:

#![allow(unused)]
fn main() {
use langchainrust::MCPToolAdapter;

let client = manager.client("fs").await?;
let adapter = MCPToolAdapter::namespaced(client, "fs", read_file_def);
assert_eq!(adapter.name(), "fs:read_file");  // what the LLM sees
}

Static + Dynamic Tool Discovery

100+ servers can declare hundreds of thousands of tokens of tool schemas — far beyond any context window. ToolDiscovery avoids injecting everything at once by splitting tools into two layers:

  • Static layer: 20-50 high-frequency tools pinned as always-on;
  • Dynamic layer: tools retrieved per-query by relevance (top-k), like a RAG step over the tool registry. Relevance is scored by [KeywordScorer] (token overlap, zero dependencies) by default; implement [ToolScorer] for vector-based scoring and inject it with with_scorer.
#![allow(unused)]
fn main() {
use langchainrust::{ToolDiscovery, ToolScorer};

let mut discovery = ToolDiscovery::new();
for def in all_tools { discovery.register(def); }       // full registry
discovery.pin("get_time");                              // static layer: always injected
discovery.pin("search_db");

// per-query injection: pinned tools + top-2 query-relevant tools (deduped)
let injected = discovery.select("find files modified today", /*top_k*/ 2, /*static_limit*/ 50);
}

Per-Tool Timeout with Hard Cap

Long-running tools must not hold callers hostage. ToolSpec gives each tool a default timeout plus a hard cap that bounds total time regardless of anything else.

#![allow(unused)]
fn main() {
use langchainrust::{MCPToolAdapter, ToolSpec};
use std::time::Duration;

let adapter = MCPToolAdapter::new(client, def)
    .with_timeout(ToolSpec::new("read_file", Duration::from_secs(30)));
// default timeout 30s; hard cap defaults to 3× (90s), or override with .with_max_timeout(..)
}

Health Probe + Circuit Breaker

With 100+ servers a single one can go down at any time. Each registered server carries a [CircuitBreaker]: list_tools doubles as the health probe, N consecutive failures trip the breaker open (the server is removed — incoming client() calls fail fast instead of hammering a dead server), and after an exponential backoff the breaker opens a half-open probe window so a recovered server can reconnect.

#![allow(unused)]
fn main() {
use langchainrust::{
    ConnectionManager, ServerSpec, ServerHealth, HealthStatus,
};

let manager = ConnectionManager::new();
manager.register(
    ServerSpec::new("fs", "http://localhost:8080/mcp")
        .with_max_failures(3), // 3 consecutive failures → trip
).await?;

// proactive probe: list_tools is the probe; returns a health snapshot
let health: ServerHealth = manager.health("fs").await?;
match health.status {
    HealthStatus::Healthy => { /* normal */ }
    HealthStatus::Degraded => { /* failures below threshold */ }
    HealthStatus::Down => { /* breaker open */ }
}

// during the open state client() fails fast
// manually remove all tripped servers (returns their names)
let removed = manager.reap_unhealthy().await;
}

Default max_failures is 3. The breaker backs off 0.5s → 1s → 2s → … (cap 30s) between reconnect attempts. Health is the liveness gate only — tool calls still fail fast through [ToolSpec] timeouts.

Per-Server Security Sandbox

100+ servers come from very different origins; each one must be narrowed to its own least-privilege boundary. [ServerSandbox] bundles:

  • Parameter-level least privilege — [ParamRule] constrains tool-call arguments: a filesystem server only allows file:///tmp/ prefixes, formats only allow enum values, and path-traversal substrings (..) are rejected. Violating calls are blocked before the request is sent to the server.
  • Outbound network whitelist — [EgressPolicy] declares which hosts the server may contact; an empty whitelist denies all egress (fail-closed).
  • Full audit log — every allowed/blocked call is recorded (server, tool, arguments, decision, reason), ring-buffered with a cap.
#![allow(unused)]
fn main() {
use langchainrust::{MCPToolAdapter, ServerSandbox, ParamRule, EgressPolicy};
use std::sync::Arc;

let sandbox = Arc::new(
    ServerSandbox::new("fs")
        .with_param_rule(ParamRule::Prefix {
            field: "path".to_string(),
            prefix: "file:///tmp/".to_string(), // only allow tmp prefixes
        })
        .with_param_rule(ParamRule::RejectContains {
            field: "path".to_string(),
            forbidden: vec!["..".to_string()],  // reject path traversal
        })
        .allow_host("example.com"),             // egress whitelist
);

let adapter = MCPToolAdapter::new(client, def).with_sandbox(sandbox);
// run() checks the sandbox first; a block returns InvalidInput and records audit
// let out = adapter.run(r#"{"path": "file:///etc/passwd"}"#).await?; // Err
}

Same-server tools share one sandbox (cheap Arc clone), so they all write into the same audit log. Read it back with sandbox.audit_log().

Sampling Recursion Protection

MCP Sampling (“Agent calls tool → tool requests Sampling → LLM calls tool → another Sampling request”) can recurse unboundedly. [SamplingGuard] bounds the whole chain on the Host side with three constraints:

  • Depth limit — nested Sampling may not exceed max_depth (default 3);
  • Cumulative token budget — each request’s max_tokens accumulates toward a chain-wide budget;
  • Timeout / deadline — the whole chain must finish within total_timeout or an explicit deadline.

Call enter(request.max_tokens) before every sampling/createMessage. The returned [SamplingLease] holds one nesting level for the duration of the call (atomic counters, safe across await) and releases it on Drop.

#![allow(unused)]
fn main() {
use langchainrust::{SamplingGuard};
use std::time::Duration;

let guard = SamplingGuard::new(/*max_depth*/ 3, /*token_budget*/ 10_000)
    .with_timeout(Duration::from_secs(60));
// guard.enter(req.max_tokens)?;   // depth/budget/timeout: any breach → Err
// ... run LLM inference ...
// lease Drop releases the nesting depth
}

The counter is AtomicUsize (not a Mutex) so re-entrant nested sampling across await cannot deadlock, and a rejected enter occupies neither depth nor budget.

MCP Gateway (unified registry + on-demand dispatch)

MCPGateway is the single entry point that composes P2-1~P2-6 into one registry: it hosts the [ConnectionManager] (lazy connect / idle reaping / circuit breaker), the [ToolNamespace] (per-server name isolation), the [ToolDiscovery] (static + dynamic selection), per-server sandboxes and timeouts, plus rate limiting and a unified audit log. Registering a server is lazy — nothing is built until the first sync / call.

#![allow(unused)]
fn main() {
use langchainrust::{MCPGateway, GatewayServerSpec};
use std::time::Duration;

let gateway = MCPGateway::new();
// register is lazy: nothing connects yet
gateway.register(
    GatewayServerSpec::new("fs", "http://localhost:8080/mcp")
        .with_rate_limit(10, Duration::from_secs(60)) // max 10 calls / minute
        .with_timeout(Duration::from_secs(30)),
).await?;
gateway.register(
    GatewayServerSpec::new("db", "http://localhost:8081/mcp"),
).await?;

// sync connects + pulls tools + populates namespace/discovery (idempotent)
gateway.sync("fs").await?;
gateway.sync_all().await?;

// per-query injection: pinned static tools + top-k relevant dynamic tools
let selected = gateway.select("find files modified today", /*top_k*/ 2, /*static_limit*/ 50);

// on-demand dispatch: "server:tool" resolves → rate-limits → breaker-gated
// client → sandbox → (timeout) → call → audit. Auto-syncs on a first-time miss.
let out = gateway.call("fs:read_file", serde_json::json!({"path": "/tmp/a.txt"})).await?;

// hang the whole registry on an agent as BaseTools (namespaced `server:tool`)
let base_tools = gateway.as_base_tools().await?;
}

Every call passes through a unified audit log (server, tool, decision, reason), ring-buffered with a cap (with_max_audit); read it with gateway.audit_log(). Ops helpers mirror the underlying manager: health, reap_unhealthy, reap_idle, release, and shutdown.

Multi-Tenancy (per-tenant isolation)

One process serving many customers (SaaS) must never leak a tenant’s tools, connections, or audit trail to another. [TenantGateway] wraps an [MCPGateway] per tenant: register is lazy (nothing is built until a tenant’s first sync_all / call), and every operation is routed by tenant_id.

#![allow(unused)]
fn main() {
use langchainrust::{TenantGateway, GatewayServerSpec};

let tenants = TenantGateway::new();
// register is scoped to the tenant — tenant "b" never sees tenant "a"'s tools
tenants.register("a", GatewayServerSpec::new("fs", "http://localhost:8080/mcp")).await?;

tenants.sync_all("a").await?;
// on-demand dispatch: auto-syncs on a first-time miss, then calls
let out = tenants.call("a", "fs:read_file", serde_json::json!({"path": "/tmp/a.txt"})).await?;

// audit logs are per-tenant — no cross-tenant visibility
let audit = tenants.audit_log("a");
// teardown: removing a tenant drops its registry and connections
tenants.remove_tenant("a");
}

TenantGateway::tenant_ids() lists live tenants; a missing tenant is created lazily on first use, so there is no setup step to forget.

Tool Orchestration (dependency DAG)

A multi-step task is often a DAG of tool calls — the output of one tool feeds the next. [ToolOrchestrator] declares steps with dependencies, then executes:

  • Validation first — duplicate ids, unknown dependencies, and dependency cycles are rejected before any tool runs (Kahn topological sort);
  • Round-based parallelism — every step whose dependencies are satisfied runs in the same round, with concurrency capped by with_max_concurrency (default 4);
  • Argument templating — ${id} substitutes a previous step’s whole JSON output, ${id.field} extracts one field, so downstream args are computed from upstream results.
#![allow(unused)]
fn main() {
use langchainrust::{ToolOrchestrator, ToolStep, MCPGateway};
use serde_json::json;

let orch = ToolOrchestrator::new()
    .with_max_concurrency(4)
    .add_step(ToolStep::new("a", "fs:read_file", json!({"path": "/tmp/orders.csv"})))
    .add_step(ToolStep::new("b", "db:query", json!({"sql": "SELECT ..."})).after("a"))
    .add_step(ToolStep::new("sum", "calc:total", json!({
        "rows": "${b.sum}",
        "file": "${a.content}"
    })).after("b"));

// MCPGateway implements ToolCaller — plug the whole DAG into the registry
let results = orch.execute(&gateway).await?;
let total = &results["sum"]["total"];
}

Any [ToolCaller] implementation drives the orchestrator; steps reference server:tool full names. Failure of one step fails the round — later steps that depended on it are skipped and the error propagates as OrchestrateError.

Protocol Version Handling (vNext)

MCP protocol versions evolve. On the stateless track there is no handshake — the client pins its _meta.protocol_version (default 2026-07-28) and the server can inspect it per request. langchainrust records the negotiation result and lets you choose how to treat mismatches:

  • SUPPORTED_PROTOCOL_VERSIONS — the versions this library implements;
  • [VersionPolicy::Degrade] (default) — unknown version: keep going on the library’s own version, recording supported = false;
  • [VersionPolicy::Reject] — unknown version: the request is refused;
  • [ProtocolInfo] — the negotiation result (requested, server_version, negotiated, supported).
#![allow(unused)]
fn main() {
use langchainrust::mcp::StatelessMcpClient;

let client = StatelessMcpClient::connect("http://localhost:8080/mcp");
assert_eq!(client.meta().protocol_version, "2026-07-28");
}

2026-07-28 Compliance Fields

The latest revision makes three result-envelope fields mandatory; v0.30.0 implements them:

FieldWhereBehavior
resultTypeEvery successful result"complete" for ordinary results; "input_required" for MRTR interim results
ttlMstools/list, resources/list, resources/read, prompts/listDefaults to 60_000 ms
cacheScopeSame as aboveDefaults to "public" (process-wide static registries)

Older peers that omit the fields are still accepted as complete. tools/list and server/discover sort tools by name before pagination, keeping client caches and prompt-cache hit rates stable.

Protocol Details

  • Version: 2026-07-28 (stateless single-track)
  • Format: JSON-RPC 2.0 over HTTP POST
  • Transport: stateless, self-contained requests; no handshake, no session
  • Methods: initialize, tools/list, tools/call, server/discover, resources/list, resources/read, prompts/list, prompts/get, completion/complete, sampling/createMessage

Sub-Protocol Support

Sub-protocolClientServer
Toolslist_tools, call_toolserve_stdio, serve_http
Resourceslist_resources, read_resource–
Promptslist_prompts, get_prompt–
Samplingcreate_message–
Completioncomplete–
Rootslist_roots–
Elicitationelicit–

OpenAI-Compatible Server

MCP and A2A cover agent ↔ agent and agent ↔ tool. The openai-server feature adds the entry point most existing applications speak: OpenAI’s /v1/chat/completions.

# `lc-agents` exposes the feature today:
lc-agents = { version = "0.30.0", features = ["openai-server"] }

The langchainrust facade forwards it as openai-server. If you build from main, enable features = ["openai-server"] on the facade as well.

Serving a chat model

#![allow(unused)]
fn main() {
use std::sync::Arc;
use langchainrust::agents::{openai_compat_router, ModelBackend};

let backend = Arc::new(ModelBackend::new(Arc::new(llm)));
let app = openai_compat_router(backend, "my-agent");

// axum::serve(listener, app).await?;
}

Endpoints

MethodPathBehavior
GET/v1/modelsReturns the configured model id
POST/v1/chat/completionsstream: false → JSON; stream: true → SSE followed by [DONE]

Tool calling

The server maps the OpenAI tool protocol end to end:

DirectionShape
Request role: "tool"requires tool_call_id; mapped to a tool message
Request tool_calls on an assistant messagepreserved for the next turn
Response message.tool_calls[]{id, type: "function", function: {name, arguments}}
Response finish_reason"tool_calls" when the model requested tools
Streaming delta.tool_calls[]each entry carries index
{
  "model": "my-agent",
  "messages": [
    { "role": "user", "content": "weather in beijing?" },
    {
      "role": "assistant",
      "content": "",
      "tool_calls": [
        {
          "id": "call_1",
          "type": "function",
          "function": { "name": "get_weather", "arguments": "{}" }
        }
      ]
    },
    { "role": "tool", "tool_call_id": "call_1", "content": "sunny" }
  ]
}

Custom backend

ChatBackend is the seam between the wire protocol and your agent/chain:

#![allow(unused)]
fn main() {
#[async_trait::async_trait]
impl ChatBackend for MyAgentBackend {
    async fn complete(&self, messages: Vec<Message>) -> Result<LLMResult, String> { ... }
    async fn stream_chat(&self, messages: Vec<Message>)
        -> Result<Pin<Box<dyn Stream<Item = Result<StreamChunk, String>> + Send>>, String> { ... }
}
}

Scope

  • tool role, tool_calls responses and streaming tool deltas are mapped.
  • temperature / max_tokens are accepted for compatibility but do not override the backend per request; sampling belongs to the backend configuration.
  • Client SDKs: Language Clients (Python / TypeScript).

Language Clients (Python / TypeScript)

The Rust core is the runtime. Other languages talk to it over the OpenAI-compatible HTTP endpoint, so they never need to write Rust.

Start the server once:

cargo run -p langchainrust --example openai_server --features openai-server
# OpenAI-compatible server on http://127.0.0.1:8080

Python

pip install -e sdks/python
from langchainrust import Client

c = Client("http://127.0.0.1:8080", model="gpt-4o-mini")

print(c.chat("What is Rust?").content)

for chunk in c.stream("hello"):
    print(chunk, end="")

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the weather for a city",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]
result = c.chat_with_tools("weather in beijing?", tools)
for call in result.tool_calls:
    print(call.name, call.arguments)

Methods

MethodReturns
chat(input, tools=None)ChatResult(content, finish_reason, tool_calls, usage)
stream(input)iterator of text deltas
chat_with_tools(input, tools)same as chat(..., tools=...)
models()the server’s model list

Errors raise LangChainRustError with an optional status.

TypeScript

npm install langchainrust
# or, from this repository: npm install ./sdks/typescript
import { Client } from "langchainrust";

const c = new Client("http://127.0.0.1:8080", "gpt-4o-mini");

console.log((await c.chat("What is Rust?")).content);

for await (const chunk of c.stream("hello")) {
  process.stdout.write(chunk);
}

const result = await c.chatWithTools("weather in beijing?", tools);
for (const call of result.toolCalls) {
  console.log(call.name, call.arguments);
}

Client exposes chat, chatWithTools, stream and models; errors are thrown as LangChainRustError with an optional status.

Any OpenAI SDK

Because the endpoint is OpenAI-compatible, existing SDKs work unchanged:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="unused")
print(client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "hi"}],
))

Scope

  • These clients are thin: they only speak HTTP and do not re-implement agent logic.
  • Sampling parameters belong to the backend configuration; the request-level temperature / max_tokens are accepted for compatibility but do not override it.

Vector Stores

LangChainRust provides a unified VectorStore trait with 10+ backends for storing and retrieving document embeddings.

Supported Backends

StoreStructFeature GateAuth
InMemoryInMemoryVectorStorealwaysNone
FileFileVectorStorealwaysNone
ChromaDBChromaDBVectorStorealwayshttp://localhost:8000
PineconePineconeStorealwaysAPI key + host
LanceDBLanceDBVectorStorealwaysOptional API key
Neo4jNeo4jVectorStorealwaysbolt:// + password
QdrantQdrantVectorStoreqdrant-integrationhttp://localhost:6334
PGVectorpgvector (helpers)pgvector-storagesqlx connection
RedisRedisDocumentStoreredis-storageRedis URL
SQLiteSQLiteDocumentStoresqlite-storageFile path
MongoDBMongoChunkedDocumentStoremongodb-persistenceMongoDB URI

VectorStore Trait

#![allow(unused)]
fn main() {
#[async_trait]
pub trait VectorStore: Send + Sync {
    async fn add_documents(&self, documents: Vec<Document>, embeddings: Vec<Vec<f32>>) -> Result<Vec<String>, VectorStoreError>;
    async fn similarity_search(&self, query_embedding: &[f32], k: usize) -> Result<Vec<SearchResult>, VectorStoreError>;
    async fn get_document(&self, id: &str) -> Result<Option<Document>, VectorStoreError>;
    async fn get_embedding(&self, id: &str) -> Result<Option<Vec<f32>>, VectorStoreError>;
    async fn delete_document(&self, id: &str) -> Result<(), VectorStoreError>;
    async fn count(&self) -> usize;
    async fn clear(&self) -> Result<(), VectorStoreError>;
}
}

InMemory & File

#![allow(unused)]
fn main() {
use langchainrust::{InMemoryVectorStore, FileVectorStore, Document};

// In-memory (no persistence)
let store = InMemoryVectorStore::new();
let ids = store.add_documents(docs, embeddings).await?;
let results = store.similarity_search(&query_vec, 5).await?;

// File-backed (JSON persistence)
let store = FileVectorStore::new(PathBuf::from("./vectors.json"), 1536)?;
}

ChromaDB & Qdrant

#![allow(unused)]
fn main() {
use langchainrust::{ChromaDBVectorStore, ChromaDBConfig};

// ChromaDB
let store = ChromaDBVectorStore::new(ChromaDBConfig::new(
    "http://localhost:8000", "my_collection", 1536,
))?;

// Qdrant (feature: qdrant-integration)
use langchainrust::{QdrantVectorStore, QdrantConfig, QdrantDistance};
let store = QdrantVectorStore::new(QdrantConfig::new("http://localhost:6334", "my_collection")
    .with_vector_size(1536)
    .with_distance(QdrantDistance::Cosine))?;
}

Document & Builder Pattern

#![allow(unused)]
fn main() {
use langchainrust::{Document, VectorStoreBuilder, VectorStoreType};

let doc = Document::new("Hello, world!")
    .with_metadata("source", "test")
    .with_id("doc-1");

// Builder pattern
let store = VectorStoreBuilder::in_memory().build().await?;
let store = VectorStoreBuilder::file_backed("./vectors.json", 1536).build().await?;
}

See also

Embeddings

LangChainRust provides a unified Embeddings trait with multiple backends for generating text embedding vectors.

Supported Providers

ProviderStructDefault ModelDimensionFeature Gate
OpenAIOpenAIEmbeddingstext-embedding-ada-0021536always
DeepSeekDeepSeekEmbeddingsdeepseek-embedding1536always
QwenQwenEmbeddingstext-embedding-v11536always
CohereCohereEmbeddingsembed-english-v3.01024always
FastEmbedFastEmbedEmbeddingsBGESmallENV15384fastembed
BagOfWordsBagOfWordsEmbeddingslocal-bowconfigurablealways
MockMockEmbeddingsmock-embeddingsconfigurablealways

Embeddings Trait

#![allow(unused)]
fn main() {
#[async_trait]
pub trait Embeddings: Send + Sync {
    async fn embed_query(&self, text: &str) -> Result<Vec<f32>, EmbeddingError>;
    async fn embed_documents(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>, EmbeddingError>;
    fn dimension(&self) -> usize;
    fn model_name(&self) -> &str;
}
}

OpenAI Embeddings

#![allow(unused)]
fn main() {
use langchainrust::{OpenAIEmbeddings, OpenAIEmbeddingsConfig};

// From environment (OPENAI_API_KEY)
let embedder = OpenAIEmbeddings::from_env_result()?;

// Explicit config
let embedder = OpenAIEmbeddings::new(
    OpenAIEmbeddingsConfig::new("sk-...")
        .with_model("text-embedding-3-small")
        .with_base_url("https://api.openai.com/v1"),
);

let vec = embedder.embed_query("What is Rust?").await?;
let vecs = embedder.embed_documents(&["doc1 text", "doc2 text"]).await?;
}

FastEmbed (Local ONNX)

#![allow(unused)]
fn main() {
use langchainrust::FastEmbedEmbeddings;
use fastembed::EmbeddingModel;

// Default model (BGESmallENV15, 384-dim)
let embedder = FastEmbedEmbeddings::default_model()?;

// Specific model
let embedder = FastEmbedEmbeddings::with_model(EmbeddingModel::BGESmallENV15)?;
let vec = embedder.embed_query("hello world").await?;
}

BagOfWords & Mock

#![allow(unused)]
fn main() {
use langchainrust::{BagOfWordsEmbeddings, MockEmbeddings};

// BagOfWords (no API needed, FNV-1a hash based)
let embedder = BagOfWordsEmbeddings::default_dim(); // 256-dim
let embedder = BagOfWordsEmbeddings::new(128);

// Mock (deterministic pseudo-random for testing)
let embedder = MockEmbeddings::new(1536);
}

Cosine Similarity

#![allow(unused)]
fn main() {
use langchainrust::cosine_similarity;

let sim = cosine_similarity(&vec_a, &vec_b)?;
// Returns f64 in range [-1.0, 1.0]
}

Memory

LangChainRust provides conversation memory management with multiple strategies: buffer, window, summary, summary-buffer, persistent, and context window.

Memory Types

TypeStructDescriptionUse Case
BufferConversationBufferMemoryStores all messagesShort conversations
WindowConversationBufferWindowMemoryKeeps last k roundsBounded history
SummaryConversationSummaryMemoryLLM-compressed summaryLong conversations
SummaryBufferConversationSummaryBufferMemorySummary + recent messagesBalanced approach
PersistentPersistentMemory traitLoad/save to storageCross-session
ContextWindowContextWindowToken-budget fittingLLM context limits
VectorStoreVectorStoreRetrieverMemorySemantic retrievalLarge history

Buffer & Window Memory

#![allow(unused)]
fn main() {
use langchainrust::{ConversationBufferMemory, ConversationBufferWindowMemory};
use std::collections::HashMap;

// Buffer -- stores everything
let mut memory = ConversationBufferMemory::new()
    .with_input_key("input".to_string())
    .with_output_key("output".to_string());

memory.save_context(
    &HashMap::from([("input".into(), "What is Rust?".into())]),
    &HashMap::from([("output".into(), "A systems language.".into())]),
).await?;

let loaded = memory.load_memory_variables(&HashMap::new()).await?;

// Window -- keeps last k rounds
let mut memory = ConversationBufferWindowMemory::new(5); // last 5 rounds
}

Summary & SummaryBuffer Memory

#![allow(unused)]
fn main() {
use langchainrust::{ConversationSummaryMemory, ConversationSummaryBufferMemory, OpenAIChat};

// Summary -- LLM compresses history
let mut memory = ConversationSummaryMemory::new(llm)
    .with_max_recent_turns(2);

// SummaryBuffer -- hybrid with token limit
let mut memory = ConversationSummaryBufferMemory::new(llm, 4000)
    .with_memory_key("history");
}

ContextWindow

#![allow(unused)]
fn main() {
use langchainrust::{ContextWindow, Strategy};

// Truncate strategy (default) -- drops oldest, preserves system messages
let cw: ContextWindow<OpenAIChat> = ContextWindow::new(4096)?;
let fitted = cw.fit(messages).await?;

// Summarize strategy -- LLM compresses older messages
let cw = ContextWindow::with_strategy(4096, Strategy::summarize(llm))?;
let fitted = cw.fit(messages).await?;
}

Budget semantics (Truncate): System 消息恒保留且不占预算;若 System 自身超预算,原样返回(可能超预算)。

Persistent Memory

#![allow(unused)]
fn main() {
use langchainrust::{PersistentMemory, PersistenceConfig};

let config = PersistenceConfig::new()
    .with_auto_save(true)
    .with_token_limit(4000);

// Implement PersistentMemory for your storage backend
// Methods: load_from_store, save_to_store, delete_session, session_exists
}

ChatMessageHistory

#![allow(unused)]
fn main() {
use langchainrust::ChatMessageHistory;

let mut history = ChatMessageHistory::new();
history.add_user_message("Hello");
history.add_ai_message("Hi there!");
history.add_system_message("You are helpful.");
let messages = history.messages();
}

Chains

Chains compose operations into pipelines. LangChainRust provides LLMChain, SequentialChain, ConversationChain, RouterChain, RetrievalQA, and document chains.

Chain Types

ChainStructDescription
LLMChainLLMChain<M>Prompt + LLM (simplest chain)
SequentialChainSequentialChainExecute chains in sequence
ConversationChainConversationChain<M>LLM with conversation memory
RouterChainRouterChainKeyword-based routing
LLMRouterChainLLMRouterChain<M>LLM-based routing
RetrievalQARetrievalQA<M>RAG question answering
ConversationRetrievalChainConversationRetrievalChain<M>RAG with memory
StuffDocumentsChainStuffDocumentsChain<M>All docs in one prompt
RefineDocumentsChainRefineDocumentsChain<M>Iterative refinement
MapReduceDocumentsChainMapReduceDocumentsChain<M>Parallel map + reduce
MapRerankDocumentsChainMapRerankDocumentsChain<M>Parallel map + rank

LLMChain

#![allow(unused)]
fn main() {
use langchainrust::{LLMChain, LLMChainBuilder};
use std::collections::HashMap;
use serde_json::Value;

let chain = LLMChain::new(llm, "Explain this topic: {topic}")
    .with_input_key("topic")
    .with_output_key("text");

let mut inputs = HashMap::new();
inputs.insert("topic".to_string(), Value::String("Rust ownership".to_string()));
let result = chain.invoke(inputs).await?;

// Builder pattern
let chain = LLMChainBuilder::new(llm, "Summarize: {text}")
    .input_key("text")
    .output_key("summary")
    .build();
}

SequentialChain

#![allow(unused)]
fn main() {
use langchainrust::SequentialChain;

let pipeline = SequentialChain::new()
    .add_chain(Arc::new(chain1), vec!["topic"], vec!["features"])
    .add_chain(Arc::new(chain2), vec!["features"], vec!["summary"]);

let result = pipeline.invoke(inputs).await?;
}

ConversationChain

#![allow(unused)]
fn main() {
use langchainrust::{ConversationChain, ConversationBufferMemory};

let chain = ConversationChain::builder(llm)
    .memory(ConversationBufferMemory::new())
    .system_prompt("You are a helpful assistant.")
    .build();

let reply = chain.predict("What is Rust?").await?;
let reply2 = chain.predict("Tell me more about ownership.").await?; // remembers context
}

RetrievalQA

#![allow(unused)]
fn main() {
use langchainrust::RetrievalQA;

let qa = RetrievalQA::new(llm, retriever)
    .with_k(4)
    .with_return_source_documents(true);

let answer = qa.query("What is LangChain?").await?;
let (answer, sources) = qa.query_with_sources("What is LangChain?").await?;
}

Document Chains

#![allow(unused)]
fn main() {
use langchainrust::{StuffDocumentsChain, RefineDocumentsChain, MapReduceDocumentsChain};

// Stuff -- all documents in one prompt
let chain = StuffDocumentsChain::new(llm)
    .with_prompt_template("Summarize: {context}\nQuestion: {input}");
let result = chain.invoke_with_documents(docs, "What is the main idea?").await?;

// Refine -- iterative refinement
let chain = RefineDocumentsChain::new(llm)
    .with_initial_prompt("Answer based on: {context}\nQuestion: {input}")
    .with_refine_prompt("Refine this answer: {existing_answer}\nNew context: {context}");

// MapReduce -- parallel map then reduce
let chain = MapReduceDocumentsChain::new(llm)
    .with_map_prompt("Summarize: {context}")
    .with_reduce_prompt("Combine summaries: {context}");
}

Tools

LangChainRust ships with 12+ built-in tools and a #[tool] macro for custom tools.

Built-in Tools

ToolStructNameKey Input
CalculatorCalculatorcalculatorexpression: String
DateTimeDateTimeTooldatetimeoperation, datetime, unit
MathSimpleMathToolmathoperation, value, value2
Python REPLPythonREPLToolpython_replcode: String
DuckDuckGo SearchDuckDuckGoSearchToolweb_searchquery: String
URL FetchURLFetchToolurl_fetchurl, operation
WikipediaWikipediaToolwikipediaquery, lang
FileFileToolfile_operationop, path, content
HTTPHTTPToolhttp_requesturl, method, body
SQLSQLToolsql_querySQL string
Computer UseComputerUseToolcomputer_useaction, coordinate
SandboxSandboxTool<S>code_interpretercode: String

Using Tools with Agents

#![allow(unused)]
fn main() {
use langchainrust::tools::{Calculator, DateTimeTool, SimpleMathTool};
use langchainrust::{FunctionCallingAgent, AgentExecutor, BaseAgent};
use std::sync::Arc;

let tools: Vec<Arc<dyn BaseTool>> = vec![
    Arc::new(Calculator::new()),
    Arc::new(DateTimeTool::new()),
    Arc::new(SimpleMathTool::new()),
];

let agent = FunctionCallingAgent::new(llm, tools.clone(), None);
let executor = AgentExecutor::new(Arc::new(agent) as Arc<dyn BaseAgent>, tools)
    .with_max_iterations(5);

let result = executor.invoke("Calculate 15 * 4 and tell me the time".to_string()).await?;
}

Security Features

HTTPTool: SSRF protection by default – blocks private IPs (127.0.0.1, 10.x, 192.168.x, cloud metadata).

FileTool: Path traversal prevention, extension whitelist, size limits.

SQLTool: SELECT-only, blocks SQL comments and dangerous patterns (sleep, benchmark, exec).

PythonREPLTool: Disabled by default; blocks dangerous imports (os, subprocess, socket); configurable timeout.

Sandbox

#![allow(unused)]
fn main() {
use langchainrust::tools::sandbox::{LocalSandbox, CodeSandbox, Language, SandboxTool};

let sandbox = LocalSandbox::new();
let result = sandbox.run("print(2 + 2)", Language::Python, 10_000).await?;
// result.stdout, result.stderr, result.exit_code

let tool = SandboxTool::new(LocalSandbox::new(), Language::Python).with_timeout(10_000);
}

Computer Use

#![allow(unused)]
fn main() {
use langchainrust::tools::ComputerUseTool;

let tool = ComputerUseTool::new_anthropic("sk-...", 1920, 1080);
// Actions: screenshot, click, type, scroll, key_press, wait
}

See also

Guardrails

Guardrails provide input/output validation to protect production agents from malicious input and sensitive information leakage.

Core Concepts

ConceptTypeDescription
InputGuardrailTraitValidates agent input before execution
OutputGuardrailTraitValidates agent output after execution
GuardrailsConfigStructConfigures input + output guardrails
GuardrailRunnerStructExecutes guardrail validation pipeline
GuardedAgentStructAgent wrapper with automatic guardrail enforcement

Guardrail Result

Input and output guardrails return different result types, so the type system enforces that Modify can only be produced on the output side.

#![allow(unused)]
fn main() {
// Input side: no Modify variant — input guardrails can only Pass or Block
pub enum InputGuardrailResult {
    Pass,
    Block { reason: String },
}

// Output side: Modify is the only place rewriting is allowed
pub enum OutputGuardrailResult {
    Pass,
    Block { reason: String },
    Modify { new_value: String },
}
}

Built-in Validators

ValidatorSideDescription
MaxLengthGuardrailInputBlocks if input exceeds character limit
ForbiddenWordsGuardrailInputBlocks on forbidden word match (case-insensitive)
SensitiveInfoGuardrailOutputDetects API keys, emails, credit cards, sensitive keywords

Usage

#![allow(unused)]
fn main() {
use langchainrust::{
    GuardrailsConfig, GuardedAgent, GuardrailRunner,
    MaxLengthGuardrail, ForbiddenWordsGuardrail, SensitiveInfoGuardrail,
};
use std::sync::Arc;

// Configure guardrails
let config = GuardrailsConfig::new()
    .with_input(Arc::new(MaxLengthGuardrail::new(1000)))
    .with_input(Arc::new(ForbiddenWordsGuardrail::new(vec![
        "hack".into(), "exploit".into(),
    ])))
    .with_output(Arc::new(SensitiveInfoGuardrail::new()));

// Wrap agent with guardrails
let mut guarded = GuardedAgent::new(
    Arc::new(executor),
    config,
);

let result = guarded.invoke("What is my password?".to_string()).await?;
// Returns Err(GuardrailError::Blocked { .. }) if guardrail triggers

// Check violations
let violations = guarded.violations();
}

SensitiveInfoGuardrail Details

Detects:

  • Sensitive keywords: “password”, “token”, “secret”, “api_key”, “credential” (plus Chinese equivalents)
  • OpenAI-style API keys (regex sk-...)
  • Email addresses
  • Credit card numbers (with Luhn validation)
#![allow(unused)]
fn main() {
let guardrail = SensitiveInfoGuardrail::new()
    .with_keywords(vec!["internal_key".into()]);
}

Custom Guardrails

#![allow(unused)]
fn main() {
use langchainrust::{InputGuardrail, InputGuardrailResult};

struct NoSQLInjection;

#[async_trait]
impl InputGuardrail for NoSQLInjection {
    fn name(&self) -> &str { "no_sql_injection" }
    async fn validate(&self, input: &str) -> InputGuardrailResult {
        if input.contains("{$") || input.contains("{$gt:") {
            InputGuardrailResult::Block { reason: "Potential NoSQL injection".into() }
        } else {
            InputGuardrailResult::Pass
        }
    }
}
}

输入护栏返回 [InputGuardrailResult](没有 Modify 变体),输出护栏返回 [OutputGuardrailResult](含 Modify)。类型系统强制“修改结果仅输出侧可产生“。

Evaluation

LangChainRust provides 10 built-in evaluators and an LLM-as-judge for quantifying prompt and model changes.

Evaluators

EvaluatorTypeDescription
ExactMatchStringReturns 1.0 for exact match, 0.0 otherwise
StringDistanceStringLevenshtein distance normalized to 0.0-1.0
ContainsKeywordRuleChecks for keyword presence (any or all)
RegexMatchRuleMatches against a regex pattern
LengthCheckRuleValidates output length within min/max bounds
BleuNLPBLEU score with character-level Chinese support
EmbeddingSimilaritySemanticCosine similarity of embeddings mapped to 0.0-1.0
LLMAsJudgeLLMLLM scores predictions on a configurable rubric
FaithfulnessLLMVerifies claims against reference context
PairwiseJudgeLLMCompares two predictions with position-bias mitigation

Core Types

#![allow(unused)]
fn main() {
pub struct Score { pub value: f64, pub label: Option<String> }  // 0.0-1.0
pub struct Example {
    pub input: String,
    pub reference: String,
    pub contexts: Vec<String>,  // retrieved RAG contexts, rank order; [] for non-RAG tasks
}
pub struct Dataset { pub examples: Vec<Example> }
// Report 携带原文 + 汇总(均值/标准差)+ 失败清单,可反序列化(落盘二次分析)
pub struct Report {
    pub per_example: Vec<ExampleReport>,          // 每条含 input/reference/prediction 原文
    pub summary: HashMap<String, ScoreSummary>,   // 每个评测器: mean / std / count
    pub failures: Vec<FailureRecord>,             // 逐条容错记录(为空 = 全部成功)
}
pub struct ExampleReport {
    pub index: usize,
    pub input: String,
    pub reference: String,
    pub prediction: String,
    pub scores: HashMap<String, Score>,
}
pub struct ScoreSummary { pub mean: f64, pub std: f64, pub count: usize }
}

Batch Evaluation

#![allow(unused)]
fn main() {
use langchainrust::{EvalRunner, ExactMatch, StringDistance, Dataset, Example, Evaluator};

let dataset = Dataset::new(vec![
    Example::new("2+2?", "4"),
    Example::new("Capital of France?", "Paris"),
]);

let runner = EvalRunner::new(vec![
    Box::new(ExactMatch),
    Box::new(StringDistance),
]);

let report = runner.run(&dataset, &predictor).await?;
// report.summary: {"exact_match": ScoreSummary { mean: 0.5, .. }, "string_distance": { .. }}
// 单条 predict / 打分失败不中断整批,记入 report.failures(P1-3)
}

Individual Evaluators

#![allow(unused)]
fn main() {
use langchainrust::{ContainsKeyword, RegexMatch, LengthCheck, Bleu, LLMAsJudge, Faithfulness};

// Rule-based
let eval = ContainsKeyword::new(vec!["Rust".into()]).all_required(true);
let score = eval.eval("What language?", "Rust is great", "").await?;

let eval = RegexMatch::new(r"\d+")?;
let score = eval.eval("Calculate", "The answer is 42", "").await?;

let eval = LengthCheck::new().min(10).max(100);

// BLEU
let eval = Bleu::new().with_char_level(true); // for Chinese

// LLM-as-judge
let eval = LLMAsJudge::new(judge_llm)
    .with_rubric("Score accuracy on a scale of 0-10")
    .with_max_score(10);

// Faithfulness (hallucination detection)
let eval = Faithfulness::new(judge_llm)
    .with_llm_split(true)  // LLM-based claim splitting
    .with_empty_score(0.0);
}

Pairwise Comparison

#![allow(unused)]
fn main() {
use langchainrust::{PairwiseJudge, Verdict};

let judge = PairwiseJudge::new(llm).with_rubric("Which answer is more accurate?");
let verdict = judge.compare("What is X?", "Answer A", "Answer B").await?;
// Verdict::AWins, BWins, or Tie
}

PairwiseJudge 也实现了 PairwiseEvaluator trait(P1-1),可以直接进 EvalRunner 统一报告:成对评测器以 (prediction, reference) 作为 A/B 两个候选(竞技场用法把待比 答案放进 reference 槽),得分 1.0 = prediction 优、0.5 = 平局、0.0 = reference 优:

#![allow(unused)]
fn main() {
let runner = EvalRunner::new(vec![/* 单点评测器 */])
    .with_pairwise(vec![Box::new(judge)]);
let report = runner.run(&dataset, &predictor).await?;
}

Trace → Golden → Regression Gate

Offline datasets go stale; the companion lc-testkit crate closes the loop from production traces to a checked-in golden set to a CI regression gate. It is a direct/dev dependency (lc-testkit = "0.24") rather than part of the facade.

The mapping from a recorded model exchange to one Example is fixed:

  • input ← the last human message in the recorded request history (agent-loop re-requests still carry the real user turn there);
  • reference ← the recorded assistant text;
  • contexts ← non-empty tool-result messages in order, so RAGAS-style context precision/recall evaluators can score retrieval too.

Agent loops record several exchanges per user turn, including pure tool-call responses with empty text — is_scoring_candidate filters those out (GoldenError names the row that has no human message or empty reference).

// One recording file (from RecordingProvider) -> golden JSONL checked into the repo.
let exchanges = read_exchanges("traces/recorded.jsonl")?;
let scored: Vec<_> = exchanges.iter().filter(|e| is_scoring_candidate(e)).cloned().collect();
let dataset = golden_dataset(&scored)?;
write_golden_jsonl(&dataset, "eval/golden.jsonl")?;

// Fully offline smoke/baseline run: a FIFO ReplayPredictor over the same file
// reproduces every recorded reference exactly — zero network, no API key.
let (dataset, replay) = replay_golden_from_file("traces/recorded.jsonl", ReplayStrategy::Fifo)?;
let report = EvalRunner::new(vec![Box::new(ExactMatch)])
    .run(&dataset, &replay)
    .await?;

For the real regression run, load the checked-in JSONL (Dataset::from_jsonl) with a predictor wrapping the new prompt/model, then compare the two reports:

#![allow(unused)]
fn main() {
use langchainrust::{compare_reports, Report};

// baseline = serialized Report from the known-good build (Report round-trips JSON).
let cmp = compare_reports(&baseline, &candidate, 0.02);
assert!(!cmp.is_regressed(), "regressed:\n{}", cmp.to_table());
}

A metric regresses when its mean drops by strictly more than the tolerance — delta < -tolerance, so a boundary move of exactly -0.02 still passes. ReportComparison also lists per-evaluator MetricDeltas plus added / dropped evaluator names, so a candidate that silently stops running an evaluator is visible in the same table.

Verifier / Best-of-N

A judge answers “is this good?”; a verifier answers “how strongly does this satisfy the criteria?” with a score fine-grained enough to compare candidates.

#![allow(unused)]
fn main() {
use langchainrust::evaluation::{best_of_n, LogprobVerifier};

let verifier = LogprobVerifier::new(openai_chat)  // requires a logprob-capable provider
    .with_max_score(9)
    .with_top_logprobs(5);

let candidates = vec![candidate_a, candidate_b, candidate_c];
let result = best_of_n(&verifier, "task description", &candidates).await?;
}

LogprobVerifier expands the first score-bearing token into its top_logprobs distribution and takes the expectation instead of the argmax. For providers without logprobs (Anthropic, most local servers), JudgeVerifier parses a single integer and reports confidence = 0.25 so callers do not over-trust it.

Judge Reliability

Judge output needs position-bias and chance correction before it can gate a release:

#![allow(unused)]
fn main() {
use langchainrust::evaluation::{
    cohen_kappa, order_swap_consistent, reliability_report, same_family, Verdict,
};

// A winner must survive both A/B and B/A.
let winner = order_swap_consistent(Verdict::AWins, Verdict::BWins); // AWins

// Chance-corrected agreement.
let kappa = cohen_kappa(&[true, false, true], &[true, false, false]);

// Same-family self-preference risk.
assert!(same_family("gpt-5", "gpt-4o"));
assert!(!same_family("gpt-5", "claude-3"));

let report = reliability_report(&original, &swapped);
}

cohen_kappa returns None for empty, mismatched or degenerate (single-label) samples. reliability_report aggregates agreement, kappa and ties.

Tracing & Callbacks

LangChainRust provides a callback system for observability, tracing, and monitoring with handlers for console, file, LangSmith, and OpenTelemetry.

Callback System

ComponentDescription
CallbackHandlerTrait with lifecycle hooks (LLM, chain, tool, retriever)
CallbackManagerManages multiple handlers, dispatches events
RunTreeStructured run tracking with parent-child relationships
RunTypeLlm, Chain, Tool, Retriever, Embedding, Prompt, Parser

Callback Handlers

HandlerStructOutput
ConsoleStdOutHandlerPrints to stdout
FileFileCallbackHandlerWrites to file (Plain/JSON/JSONLines)
LangSmithLangSmithHandlerSends to LangSmith platform
OpenTelemetryOtelHandlerEmits OTel spans with GenAI SemConv

Setup

#![allow(unused)]
fn main() {
use langchainrust::{
    CallbackManager, StdOutHandler, FileCallbackHandler,
    LangSmithHandler, LogFormat,
};
use std::sync::Arc;

let manager = CallbackManager::new()
    .add_handler(Arc::new(StdOutHandler::new()))
    .add_handler(Arc::new(FileCallbackHandler::new("./traces.jsonl")?
        .with_format(LogFormat::JsonLines)))
    .add_handler(Arc::new(LangSmithHandler::from_env()?));
}

LangSmith Configuration

Environment variables:

VariableDescriptionDefault
LANGSMITH_API_KEYAPI key (starts with ls_)Required
LANGSMITH_TRACINGEnable tracingtrue
LANGSMITH_PROJECTProject namedefault
LANGSMITH_ENDPOINTAPI endpointLangSmith official
LANGSMITH_WORKSPACE_IDWorkspace IDFor org accounts

OpenTelemetry (GenAI SemConv)

#![allow(unused)]
fn main() {
// Feature: opentelemetry
use langchainrust::OtelHandler;

let handler = OtelHandler::from_global("langchainrust");
// Sets GenAI Semantic Convention attributes:
// gen_ai.system, gen_ai.request.model, gen_ai.response.model,
// gen_ai.response.finish_reason, gen_ai.client.token.usage.*,
// gen_ai.request.max_tokens, gen_ai.request.temperature,
// gen_ai.operation.name, gen_ai.tool.name
}

Tracing Subsystem

#![allow(unused)]
fn main() {
use langchainrust::{Tracer, SpanKind, SpanGuard, InMemoryTracingBackend};

let backend = Arc::new(InMemoryTracingBackend::new());
let tracer = Tracer::new(backend.clone());

let mut span = tracer.start("llm_call", SpanKind::Llm);
span.with_tokens(SpanTokenUsage { prompt_tokens: 10, completion_tokens: 20, total_tokens: 30 });
span.end(); // RAII: ends on drop if not called

// Inspect traces
let spans = backend.spans();
}

RunTree

#![allow(unused)]
fn main() {
use langchainrust::{RunTree, RunType};

let run = RunTree::new("my_chain", RunType::Chain, serde_json::json!({"input": "hello"}))
    .with_tag("production")
    .with_project("my-project");

let child = run.create_child("llm_step", RunType::Llm, serde_json::json!({"prompt": "..."}));
}

LCEL Pipelines

The LangChain Expression Language (LCEL) provides pipe-based composition for Runnable components, inspired by Python LangChain’s prompt | llm | parser syntax.

Core Types

TypeDescription
Runnable<I, O>Base execution trait (invoke, batch, stream, transform)
RunnableExtExtension providing pipe() composition
RunnableSequence<I, O>Ordered pipeline of chained steps
RunnableLambda<I, O>Closure wrapper (sync/async)
RunnablePassthrough<I>Identity pass-through with true streaming
RunnableParallel<I>Fan-out/fan-in, results in HashMap<String, Value>
RunnableBranch<I, O>Conditional routing (first match wins)
RunnableBinding<I, O>Pre-bind config/kwargs to a Runnable
RunnableWithFallbacks<I, O>Primary + fallback chain
RunnableAssignInject key-value pairs into HashMap mid-pipeline
RunnableRetry<I, O>Retry with exponential backoff

Pipe Composition

#![allow(unused)]
fn main() {
use langchainrust::{RunnableExt, RunnableLambda, RunnablePassthrough};

// Chain runnables with pipe()
let chain = prompt.pipe(llm).pipe(parser);

// Lambda steps
let step1 = RunnableLambda::new_sync(|input: String| input.to_uppercase());
let step2 = RunnableLambda::new_sync(|input: String| format!("Result: {}", input));
let pipeline = step1.pipe(step2);

let result = pipeline.invoke("hello".to_string(), None).await?;
// "Result: HELLO"
}

Parallel & Branch

#![allow(unused)]
fn main() {
use langchainrust::{RunnableParallel, RunnableBranch, RunnablePassthrough};

// Parallel fan-out
let parallel = RunnableParallel::<String>::new()
    .with("upper", RunnableLambda::new_sync(|s: String| s.to_uppercase()))
    .with("len", RunnableLambda::new_sync(|s: String| s.len().to_string()));

let result = parallel.invoke("hello".to_string(), None).await?;
// HashMap {"upper": "HELLO", "len": "5"}

// Branch (conditional routing)
let branch = RunnableBranch::new(default_step)
    .when(condition_runnable, branch_a)
    .when(condition_runnable, branch_b);
}

Fallbacks & Retry

#![allow(unused)]
fn main() {
use langchainrust::{RetryConfig, RetryOn};

// Fallbacks
let chain = primary_llm.with_fallbacks(vec![fallback_llm1, fallback_llm2]);

// Retry with exponential backoff
let retry_config = RetryConfig::new(3)
    .with_initial_delay(Duration::from_millis(500))
    .with_max_delay(Duration::from_secs(10))
    .with_backoff_multiplier(2.0)
    .with_retry_on(RetryOn::TransientErrors); // 429, 500, 502, 503, 504

let chain = llm.with_retry(retry_config);
}

Assign & Binding

#![allow(unused)]
fn main() {
use langchainrust::RunnableAssign;

// Inject context into HashMap pipeline
let assign = RunnableAssign::new()
    .with("context", retriever_step);

// Bind config
let bound = llm.pipe(RunnableConfig::new().with_tag("production"));
}

Stream Events

#![allow(unused)]
fn main() {
use langchainrust::LcelStreamEvent;

// LCEL streaming via transform()
let stream = chain.transform(input_stream, config).await?;
while let Some(event) = stream.next().await {
    match event? {
        LcelStreamEvent::OnLlmStart { .. } => { /* ... */ }
        LcelStreamEvent::OnLlmStream { token } => print!("{}", token),
        LcelStreamEvent::OnLlmEnd { .. } => { /* ... */ }
        LcelStreamEvent::OnToolEnd { output } => { /* ... */ }
        LcelStreamEvent::OnChainEnd { output } => { /* ... */ }
    }
}
}

Streaming

LangChainRust supports streaming at multiple levels: LLM token streaming, agent event streaming, chain streaming, and LCEL pipeline streaming.

Streaming Levels

LevelMethodOutput
LLMstream_chat()Token-by-token text
AgentAgentExecutor::stream()AgentStreamEvent stream
ChainBaseChain::stream()StreamToken stream
LCELRunnable::stream() / transform()LcelStreamEvent stream
Structured Outputstream_structured_output()Partial T values

LLM Streaming

#![allow(unused)]
fn main() {
use langchainrust::openai::OpenAIChat;

let llm = OpenAIChat::new(OpenAIConfig { streaming: true, ..Default::default() });
let messages = vec![
    Message::system("You are a helpful assistant."),
    Message::human("Explain Rust ownership."),
];

let mut stream = llm.stream_chat(messages, None).await?;
while let Some(chunk) = stream.next().await {
    let token = chunk?;
    print!("{}", token.content);
}
}

Agent Streaming

#![allow(unused)]
fn main() {
use langchainrust::AgentExecutor;

let stream = executor.stream(input);
while let Some(event) = stream.next().await {
    match event? {
        AgentStreamEvent::Text { content } => print!("{}", content),
        AgentStreamEvent::ToolStart { name, input } => {
            println!("[Tool: {}] Input: {}", name, input);
        }
        AgentStreamEvent::ToolEnd { name, output } => {
            println!("[Tool: {}] Output: {}", name, output);
        }
        AgentStreamEvent::FinalAnswer { content } => {
            println!("Answer: {}", content);
        }
        AgentStreamEvent::PipelineStep { step, detail } => { /* ... */ }
        AgentStreamEvent::Error { message } => eprintln!("Error: {}", message),
    }
}
}

Chain Streaming

#![allow(unused)]
fn main() {
use langchainrust::{LLMChain, StreamToken};

let chain = LLMChain::new(llm, "Explain: {topic}");
let mut stream = chain.stream(inputs).await?;
while let Some(token) = stream.next().await {
    match token? {
        StreamToken { token, is_final: false } => print!("{}", token),
        StreamToken { token, is_final: true } => println!("\n[Done]"),
    }
}
}

LCEL Streaming

#![allow(unused)]
fn main() {
use langchainrust::LcelStreamEvent;

let stream = pipeline.stream(input, None).await?;
while let Some(event) = stream.next().await {
    match event? {
        LcelStreamEvent::OnLlmStream { token } => print!("{}", token),
        LcelStreamEvent::OnToolEnd { output } => println!("Tool: {}", output),
        LcelStreamEvent::OnChainEnd { output } => println!("Chain: {:?}", output),
        _ => {}
    }
}
}

Structured Output Streaming

#![allow(unused)]
fn main() {
use langchainrust::StreamingStructuredOutputExt;

let stream = llm.stream_structured_output::<MyStruct>(schema, "prompt").await?;
while let Some(partial) = stream.next().await {
    let partial: MyStruct = partial?;
    // Fields populate incrementally as tokens arrive
}
}

Tool Macro

The #[tool] procedural macro auto-generates BaseTool + Tool implementations from a simple function, reducing ~50 lines of boilerplate to 3 lines.

Basic Usage

#![allow(unused)]
fn main() {
use langchainrust::{tool, BaseTool, Tool, ToolError};

#[tool(description = "Greets a person by name")]
fn greet(
    #[param(desc = "The name of the person to greet")]
    name: String,
) -> Result<String, ToolError> {
    Ok(format!("Hello, {}!", name))
}
}

This generates:

  • GreetTool struct (with new() and Default)
  • GreetInput struct (with Deserialize + JsonSchema)
  • impl Tool for GreetTool (type-safe invoke)
  • impl BaseTool for GreetTool (string-based run, name, description, args_schema)
  • The original greet function is preserved

Parameters

#![allow(unused)]
fn main() {
#[tool(description = "Adds two numbers")]
fn add_numbers(
    #[param(desc = "The first number")]
    a: i64,
    #[param(desc = "The second number")]
    b: i64,
) -> Result<i64, ToolError> {
    Ok(a + b)
}
// Generates: AddNumbersTool, AddNumbersInput
}

Parameter type rules:

TypeSchemaRequired
String, i64, f64, boolPrimitiveYes
Option<T>NullableNo
Vec<T>ArrayYes

Optional Parameters

#![allow(unused)]
fn main() {
#[tool(description = "Says hello with optional title")]
fn hello_optional(
    #[param(desc = "The person's name")]
    name: String,
    #[param(desc = "Optional title (e.g., Dr., Mr.)")]
    title: Option<String>,
) -> Result<String, ToolError> {
    match title {
        Some(t) => Ok(format!("Hello, {} {}!", t, name)),
        None => Ok(format!("Hello, {}!", name)),
    }
}
// Generates: HelloOptionalTool, HelloOptionalInput
}

Using Generated Tools

#![allow(unused)]
fn main() {
// Type-safe invocation via Tool trait
let tool = GreetTool::new();
let input = GreetInput { name: "Alice".into() };
let result = tool.invoke(input).await?;
assert_eq!(result, "Hello, Alice!");

// String-based invocation via BaseTool trait (for agents)
let tool = GreetTool::new();
assert_eq!(tool.name(), "greet");
assert_eq!(tool.description(), "Greets a person by name");
let result = tool.run(r#"{"name":"Bob"}"#.to_string()).await?;
assert_eq!(result, "\"Hello, Bob!\"");

// Get JSON schema for LLM function calling
let schema = tool.args_schema(); // Auto-generated from JsonSchema
}

Attributes Reference

AttributeTargetRequiredDescription
#[tool(description = "...")]FunctionYesTool description shown to the LLM
#[param(desc = "...")]ParameterNoParameter description in JSON schema

The struct and input types are named by converting the function name to PascalCase and appending Tool or Input (e.g., add_numbers becomes AddNumbersTool / AddNumbersInput).

Hooks

Hooks provide a composable middleware mechanism for intercepting agent lifecycle events: completions, tool calls, streaming, and errors.

AgentHook Trait

#![allow(unused)]
fn main() {
#[async_trait]
pub trait AgentHook: Send + Sync {
    fn on_before_completion(&self, ctx: &mut CompletionContext) -> CompletionAction;
    fn on_after_completion(&self, ctx: &mut CompletionResult) -> Result<(), HookError>;
    fn on_before_tool_call(&self, ctx: &mut ToolCallContext) -> ToolCallAction;
    fn on_after_tool_call(&self, ctx: &mut ToolResultContext) -> Result<(), HookError>;
    fn on_stream_chunk(&self, chunk: &str) -> StreamAction;
    fn on_agent_start(&self, input: &str) -> Result<(), HookError>;
    fn on_agent_end(&self, output: &str) -> Result<(), HookError>;
    fn on_error(&self, error: &HookError) -> ErrorAction;
}
}

All methods have default no-op implementations – implement only what you need.

Built-in Hooks

HookStructPurpose
ApprovalApprovalHookRequire manual approval before tool calls
Content FilterContentFilterHookFilter sensitive words from stream output
LoggingLoggingHookLog all lifecycle events
Prompt InjectionPromptInjectionHookDetect & sanitize prompt injection in tool outputs
Token BudgetTokenBudgetHookEnforce token budget / LLM call quota (rate limiting)

ApprovalHook

#![allow(unused)]
fn main() {
use langchainrust::hooks::ApprovalHook;

// Manual approval (default) -- rejects tool calls
let hook = ApprovalHook::new();

// Auto-approve all tool calls
let hook = ApprovalHook::auto_approve();

// Configurable
let hook = ApprovalHook::new().with_auto_approve(false);
}

ContentFilterHook

#![allow(unused)]
fn main() {
use langchainrust::hooks::ContentFilterHook;

let hook = ContentFilterHook::new(vec![
    "password".into(), "secret".into(), "internal".into(),
])
.with_placeholder("[REDACTED]")  // Replace sensitive words
.with_drop_token(false);         // If true, filter entire chunk instead of replacing
}

PromptInjectionHook

Detects and sanitizes prompt-injection text in tool outputs. A malicious page / retrieved document that says “ignore previous instructions” is replaced with a safe marker before it can reach the next plan() prompt, breaking cross-round pollution.

#![allow(unused)]
fn main() {
use langchainrust::hooks::PromptInjectionHook;

let hook = PromptInjectionHook::new();   // Default injection patterns

// Custom patterns and marker
let hook = PromptInjectionHook::new()
    .with_patterns(vec!["evil-text".into()])
    .with_marker("[BLOCKED:{}]");        // `{}` is replaced with the matched pattern

let detected = hook.detected_count();    // Total injections sanitized so far
}

TokenBudgetHook

Enforces a token budget and/or an LLM call quota before each completion. on_before_completion rejects (aborting the run) when the estimated usage would exceed the budget; on_after_completion accumulates real token usage.

#![allow(unused)]
fn main() {
use langchainrust::hooks::TokenBudgetHook;

let hook = TokenBudgetHook::new(10_000) // Total token budget for the run
    .with_max_calls(20);                // Optional: max LLM calls
}

LoggingHook

#![allow(unused)]
fn main() {
use langchainrust::hooks::LoggingHook;

let hook = LoggingHook::new();           // Basic logging
let hook = LoggingHook::with_tokens();   // Include token usage logging
}

Adding Hooks to Agent

#![allow(unused)]
fn main() {
use langchainrust::{AgentExecutor, BaseAgent};
use langchainrust::hooks::{ApprovalHook, ContentFilterHook, LoggingHook};

let executor = AgentExecutor::new(agent, tools)
    .hook(ApprovalHook::new())                                          // Require approval
    .hook(ContentFilterHook::new(vec!["secret".into()]))                // Filter words
    .hook(LoggingHook::with_tokens());                                   // Log events
}

Tool Permission Policy

Beyond hooks, AgentExecutor can enforce a tool permission policy before every tool execution — two gates:

  1. Permission tier (ToolRisk): Safe < Standard < Dangerous. A tool whose risk exceeds max_permitted is rejected.
  2. Sandbox gate: a Dangerous tool must be declared as having been moved into a sandboxed environment (ToolPolicy::sandboxed) or it is rejected, preventing dangerous tools from running un-sandboxed. (Actual sandbox backends live in lc-tools — SandboxTool / CodeSandbox — which the operator wires onto the tool.)
#![allow(unused)]
fn main() {
use langchainrust::{AgentExecutor, ToolPolicy, ToolRisk};

let policy = ToolPolicy::new()
    .risk("code_interpreter", ToolRisk::Dangerous) // Declare risk tier
    .sandboxed("code_interpreter")                // Declare it runs in a sandbox
    .with_max_permitted(ToolRisk::Standard);      // Gate: reject riskier tools

let executor = AgentExecutor::new(agent, tools).with_tool_policy(policy);
}

A rejected tool call aborts the run with an AgentError. When no policy is configured, tool execution is unrestricted.

Action Types

ActionHookVariants
CompletionActionon_before_completionContinue, Modify { messages }, Reject { reason }
ToolCallActionon_before_tool_callContinue, Modify { name, arguments }, Reject { reason }, Skip
StreamActionon_stream_chunkForward(String), Filter, Replace(String)
ErrorActionon_errorPropagate, Retry, Ignore

Custom Hook

#![allow(unused)]
fn main() {
use langchainrust::hooks::{AgentHook, ToolCallAction, ToolCallContext, StreamAction};

struct RateLimitHook { max_calls: usize, calls: AtomicUsize }

#[async_trait]
impl AgentHook for RateLimitHook {
    fn on_before_tool_call(&self, _ctx: &mut ToolCallContext) -> ToolCallAction {
        if self.calls.fetch_add(1, Ordering::SeqCst) >= self.max_calls {
            ToolCallAction::Reject { reason: "Rate limit exceeded".into() }
        } else {
            ToolCallAction::Continue
        }
    }
}
}

Multimodal

LangChainRust supports multimodal interactions including audio transcription (Whisper), text-to-speech (TTS), image generation (DALL-E), and vision with images.

MultimodalModel Trait

#![allow(unused)]
fn main() {
#[async_trait]
pub trait MultimodalModel: BaseChatModel + Send + Sync {
    async fn transcribe(&self, audio: AudioContent) -> Result<String, MultimodalError>;
    async fn generate_speech(&self, text: &str) -> Result<Vec<u8>, MultimodalError>;
    async fn generate_image(&self, prompt: &str) -> Result<ImageContent, MultimodalError>;
}
}

All methods default to returning MultimodalError::Unsupported. Only OpenAI implements them currently.

Content Types

TypeConstructionDescription
AudioContentfrom_url(), from_base64()Audio input for transcription
ImageContentfrom_url(), from_base64()Image URL or base64 data
FileContentfrom_url(), from_base64(), with_name()Generic file with MIME type
#![allow(unused)]
fn main() {
use langchainrust::{AudioContent, ImageContent, FileContent};

let audio = AudioContent::from_url("https://example.com/audio.wav");
let audio = AudioContent::from_base64("base64data...");

let image = ImageContent::from_url("https://example.com/photo.jpg");
let image = ImageContent::from_base64("iVBORw0KGgo...");

let file = FileContent::from_url("https://example.com/doc.pdf")
    .with_name("report.pdf");
}

Whisper (Speech-to-Text)

#![allow(unused)]
fn main() {
use langchainrust::MultimodalModel;

let audio = AudioContent::from_base64(audio_base64);
let transcript = llm.transcribe(audio).await?;
}

OpenAI-specific with voice options:

#![allow(unused)]
fn main() {
use langchainrust::openai::OpenAIChat;

// Provider-specific method with more control
let transcript = llm.whisper_transcribe(audio).await?;
}

TTS (Text-to-Speech)

#![allow(unused)]
fn main() {
use langchainrust::MultimodalModel;

let speech = llm.generate_speech("Hello, world!").await?;

// Provider-specific with voice selection
use langchainrust::openai::TtsVoice;
let speech = llm.tts_generate("Hello!", TtsVoice::Alloy).await?;
// Voices: Alloy, Echo, Fable, Onyx, Nova, Shimmer
}

DALL-E (Image Generation)

#![allow(unused)]
fn main() {
use langchainrust::MultimodalModel;

let image = llm.generate_image("A cat wearing a hat").await?;

// Provider-specific with size control
use langchainrust::openai::DallEImageSize;
let image = llm.dalle_generate("A cat wearing a hat", DallEImageSize::S1024).await?;
// Sizes: S256, S512, S1024, S1792x1024, S1024x1792
}

Vision (Image Understanding)

#![allow(unused)]
fn main() {
use langchainrust::Message;

let messages = vec![
    Message::system("Describe what you see."),
    Message::human_with_image("What is in this image?", "https://example.com/cat.jpg"),
];
let response = llm.chat(messages, None).await?;
}

Anthropic Images

Anthropic supports image inputs via the same Message::human_with_image pattern. The image content is formatted according to the Anthropic API specification.

Sessions

Sessions provide multi-turn conversation lifecycle management: create, retrieve, archive, and chat with automatic history maintenance.

⚠️ v0.22.0: the event-sourcing path (EventSessionManager) is the recommended API. The legacy SessionManager is #[deprecated] — removal was planned for 0.23.0, but as of 0.24.0 it still ships and compiles; #[allow(deprecated)] silences the warnings. See Event Sourcing (v0.22.0) below.

Core Types

TypeDescription
SessionManagerLegacy API (deprecated in 0.22.0; removal was planned for 0.23.0 but it still ships in 0.24.0)
EventSessionManagerRecommended event-sourced manager (v0.22.0)
SessionA conversation with messages, metadata, and status
SessionStoreTrait for pluggable storage backends (legacy)
EventStoreAppend-only event log trait (v0.22.0)
MemorySessionStore / MemoryEventStoreIn-memory implementations
SessionStatusActive, Archived, Deleted

Basic Usage

#![allow(unused)]
fn main() {
use langchainrust::{SessionManager, MemorySessionStore};
use std::sync::Arc;

let manager = SessionManager::new(Arc::new(MemorySessionStore::new()));

// Create a new session
let session_id = manager.create_session().await?;

// Create a session for a specific user
let session_id = manager.create_session_for("user_123").await?;

// Chat (auto-maintains history)
let reply = manager.chat(&session_id, &llm, "What is Rust?".to_string()).await?;
let reply2 = manager.chat(&session_id, &llm, "Tell me more about ownership.".to_string()).await?;

// Get conversation history
let messages = manager.history(&session_id).await?;

// Archive or clear
manager.archive(&session_id).await?;
manager.clear(&session_id).await?;

// List sessions for a user
let sessions = manager.list_by_user("user_123").await?;
}

Session Struct

#![allow(unused)]
fn main() {
pub struct Session {
    pub id: String,
    pub user_id: Option<String>,
    pub messages: Vec<Message>,
    pub metadata: HashMap<String, Value>,
    pub created_at: DateTime<Utc>,
    pub updated_at: DateTime<Utc>,
    pub status: SessionStatus,
}
}

Methods: add_message(), recent_messages(n), clear(), archive()

Custom SessionStore

#![allow(unused)]
fn main() {
use langchainrust::{SessionStore, Session, SessionError};

struct RedisSessionStore { /* ... */ }

#[async_trait]
impl SessionStore for RedisSessionStore {
    async fn create(&self, session: Session) -> Result<String, SessionError>;
    async fn get(&self, id: &str) -> Result<Option<Session>, SessionError>;
    async fn update(&self, session: &Session) -> Result<(), SessionError>;
    async fn delete(&self, id: &str) -> Result<(), SessionError>;
    async fn list_by_user(&self, user_id: &str) -> Result<Vec<Session>, SessionError>;
}
}

SessionManager Methods

MethodDescription
create_session()Create an anonymous session
create_session_for(user_id)Create a session for a user
get_session(id)Get session by ID
chat(id, llm, message)Send message and get reply (auto-persists)
history(id)Get conversation messages
clear(id)Clear messages
archive(id)Archive session
list_by_user(user_id)List sessions for a user

Event Sourcing (v0.22.0)

The legacy manager is “rewrite-style”: each turn reads the whole Session, mutates it, and writes it back — a crash mid-way loses messages, and there is no way to fork or time-travel. 0.22.0 rewrites sessions as event sourcing: the session is an append-only event log; history is a projection (replay) of the log.

#![allow(unused)]
fn main() {
use langchainrust::sessions::{EventSessionManager, MemoryEventStore, AutoCompaction};
use std::sync::Arc;

let manager = EventSessionManager::new(Arc::new(MemoryEventStore::new()))
    .with_max_context_turns(10)                      // turn window (default: full history)
    .with_auto_compaction(AutoCompaction::new(20)?); // snapshot beyond 20 turns

let id = manager.create_session().await?;            // uuid v7

let reply = manager.chat(&id, &llm, "What is Rust?".to_string()).await?;

let history = manager.history(&id).await?;   // projected Vec<Message>
// clear / archive / delete: append Metadata events to the EventStore directly
}

Old API → New API Mapping

Old (0.21.x)New (0.22.0)Note
SessionManager::new(Arc<dyn SessionStore>)EventSessionManager::new(Arc<dyn EventStore>)storage becomes an append-only log
create_session_for(user)create_session() (uuid v7)ownership via Metadata events
chat(&id, &llm, msg)same signaturepersistence changes from rewrite to append — crash-safe
history(&id)history(&id) / replay_session(&id)replay_session yields the old mutable Session view
—fork_session(&id, branch, until)new: branch from any point in the log
max_context_messages(n)with_max_context_turns(n)turn window; user/ai always paired
—with_auto_compaction(AutoCompaction)new: deterministic snapshot past N turns (no LLM call)
clear / archive / delete_sessionappend Metadata events to the EventStore directly (convenience wrappers planned for 0.23.0 had not landed as of 0.24.0)the log is immutable; cleanup is a state event
SessionStoreEventStoreappend / append_batch / read / fork; idempotency key (session, branch, id)

Fork

#![allow(unused)]
fn main() {
// Copy the prefix up to some event sequence number into a new branch;
// the trunk is unaffected. Returns a manager pinned to the branch.
let mut experiment = manager.fork_session(&id, "experiment", None).await?;
experiment.chat(&id, &llm, "branch message".to_string()).await?;
// Re-forking the same (session, branch) is idempotent
}

Crash Safety

  • append is idempotent per (session_id, branch, id) — replaying a half-written batch after a crash never duplicates events.
  • project() / to_session() rebuild state from the log; orphan tool results are detected and rejected, so the history fed to the LLM is always properly paired.
  • SessionCheckpoint trait + NoopCheckpoint are the placeholder for durable checkpoints: durable projections were planned for 0.23.0, but as of 0.24.0 the no-op implementation is still the only one — recovery replays the full event log.

Event Types

EventPayload variantMeaning
SessionCreatedsession established (uuid v7)
UserMessage / AssistantMessageone Turn per round
ToolCall / ToolResultappended as a pair
Snapshotdeterministic compaction snapshot
Metadatalifecycle / ownership key-value state

A2A Protocol

The A2A (Agent-to-Agent) protocol is Google’s open standard for agent interoperability — letting agents built by different teams and vendors discover and call each other. LangChainRust provides both a server and client implementation over JSON-RPC 2.0.

Layering: A2A handles “Agent ↔ Agent” communication; MCP handles “Agent ↔ tools / data sources.” They compose: MCP supplies tools, agents collaborate over A2A.

Feature Overview

FeatureTypeDescription
A2AServerServerExpose your agent as an A2A service (handler functions, pluggable into any HTTP framework)
A2AClientClientDiscover and call remote agents (send_task / get_task / cancel_task)
AgentCardProtocolAgent self-description card; v1.0.1 supportedInterfaces[]
AgentInterface / A2ATransportProtocolPer-interface (protocolVersion, transport, url) + optional tenant tag
negotiate / is_v101ProtocolClient/server transport + version negotiation
sign_agent_card / verify_card_signatureSecurityCard signing (RFC 8785-lite canonicalization, HMAC) stored in card.signature
sign_card_jws / verify_card_jwsSecurityCompact JWS HS256 card signing (header-friendly)
AgentRegistry / RegistryClientDiscoveryRegistry-based multi-agent discovery
FederationGatewayGatewayCross-org federation with call policies and data contracts
ResilientA2AClientClientRetries / timeouts / circuit breaking wrapper
SkillRouter / SkillMapRouterClientRoute tasks to agents by skill

Agent Card (v1.0.1)

A v1.0.1 card declares multiple interfaces instead of a single protocol binding:

#![allow(unused)]
fn main() {
use langchainrust::a2a::protocol::{AgentCard, AgentInterface, A2ATransport, A2A_VERSION_V101};

let card = AgentCard::new("agent-a", "A helpful agent", "https://a.example")
    .with_supported_interface(AgentInterface::new(
        A2A_VERSION_V101,           // "1.0.1", a separate protocolVersion per interface
        A2ATransport::HttpJson,     // JsonRpc | HttpJson | Grpc
        "https://a.example/a2a",
    ))
    .with_supported_interface(
        AgentInterface::new("1.0.1", A2ATransport::JsonRpc, "https://a.example/a2a/jsonrpc")
            .with_tenant("acme"),   // enterprise multi-tenancy (optional)
    );

assert!(card.is_v101());
}

The old v0.3 fields are kept for reader compatibility; new cards should fill supportedInterfaces and treat the old fields as fallback.

Negotiation

The client picks a mutually supported interface given the transport it wants and the versions it speaks; a miss is a clear error:

#![allow(unused)]
fn main() {
let picked = card.negotiate(A2ATransport::HttpJson, &["1.0.1", "1.0"])?;
}

Card Signing

The Agent Card drives discovery — a tampered card (swapped endpoint, downgraded protocol) is a supply-chain attack. Signing canonicalizes the card JSON (RFC 8785-lite: recursive key ordering) and applies HMAC-SHA256.

#![allow(unused)]
fn main() {
use langchainrust::a2a::client::{sign_agent_card, verify_card_signature, sign_card_jws, verify_card_jws};
use langchainrust::a2a::protocol::AgentCard;

let secret = b"shared-secret-between-registrar-and-clients";

// In-place signing, hex-encoded into card.signature
let mut card = AgentCard::new("agent-a", "A helpful agent", "https://a.example");
sign_agent_card(&mut card, secret)?;

// Verify — tampering with any field fails
verify_card_signature(&card, secret)?;

// Compact JWS token (header-friendly)
let jws = sign_card_jws(&card, secret)?;
verify_card_jws(&card, &jws, secret)?;
}

Boundaries: HS256 symmetric only (ES256 planned 0.22.1); canonicalization is RFC 8785-lite, not full JCS; expiry rides the card’s expiresAt field.

A2A Server (Expose Your Agent)

A2AServer provides handler functions you plug into any HTTP framework (axum, actix, warp) — it does NOT start its own listener.

#![allow(unused)]
fn main() {
use langchainrust::a2a::A2AServer;
use langchainrust::LLMChain;
use std::sync::Arc;

let chain = Arc::new(LLMChain::new(llm, "You are a helpful assistant"));
let server = A2AServer::new(chain)
    .with_card(card);

// In your HTTP handlers:
// GET  /.well-known/agent-card.json → server.get_agent_card()
// POST /                       → server.handle_a2a_request(body).await
}

Endpoints: GET /.well-known/agent-card.json (discovery) and POST / (JSON-RPC requests). Tasks from tasks/send are stored in an in-memory RwLock<HashMap> — wrap with a database-backed TaskStore for production.

A2A Client (Call Remote Agent)

#![allow(unused)]
fn main() {
use langchainrust::a2a::{A2AClient, A2AMessage};

let client = A2AClient::new("http://remote-agent:8080".to_string()).unwrap();

// Discover
let card = client.get_agent_card().await?;

// Dispatch / poll / cancel
let task = client.send_task(A2AMessage::user("hello")).await?;
let task = client.get_task(&task.id).await?;
let task = client.cancel_task(&task.id).await?;
}

Protocol Flow

StepOperationRole
1. Discoverget_agent_card fetches the remote Agent CardClient
2. Dispatchsend_task submits a task (A2AMessage)Client → Server
3. Progressget_task queries status and result by task IDClient
4. Cancelcancel_task cancels an unfinished taskClient

Status Matrix

CapabilityStatus
tasks/send / tasks/get / tasks/cancelImplemented
Agent Card discoveryImplemented
v1.0.1 supportedInterfaces + negotiationImplemented (0.22.0)
Card signing JWS HS256Implemented (0.22.0; ES256 / full JCS → 0.22.1)
API key / Static Bearer authImplemented
gRPC binding→ 0.22.1 (tonic codegen needs protoc)
OAuth / OIDCPlanned (0.22.1)
Task state machine / streaming push / TLSPlanned

A deployment walkthrough lives at docs/internal/deployment/a2a-http-server-deployment.md; the runnable example is crates/lc/examples/a2a_http_server.rs.

Migration: v0.7 to v0.8

Version 0.8.0 introduced a major workspace restructure, migrating from a single-crate architecture to a multi-crate workspace. This guide covers what changed and how to migrate.

What Changed

Before (v0.7): Single Crate

All code lived in src/ of a single langchainrust crate.

After (v0.8): Workspace Architecture

The project is now organized as a Cargo workspace with a facade crate:

langchainrust/
├── crates/
│   ├── lc/               # Facade crate (package: langchainrust)
│   ├── lc-core/           # Runnable, BaseTool, output parsers, structured output
│   ├── lc-schema/         # Message types
│   ├── lc-shared/         # Shared utilities, Document type
│   ├── lc-providers/      # OpenAI, Ollama, DeepSeek, Anthropic, Gemini, etc.
│   ├── lc-agents/         # ReAct, FunctionCalling, PlanExecute, CRAG, etc.
│   ├── lc-chains/         # LLMChain, SequentialChain, RouterChain, etc.
│   ├── lc-memory/         # Buffer, Window, Summary, Persistent
│   ├── lc-embeddings/     # OpenAI, DeepSeek, Qwen, Local, Mock
│   ├── lc-vector-stores/  # InMemory, Qdrant, ChromaDB, Pinecone, etc.
│   ├── lc-rag/            # RAGPipeline, BM25, Hybrid, HyDE, GraphRAG
│   ├── lc-prompts/        # PromptTemplate, ChatPromptTemplate
│   ├── lc-callbacks/      # StdOut, LangSmith, File, OTel
│   ├── lc-tools/          # Calculator, DateTime, PythonREPL, etc.
│   ├── lc-tools-derive/   # #[tool] procedural macro
│   ├── lc-evaluation/     # 10 evaluators
│   ├── lc-guardrails/     # Input/output safety
│   ├── lc-sessions/       # Session management
│   ├── lc-langgraph/      # StateGraph, CompiledGraph
│   ├── lc-mcp/            # MCP client/server
│   └── lc-a2a/            # Agent-to-Agent protocol

Import Compatibility

The facade crate (lc) re-exports all public types under langchainrust::, so most import paths are unchanged:

#![allow(unused)]
fn main() {
// These still work exactly as before:
use langchainrust::OpenAIChat;
use langchainrust::agents::ReActAgent;
use langchainrust::memory::ConversationBufferMemory;
use langchainrust::chains::LLMChain;
}

Breaking Changes

Feature Flags

Some features moved or were renamed. Update your Cargo.toml:

Old (v0.7)New (v0.8)
features = ["qdrant"]features = ["qdrant-integration"]
features = ["mongo"]features = ["mongodb-persistence"]
features = ["redis"]features = ["redis-storage"]
features = ["sqlite"]features = ["sqlite-storage"]

Error Types

Error types are now defined in their respective sub-crates. The facade re-exports them, so most code compiles unchanged. If you referenced internal error paths directly, update to the re-exported names.

Internal Module Paths

If you used langchainrust::internal_module::... paths, they may have moved. Prefer the top-level re-exports:

#![allow(unused)]
fn main() {
// PREFER:
use langchainrust::{RAGPipeline, BM25Retriever, GraphRAG};

// AVOID (internal path may change):
use langchainrust::retrieval::pipeline::RAGPipeline;
}

New in v0.8+

  • LCEL (v0.9): pipe(), RunnableSequence, RunnableLambda, RunnableParallel, RunnableBranch, RunnableBinding
  • #[tool] macro (v0.10): Auto-generate tool implementations from functions
  • Fallbacks (v0.10): RunnableWithFallbacks for resilient pipelines
  • Assign (v0.10): RunnableAssign for injecting context into HashMap pipelines
  • Document chain streaming (v0.10): Stuff, Refine, MapReduce chains support streaming

Dependency Update

# Before (v0.7)
[dependencies]
langchainrust = "0.7"

# After (v0.8+)
[dependencies]
langchainrust = "0.11"  # Facade crate, all features available via feature flags