Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

LLM Providers

LangChainRust supports 10+ LLM providers through a unified BaseChatModel trait.

Supported Providers

ProviderConfigAuth
OpenAIOpenAIChatOPENAI_API_KEY
AnthropicAnthropicChatANTHROPIC_API_KEY
GeminiGeminiChatGEMINI_API_KEY
OllamaOllamaChatLocal (no key)
DeepSeekDeepSeekChatDEEPSEEK_API_KEY
QwenQwenChatQWEN_API_KEY
MoonshotMoonshotChatMOONSHOT_API_KEY
ZhipuZhipuChatZHIPU_API_KEY
MistralMistralChatMISTRAL_API_KEY
Azure OpenAIAzureOpenAIChatAZURE_OPENAI_API_KEY
CohereCohereChatCOHERE_API_KEY

Auto-Detection

LLMClient::from_env() auto-detects the provider from environment variables:

#![allow(unused)]
fn main() {
use langchainrust::LLMClient;

let llm = LLMClient::from_env().expect("Set an API key");
}

Priority: OpenAI → Anthropic → Mistral → Ollama (local fallback).

OpenAI-Compatible Wrappers

DeepSeek, Qwen, Moonshot, Zhipu, and Mistral all wrap OpenAIChat internally via into_openai_config() — they use the same chat format but with different base URLs and default models.

Native Providers

Azure OpenAI and Cohere have native implementations because their URL/auth format differs from OpenAI’s.

Streaming

All providers support streaming via stream_chat():

#![allow(unused)]
fn main() {
let stream = llm.stream_chat(messages, None).await?;
while let Some(token) = stream.next().await {
    print!("{}", token?.content);
}
}

Multimodal

OpenAI supports Whisper (STT), TTS, and DALL-E via the MultimodalModel trait:

#![allow(unused)]
fn main() {
use langchainrust::MultimodalModel;

let transcript = llm.transcribe(audio).await?;
let speech = llm.generate_speech("Hello!").await?;
let image = llm.generate_image("A cat wearing a hat").await?;
}

Retry & Fallback

#![allow(unused)]
fn main() {
use langchainrust::runnables::{RetryConfig, RunnableExt};

let llm_with_retry = llm.with_retry(RetryConfig::new(3));
}

See also