Python SDK and proxy gateway that calls 100+ LLM APIs in OpenAI format with cost tracking.
Python SDK and proxy gateway that calls 100+ LLM APIs in OpenAI format with cost tracking.
🚅 LiteLLM
Open Source AI Gateway for 100+ LLMs. Self-hosted. Enterprise-ready. Call any LLM in OpenAI format.
LiteLLM Proxy Server (AI Gateway) | Hosted Proxy | Enterprise Tier | Website
LiteLLM is an open source AI Gateway that gives you a single, unified interface to call 100+ LLM providers — OpenAI, Anthropic, Gemini, Bedrock, Azure, and more — using the OpenAI format.
Use it as a Python SDK for direct library integration, or deploy the AI Gateway (Proxy Server) as a centralized service for your team or organization.
Jump to LiteLLM Proxy (LLM Gateway) Docs
Jump to Supported LLM Providers
Managing LLM calls across providers gets complicated fast — different SDKs, auth patterns, request formats, and error types for every model. LiteLLM removes that friction:
• Unified API — one interface for 100+ LLMs, no provider-specific SDK juggling
• Drop-in OpenAI compatibility — swap providers without rewriting your code
• Production-ready gateway — virtual keys, spend tracking, guardrails, load balancing, and an admin dashboard out of the box
• 8ms P95 latency at 1k RPS (benchmarks)
All Supported Endpoints - /chat/completions, /responses, /embeddings, /images, /audio, /batches, /rerank, /a2a, /messages and more.
uv add litellm
An independent litellm-core distribution provides the Python SDK with the same
import litellm API and runtime dependencies. It has no optional extras, CLI entry
points, or bundled dashboard. Install one SDK distribution per environment because
litellm and litellm-core own overlapping Python files. Use litellm for the
proxy, CLI, and optional extras
Build and install core from this checkout while its release integration is pending:
python litellm/scripts/build_core_distribution.py --out-dir dist/core
python -m pip install dist/core/litellm_core-*.whl
The builder requires Git, uv and the Rust build toolchain. It reads the release version
from the root `pyproject.toml`, leaves source files unchanged, and produces a wheel
and a self-contained sdist in dist/core. Run installation in a fresh environment
without litellm
from litellm import completion
import os
os.environ["OPENAI_API_KEY"] = "your-openai-key"
os.environ["ANTHROPIC_API_KEY"] = "your-anthropic-key"
response = completion(model="openai/gpt-4o", messages=[{"role": "user", "content": "Hello!"}])
response = completion(model="anthropic/claude-sonnet-4-20250514", messages=[{"role": "user", "content": "Hello!"}])
Getting Started - E2E Tutorial - Setup virtual keys, make your first request
uv tool install 'litellm[proxy]'
litellm --model gpt-4o
import openai
client = openai.OpenAI(api_key="anything", base_url="http://0.0.0.0:4000")
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Hello!"}]
)
Auto Routing - Choose a model for each prompt (Python SDK + AI Gateway)
Docs: Autorouter SDK
Install with uv add litellm and set OPENAI_API_KEY for the models below.
import asyncio
from litellm import Router …
Builder tool · Builder tools / LLM gateway · Open Source · MIT core + enterprise directory · Python
Website https://docs.litellm.ai/docs/ · Repo https://github.com/BerriAI/litellm · Docs https://docs.litellm.ai/docs/
63,137 stars · 13,211 forks · 5,130 open issues · pushed 2026-11-12 · verified 2026-11-12
Tags: ai-agents, tool, llm-gateway
What is LiteLLM? LiteLLM is a coding website listed on Scriptiii Webnest. Python SDK and proxy gateway that calls 100+ LLM APIs in OpenAI format with cost tracking. Pricing: Free. Free to use. Best for: ai-agents, tool, llm-gateway.
Everything you need to know about LiteLLM
Similar websites in Coding
Open-source vector database that stores objects and vectors with hybrid search and filtering.
Coding·Free