LiteLLM, developed by the Y Combinator-backed company BerriAI, is an open-source project that has become one of the most popular abstraction layers for working with large language models. The fundamental problem it solves is fragmentation: every model provider — OpenAI, Anthropic, Google, Azure, AWS Bedrock, Cohere, Hugging Face, and dozens more — has its own API shape, authentication, and quirks. LiteLLM normalizes all of them behind a single OpenAI-compatible interface, so application code can target one consistent format and switch or combine providers freely.
LiteLLM ships in two main forms. The Python SDK lets developers call any supported model with the same function signature, making it trivial to add fallbacks, retries, and provider switching directly in code. The Proxy Server (also called the AI Gateway) is a self-hostable service that sits in front of all model traffic and adds the controls enterprises need: centralized API key management, per-team and per-key budgets and rate limits, cost tracking, load balancing across deployments, response caching, guardrails, and structured logging. This makes it a practical control plane for organizations that want governance and observability over their AI usage without rewriting applications.
Because it is open source and supports well over 100 providers, LiteLLM has been widely adopted as default infrastructure, and many other AI tools integrate with or build on top of it. Its OpenAI-compatible design also means existing OpenAI-oriented tooling and libraries work with minimal changes, lowering migration friction.
BerriAI went through Y Combinator (W23) and raised seed funding of roughly $1.6M co-led by Y Combinator with participation from Gravity Fund, Pioneer Fund, and others. The project has grown a large open-source community and reported meaningful annual recurring revenue from its enterprise proxy offering. LiteLLM targets developers and platform teams who want a vendor-neutral, governable interface to the entire LLM provider landscape.