liteLLM

One library to standardize all LLM APIs

★★★★★ 5.0/5 from 17 reviews

Unified API AI Infrastructure Tools

What is liteLLM?

Simplify using OpenAI, Azure, Cohere, Anthropic, Replicate, Google LLM APIs. TLDR Call all LLM APIs using the chatGPT format - completion(model, messages) Consistent outputs and exceptions for all LLM APIs Logging and Error Tracking for all models

Pros & Cons

Pros

Support for custom models
Caching and load balancing
Easy provider switching
OpenAI-compatible API
Performance analysis integration
Standardized API
Unified interface for LLMs
Versatile API calling

Tool Details

Categories	Unified API, AI Infrastructure Tools
Website	github.com
Became Popular	August 9, 2023
Platforms	Web
Social	Twitter · GitHub

Recent Reviews (4)

Miguel Feb 6, 2025

★★★★★

liteLLM is a must for working with different models. We use different models for different tasks and subtasks. With liteLLM the code stays exactly the same and we can just focus on choosing the right prompts and models for the task.

Ahmed Oct 2, 2025

★★★★★

Big fan of liteLLM: one API for OpenAI/Anthropic/Groq/etc. Makes multi-model stacks painless

Ali May 6, 2024

★★★★★

I find myself recommending this library to serious LLM-powered app developers that are trying to standardize their codebase by unifying all the APIs they use. Love it!

Standardized API

Timoa Nov 8, 2024

★★★★★

Used as an LLM proxy, it allows the caching and load balancing between multiple AI services (Groq, OpenRouter, etc.) and even local with Ollama. It uses an OpenAI-compatible API that allows (when we can set the base URL) to use it in many apps or services. I use it configured with Langfuse which provides the performance analysis (monitoring) of each prompt/session.

Caching and load balancing OpenAI-compatible API Performance analysis integration

Frequently Asked Questions about liteLLM

When did liteLLM become popular?

liteLLM became popular around August 9, 2023.

What are the main advantages of using liteLLM?

The top advantages of liteLLM include: support for custom models, caching and load balancing, easy provider switching, OpenAI-compatible API, performance analysis integration.

What is liteLLM's overall user rating?

liteLLM has an overall rating of 5.0/5 based on 17 user reviews.

What type of tool is liteLLM?

liteLLM belongs to the following categories: Unified API, AI Infrastructure Tools.