LiteLLM: Your Gateway to 100+ LLM APIs

LiteLLM is a powerful Python SDK and Proxy Server designed to streamline interactions with over 100 Large Language Models (LLMs) using the OpenAI API format. It supports popular providers like Bedrock, Azure, OpenAI, VertexAI, Cohere, and many more, making it a versatile tool for developers and businesses alike.

Key Features

1. Unified API Access

LiteLLM allows you to call various LLM APIs through a single interface, simplifying the integration process. Whether you need text completion, embeddings, or image generation, LiteLLM has you covered.

2. Proxy Server Functionality

The LiteLLM Proxy Server acts as a gateway, managing requests and responses between your application and the LLM providers. It ensures consistent output formats and provides built-in retry and fallback mechanisms for enhanced reliability.

3. Budgeting and Rate Limiting

With LiteLLM, you can set budgets and rate limits per project, API key, or model, allowing for better cost management and resource allocation.

4. Support for Streaming Responses

LiteLLM supports streaming responses from models, enabling real-time interaction and faster processing of large datasets.

5. Logging and Observability

The SDK includes predefined callbacks for logging interactions with various observability tools like Lunary, Langfuse, and Slack, ensuring you can track usage and performance effectively.

Getting Started

To start using LiteLLM, follow these simple steps:

1. Installation: Install the SDK via pip:

   pip install litellm
   

2. Set Up Environment Variables: Configure your API keys for the providers you intend to use:

   import os
   os.environ["OPENAI_API_KEY"] = "your-openai-key"
   

3. Make Your First Call: Use the SDK to interact with an LLM:

   from litellm import completion
   messages = [{"content": "Hello, how are you?", "role": "user"}]
   response = completion(model="gpt-3.5-turbo", messages=messages)
   print(response)
   

Pricing

LiteLLM offers a flexible pricing model. For the latest pricing information, please check the official website.

Conclusion

LiteLLM is a game-changer for developers looking to harness the power of multiple LLMs without the hassle of managing different APIs. Its ease of use, combined with robust features, makes it an essential tool for any AI-driven project.

Try LiteLLM Today!

Ready to simplify your LLM interactions? Get started with LiteLLM now!