LiteLLM Setup

Route Claudin through a LiteLLM proxy for one unified API to 100+ model providers.

Overview

LiteLLM is an open-source LLM gateway that provides a unified API to 100+ model providers. Running the LiteLLM Proxy in front of Claudin lets you route requests through LiteLLM to any of its supported providers, all through Claudin's existing OpenAI-compatible custom preset — no code changes.

Prerequisites: LiteLLM installed (pip install litellm[proxy]), a litellm_config.yaml, and the proxy running on a local or remote port.

1. Start the LiteLLM proxy

pip install litellm[proxy]

Create a litellm_config.yaml with your model aliases:

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY

  - model_name: claude-sonnet-4
    litellm_params:
      model: anthropic/claude-sonnet-4-5-20250929
      api_key: os.environ/ANTHROPIC_API_KEY

  - model_name: gemini-2.5-flash
    litellm_params:
      model: gemini/gemini-2.5-flash
      api_key: os.environ/GEMINI_API_KEY

  - model_name: llama-3.3-70b
    litellm_params:
      model: together_ai/meta-llama/Llama-3.3-70B-Instruct-Turbo
      api_key: os.environ/TOGETHER_API_KEY

Run the proxy — it starts at http://localhost:4000 by default:

litellm --config litellm_config.yaml --port 4000

2. Point Claudin at LiteLLM

No environment variables to set — everything goes through /provider:

Run claudin, then /provider.
Pick Add profile → preset custom (or openai with an overridden base URL).
Set Base URL to http://localhost:4000 (or wherever your proxy runs).
Set API key to your LiteLLM master key, or any placeholder if your local proxy doesn't enforce auth.
Set Model to the alias from your config — gpt-4o, claude-sonnet-4, gemini-2.5-flash, etc.
Save and activate the profile.

To switch upstream providers later, edit the profile and change Model to a different LiteLLM alias — no restart beyond the next request.

3. Example configs

Multi-provider routing with spend tracking:

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY

  - model_name: claude-sonnet-4
    litellm_params:
      model: anthropic/claude-sonnet-4-5-20250929
      api_key: os.environ/ANTHROPIC_API_KEY

  - model_name: deepseek-chat
    litellm_params:
      model: deepseek/deepseek-chat
      api_key: os.environ/DEEPSEEK_API_KEY

litellm_settings:
  set_verbose: false
  num_retries: 3

With a master key for auth:

litellm --config litellm_config.yaml --port 4000 --master_key sk-my-master-key

Then in /provider, set the profile's API key to sk-my-master-key and Base URL to http://localhost:4000.

4. Notes

The Claudin profile's Model must match a model_name from your LiteLLM config, not the upstream raw provider model name.
If your proxy requires authentication, use the proxy key (or master_key) as the profile's API key.
LiteLLM's OpenAI-compatible endpoint accepts the same request format as OpenAI, so Claudin works without any code changes.
Switch upstream providers by editing the profile's Model field — nothing else needs to change.

5. Troubleshooting

Issue	Likely cause	Fix
404 or Model Not Found	Model alias doesn't exist in the LiteLLM config	Verify the `model_name` in `litellm_config.yaml` matches the profile's Model
Connection refused	LiteLLM proxy isn't running	Start it with `litellm --config litellm_config.yaml --port 4000`
Auth failed	Wrong or missing master key on the profile	Run `/provider`, edit the profile, paste the correct key
Upstream provider error	The backend provider key is missing or invalid	Ensure the upstream API key (e.g. `OPENAI_API_KEY`) is set in your LiteLLM proxy process environment
Tools fail but chat works	The selected model has weak function/tool-calling support	Switch to a model with strong tool support (e.g. GPT-4o, Claude Sonnet)

You can also run /provider doctor inside Claudin to check the active profile.