Self-Hosted vs. Managed: LiteLLM Proxy Against OpenRouter

G'day! Welcome back. Last time we wired up LiteLLM on the Dell server as a self-hosted router for every model this lab talks to — Claude, GPT, Gemini, and our own fine-tuned Dolphin model on Hugging Face — all behind one OpenAI-compatible endpoint. A few of you asked the obvious follow-up: why not just use OpenRouter? It does the same multi-provider routing thing, hosted, zero maintenance, no Docker container to babysit.

Fair question. We actually run both, for different reasons, and today we're picking apart where each one wins. Let's crack into it.

The Basic Pitch, Side by Side

Both solve the same core annoyance: one API shape, many providers behind it. Where they differ is who runs the router and who your keys and traffic pass through:

• LiteLLM (self-hosted) — a proxy you run yourself, on your own hardware, using your own provider API keys. Every request leaves your network already addressed to Anthropic, OpenAI, or Google directly.

• OpenRouter (managed) — a hosted service you send requests to instead of the providers themselves. They hold relationships with the providers, you hold a relationship with them.

That one distinction — whose infrastructure sits in the middle — is where basically every other trade-off in this post comes from.

Where OpenRouter Wins

Zero infrastructure. There's no container to run, no YAML to maintain, no Dell laptop that needs to stay up for the router to work. Sign up, get a key, start calling models.

Model breadth without the busywork. OpenRouter aggregates providers we've never bothered wiring up ourselves — a long tail of open-weight models hosted by third parties, added the moment they land, with no config change on our end.

Built-in provider fallback and load balancing across hosts of the same open model — if three different providers all serve Llama or Mistral variants, OpenRouter can shop between them for price or latency automatically. Replicating that ourselves in LiteLLM means manually listing every one of those upstream hosts.

No egress cost or uptime burden on our end. If the Dell server's power flakes out, LiteLLM goes down with it. OpenRouter's uptime is somebody else's problem, at a much bigger scale than one home lab laptop.

Where LiteLLM Wins

Our keys, our relationship, our pricing. Every request through our LiteLLM box uses our own Anthropic, OpenAI, and Google keys, billed directly at each provider's own rate. OpenRouter marks up the same underlying calls for the convenience of aggregation — usually a modest percentage, but it's a permanent tax on every token, forever, versus a one-time setup cost for us.

Nothing routes through a third party. Requests go straight from the Dell server to Anthropic or Google — no intermediary sees the prompts, the responses, or the fact that we're calling them at all. For a lab that already treats self-hosting as the default — Nextcloud instead of Google Drive, Vaultwarden instead of a cloud password manager, Immich instead of Google Photos — routing every AI request through a hosted middleman undercuts the whole philosophy.

Our self-hosted model is a first-class citizen. The Dolphin3.0-Mistral-24B model running on our Hugging Face ZeroGPU Space sits in the exact same model_list as Claude and GPT, called the exact same way. OpenRouter has no idea our model exists — it can only route to providers it has a relationship with.

Fallback logic we actually control. Our claude-haiku-4-5 to claude-sonnet-5 to gemini-flash chain reflects our judgment about acceptable degradation, not a generic policy applied across OpenRouter's whole user base.

fallbacks:
- claude-haiku-4-5: ["claude-sonnet-5", "gemini-flash"]
- claude-sonnet-5: ["claude-opus-5", "gpt-5.5"]

No rate limits beyond what each provider gives our own account. OpenRouter pools demand across every customer sharing a model; under load, that can mean throttling that has nothing to do with our own usage.

The Privacy Angle, Properly Weighed

Worth being honest here: self-hosting LiteLLM doesn't make prompts disappear into a black box — Anthropic, OpenAI, and Google still see everything, same as they would through OpenRouter. What self-hosting removes is the extra hop: one fewer party in the chain with visibility into what we're asking and getting back. For anything genuinely sensitive, that's not nothing, and it lines up with the same instinct that put Frigate's camera feeds on our own Dell server instead of a cloud NVR.

So Which One Should You Run

Honestly? Both, for what each is actually good at — same answer as the Pi-hole versus Control D debate a while back. LiteLLM stays the default for anything hitting Claude, GPT, Gemini, or our own Dolphin model, because we already hold keys with all of them and don't want a markup or a middleman on traffic we can route ourselves. OpenRouter earns its keep for the long tail — trying a random new open-weight model for an afternoon without signing up for yet another provider account, or as an emergency fallback if every provider we've directly configured is somehow having a bad day at once.

The Verdict

If you already hold API keys with the big providers and have somewhere to run a Docker container, LiteLLM pays for its setup effort almost immediately — no markup, no middleman, and your own self-hosted models slot in as first-class citizens. If you want to poke at the ever-growing pile of open-weight models without opening ten provider accounts, OpenRouter is genuinely the easier front door. We're not choosing one — we're running LiteLLM as the default and keeping OpenRouter in the back pocket for everything it does that our own router can't.

Pick the router that matches what you're actually optimizing for — cost and control, or breadth and zero maintenance — and don't feel bad running both. Cheers!

"If you've got any questions or need a hand wrangling your own setup, don't hesitate to reach out at [email protected] or connect with me via www.yazan.me. I'm always keen to help out!"