Olla v0.1.0
The latest stable build of our open-source LLM proxy and load balancer.
TensorFoundry: Self-Hosted AI Software
and AI Infrastructure Consulting.
We build software for running LLMs on hardware you control, and help organisations plan, validate and tune the AI infrastructure they own.
New to running models in-house? Read our guide to deploying LLMs on your own infrastructure.
LATEST FROM THE FOUNDRY
What's new
Fresh releases, announcements and writing from the workshop.
Olla v0.1.0: Text Embeddings Inference, a Reworked Dashboard & Behaviour Changes
Olla v0.1.0 adds a Text Embeddings Inference backend, a reworked dashboard, retry controls and corrected request stats. Several behaviour changes affect existing setups, so check them before you upgrade.
MCP Goes Stateless: What 2026-07-28 Changes
MCP 2026-07-28 drops sessions, the initialize handshake and server-initiated requests. What changed, how to port a server across, and what we get now.
PRODUCT ECOSYSTEM
From user to backend - every layer
TensorFoundry covers the full LLM stack. Pick the layers you need.
FEATURED PRODUCTS
Start with what you need
From open-source tools to enterprise platforms
OUR JOURNEY
Engineering private AI infrastructure since 2018
From ML pipelines to self-hosted LLM software: the work behind our products and our advice.
Why Local AI?
Run models where your data already lives, on hardware you control.
Privacy First
Prompts, documents and outputs stay on your network. Nothing is sent to a third-party model provider unless you choose to.
Low Overhead
No round trip to a cloud region. Olla, our open-source proxy, adds about 0.30 ms at the median in our published benchmarks.
Predictable Cost
Steady workloads on hardware you own avoid per-token billing. Our cost comparison shows where self-hosting breaks even.
Optimised Performance
Works with the inference engines you already run, such as vLLM, SGLang, llama.cpp and Ollama, on NVIDIA, AMD and Apple Silicon.
OPEN SOURCE
Join the Olla Community
Open-source AI inference proxy perfect for small businesses and development teams. Unified interface for Ollama, LM Studio, vLLM and others with load balancing and failover.
86.4K+
GHCR container pulls
300+
GitHub Stars
Apache 2.0
Open Source
EARLY ACCESS
Get Early Access to the Full Stack
Join the waitlist for priority access to Alloy, FoundryOS, Kaizen, Pivotal and Forge as they launch from late 2026 into 2027.
Get exclusive updates, beta access and founding member pricing.
Priority Beta Access
Up to 50% Off Licensing
Direct Engineering Access
Now Olla, Alloy, FoundryOS, Kaizen + Pivotal
Late 2026 Alloy public release
Early 2027 FoundryOS, Kaizen, AgentOS + Pivotal 5
Planning, validating or tuning private AI?
Independent advice backed by reproducible evidence.