Best Self-Hosted LLMs in 2026

Self-hosting an LLM used to mean accepting significantly worse performance than the proprietary APIs. In 2026, that's changed. The best self-hosted models now approach proprietary APIs much more closely on coding benchmarks, but they require real hardware to run them. The question isn't just capability anymore. It's whether the economics make sense for your team, and which model fits your GPU setup.

This guide covers the top self-hosted LLMs from the Onyx Self-Hosted LLM Leaderboard, organized by the hardware tier they require. Data is updated as of October 13, 2026.

How this guide is sourced: Hardware, benchmark, and license data comes from the Onyx Self-Hosted LLM Leaderboard. The recommendations in each section interpret that data for specific deployment tiers.

---

TL;DR: The right self-hosted model depends almost entirely on what GPU you have. Kimi K3 is the strongest self-hosted model in this snapshot, but it is a 24x H100-class deployment. GLM-5.3, MiMo-V2.6-Pro, DeepSeek-V4-Pro, Kimi K2.6, and GLM-5.2 sit in the 16x H100 tier. DeepSeek-V4.1-Flash is the best 8x H100 pick, with 90.6% Terminal-Bench 2.1 and 74.2% DeepSWE. GLM-5.3-Flash and DeepSeek-V4-Flash fit the 4x H100 tier. Single-H100 and consumer-GPU options remain useful, but the top rankings are now clustered around large MoE models.

---

What Is a Self-Hosted LLM?

A self-hosted LLM is a model you run on hardware you control rather than calling via an API. Your data stays on your infrastructure, you don't pay per token, and you can customize the model for your specific use case.

Reasons teams choose to self-host:

• Data privacy: Sensitive documents, code, and queries never leave your infrastructure
• Cost at scale: At high query volumes, self-hosting is cheaper than API pricing
• Air-gapped environments: Defense, healthcare, and regulated industries requiring offline deployment
• Model customization: Fine-tuning and LoRA adapters for domain-specific tasks
• Latency control: Predictable inference performance without rate limits

---

---

What Is Onyx?

Onyx is an open-source AI platform that connects self-hosted LLMs to company knowledge, permissions, and workplace workflows. The model runtime handles inference, while Onyx provides connectors, search, chat, agents, deep research, citations, and a governed user interface.

For self-hosted model deployments, Onyx is the layer that makes the model useful to a team instead of only to the infrastructure group. It can connect a local model endpoint to Slack, Confluence, Jira, Google Drive, SharePoint, GitHub, and other sources while preserving source-system access controls.

Hardware Tiers

---

Self-Hosted Models at a Glance

24x H100 (1,920GB): Kimi K3

16x H100 (1,280GB): GLM-5.3, MiMo-V2.6-Pro, DeepSeek-V4-Pro, Kimi K2.6, GLM-5.2

8x H100 (640GB): DeepSeek-V4.1-Flash

4x H100 (320GB): GLM-5.3-Flash, DeepSeek-V4-Flash

Dual GPU (2x H100): Step-3.7-Flash

Single H100: Nemotron 3 Super

RTX 4090: Qwen3.6-27B, Qwen3.6-35B-A3B, Gemma 4 31B

RTX 3090: Gemma 4 12B

---

Best Self-Hosted LLMs by Hardware Tier

Ultra Cluster Tier (24x H100 80GB)

Kimi K3 is the highest-ranked self-hosted model in the October refresh. It is also the most demanding deployment in this guide, with 2.8T total parameters, 104B active parameters, and a 24x H100 80GB minimum.

Kimi K3: Open-weight model with a 1M-token context window, 93.5% GPQA Diamond, 90.4% BrowseComp, and 1,486 Chatbot Arena Elo. Choose it only if you have multi-node infrastructure ready for a 1,400GB INT4 working set.

---

Multi-Node Cluster Tier (16x H100 80GB)

These models are still enterprise-cluster deployments. GLM-5.3 and MiMo-V2.6-Pro are the newest additions, while DeepSeek-V4-Pro, Kimi K2.6, and GLM-5.2 remain strong older choices.

GLM-5.3: 753B total / 40B active parameters. It is the strongest 16x H100 model in the refreshed self-hosted tierlist, with 88.2% Terminal-Bench 2.1, 66.9% DeepSWE, and 62.5% HLE.

MiMo-V2.6-Pro: MIT-licensed, 1.02T total / 42B active parameters. It is a coding-agent-focused upgrade over MiMo-V2.5-Pro, with 72.6% DeepSWE and a 46 Artificial Analysis Intelligence Index score.

DeepSeek-V4-Pro: MIT-licensed, 1.6T total / 49B active parameters. Leads this tier on SWE-bench Verified at 80.6% and LiveCodeBench at 93.5%, with 3,200GB VRAM at FP16 (800GB at INT4) and 64.1 tokens/sec throughput. The top pick when raw coding capability matters more than hardware footprint.

Kimi K2.6: Modified MIT license, 1T total / 32B active parameters. Close behind on SWE-bench Verified at 80.2% and ahead on GPQA Diamond at 90.5%, with strong agentic scores (73.1% OSWorld, 83.2% BrowseComp). Choose it for agentic and tool-use workloads at this tier.

GLM-5.2: MIT-licensed, 753B total / 40B active parameters. Leads on GPQA Diamond at 91.2% and runs at 193.3 tokens/sec, the fastest of the three. Choose it when reasoning benchmarks and throughput matter more than published SWE-bench numbers.

---

8x H100 Cluster Tier (640GB)

DeepSeek-V4.1-Flash is the best current middle ground between full cluster models and the older 4x H100 flash tier.

DeepSeek-V4.1-Flash: MIT-licensed, 552B total parameters with 8B to 16B active parameters. It needs 1,104GB at FP16 or 276GB at INT4 and is listed for an 8x H100 80GB minimum. Pick it when agentic coding performance matters and a 16x H100 deployment is too large.

---

4x H100 Cluster Tier (320GB)

The 4x H100 tier now has two strong flash models. GLM-5.3-Flash is newer and leads this tier on Terminal-Bench 2.1, while DeepSeek-V4-Flash still has broader classic coding scores.

DeepSeek-V4-Flash: MIT-licensed, 284B total / 13B active parameters. Scores 79% on SWE-bench Verified and 91.6% on LiveCodeBench, close to the 16x H100 tier at a quarter of the VRAM (568GB at FP16, 142GB at INT4). Runs at 120.4 tokens/sec, roughly double DeepSeek-V4-Pro's throughput. The practical choice for teams that want DeepSeek-V4 quality without a 16-GPU cluster.

GLM-5.3-Flash: MIT-licensed, 321B total / 18B active parameters. Its 642GB FP16 and 161GB INT4 requirements put it in the 4x H100 tier, and its 84.3% Terminal-Bench 2.1 score makes it the strongest newer flash model for agentic terminal tasks.

---

Dual H100 Tier (2x H100 80GB)

Step-3.7-Flash: Apache 2.0, 198B total / 11B active parameters. Scores 76.5% on SWE-bench Verified and runs at 393.8 tokens/sec, the fastest model in this guide by a wide margin. Fits on 2x H100 80GB at 396GB VRAM (FP16) or 99GB at INT4. The pick when inference speed matters as much as capability.

---

Single H100 Tier (1x H100 80GB)

Single H100 deployments are the most common enterprise self-hosting configuration. At $15K-$25K for a used/lease H100, this tier represents strong cost-performance for teams running inference workloads.

Nemotron 3 Super: NVIDIA Open Model license, 120B total / 12B active parameters via MoE. Scores 60.47% on SWE-bench Verified and 90.21% on AIME 2025, with 240GB VRAM at FP16 (60GB at INT4) fitting comfortably on a single H100 80GB. Runs at 191.6 tokens/sec at $0.25 / $0.775 per 1M input/output tokens if served through NVIDIA's hosted endpoints, or free to self-host on your own hardware.

---

RTX 4090 Tier (24GB VRAM)

The RTX 4090 has become the benchmark consumer GPU for LLM enthusiasts and small teams. At 24GB VRAM, it runs 27B-35B dense and MoE models in INT4 quantization comfortably.

Qwen3.6-27B: Apache 2.0, 27B dense parameters, runs on a single RTX 4090 at 54GB VRAM (FP16) or 14GB at INT4. Scores 77.2% on SWE-bench Verified and 83.9% on LiveCodeBench. The default pick for RTX 4090 owners who want the strongest all-round dense model at this size.

Qwen3.6-35B-A3B: Apache 2.0, 35B total / 3B active parameters via MoE. Scores 73.4% on SWE-bench Verified while running at 153.2 tokens/sec, close to triple Qwen3.6-27B's throughput, at only 70GB VRAM (FP16) or 18GB (INT4). Choose it when inference speed matters more than the last few points of benchmark score.

Gemma 4 31B: Apache 2.0 licensed, 31B dense parameters, 1451 Chatbot Arena Elo and 88.4% MMMLU. Free to run at $0 per token, with 62GB VRAM at FP16 (16GB at INT4). The pick for teams that want a fully open, no-cost model with strong general reasoning.

---

Budget Tier (RTX 3090)

The most accessible tier for individual developers and small teams. Gemma 4 12B is the best-in-class pick here, offering strong instruction-following on consumer hardware that costs $400-$700.

Gemma 4 12B: Apache 2.0 licensed, 12B dense parameters, 97.2% IFEval and 83.4% MMMLU, at only 24GB VRAM (FP16) or 6GB at INT4. Runs at 108.8 tokens/sec and costs $0.1 / $0.3 per 1M input/output tokens on hosted endpoints, or free to self-host. The right pick when hardware is the hard constraint but instruction-following quality still matters.

---

Choosing the Right Self-Hosted Model

---

Connecting Self-Hosted LLMs to Your Company's Data

Self-hosting solves the inference problem, not the retrieval and permissions problem. Most teams still need a way to connect the model to internal knowledge sources and keep access controls intact.

Onyx is relevant here as the application layer around a self-hosted model. You can point Onyx at a vLLM or Ollama deployment, connect sources like Slack, Confluence, Jira, and GitHub, and keep permission-aware search and chat in front of users. That makes the hardware choices in this guide more actionable: the model handles inference, while Onyx handles retrieval, orchestration, and team-facing access.

---

Recommended Self-Hosted LLM Stack

The model runtime should not be the user experience. Put a governed layer like Onyx in front of the model so employees can search internal knowledge, see citations, and use the same permissions they already have in source systems.

Frequently Asked Questions

What is the best self-hosted LLM in 2026?

In this leaderboard snapshot, the best self-hosted LLM depends mostly on hardware tier. Kimi K3 is the strongest overall but needs a 24x H100 cluster. GLM-5.3, MiMo-V2.6-Pro, DeepSeek-V4-Pro, Kimi K2.6, and GLM-5.2 are the top 16x H100 options. DeepSeek-V4.1-Flash is the best 8x H100 model. GLM-5.3-Flash and DeepSeek-V4-Flash are the practical 4x H100 choices. On an RTX 4090, Qwen3.6-27B is still the general-purpose pick, and Qwen3.6-35B-A3B trades some score for MoE throughput.

Can I run a frontier-class LLM on a single GPU?

Yes, but not the top tier. Nemotron 3 Super achieves 60.47% SWE-bench Verified and 90.21% AIME 2025 on a single H100 80GB. On an RTX 4090, Qwen3.6-27B delivers 77.2% SWE-bench Verified and 83.9% LiveCodeBench, capability that would have required a multi-GPU cluster two years ago. The best 2026 self-hosted models now need 8x, 16x, or 24x H100 clusters.

What tools do I use to run these models locally?

The most common inference runtimes are Ollama (easiest setup for consumer hardware), vLLM (best throughput for production deployments), LM Studio (GUI-based for Mac/Windows), and llama.cpp (CPU and low-VRAM support). Most models in this guide are available through Ollama or HuggingFace directly.

Which self-hosted LLMs support air-gapped deployment?

All models in this guide can run in fully air-gapped environments after the initial model weight download, subject to each model's license. MIT and Apache 2.0 models have the cleanest commercial terms. Kimi K3 and GLM-5.3 use model-specific licenses, so legal review matters before regulated deployment.

How do I estimate self-hosting cost versus API costs?

A single H100 80GB server node (cloud or on-premise) runs approximately $2-4/hour on cloud or $15K-$25K to purchase. Nemotron 3 Super running on that node can serve substantial daily token volume at 191.6 tokens/sec. At DeepSeek-V4-Flash's $0.14/M input price for hosted access, high-volume teams typically break even on self-hosting within a few months of steady utilization. For teams with consistent 24/7 inference demand, self-hosting is typically cheaper over a 2-year horizon than paying per-token. See Best Open Source LLMs 2026 for a full comparison of open-source model licenses, or Best LLMs for Coding 2026 for coding-focused rankings including API options.

What is the best model for my specific hardware?

It depends on your GPU's VRAM and whether you are using FP16 or INT4 quantization. The Onyx LLM Hardware Requirements Calculator lets you enter your hardware and instantly see which models fit, including VRAM requirements and recommended quantization settings for each model in this guide.