Local runtimes, inference servers, and LLM gateways.
GitHub stars among selected inference and serving alternatives.
Ranked alternatives
Popularity-ranked entries linked back to graph nodes and deterministic sources.
| Rank | Component | Kind | Openness | Popularity metric | Tasks | Source |
|---|---|---|---|---|---|---|
| 1 | Ollamasoftware:ollama | software | open source | 173,713GitHub stars as of 2026-06-10 | local-inference, model-serving | source |
| 2 | llama.cppsoftware:llama.cpp | software | open source | 115,795GitHub stars as of 2026-06-10 | local-inference, model-serving | source |
| 3 | vLLMsoftware:vllm | software | open source | 82,361GitHub stars as of 2026-06-10 | model-serving, inference | source |
| 4 | LiteLLMsoftware:litellm | software | open core | 49,818GitHub stars as of 2026-06-10 | model-routing, api-proxy | source |
| 5 | LocalAIsoftware:localai | software | open source | 46,758GitHub stars as of 2026-06-10 | local-inference | source |
| 6 | Raysoftware:ray | software | open source | 42,824GitHub stars as of 2026-06-10 | distributed-inference | source |
| 7 | SGLangsoftware:sglang | software | open source | 28,903GitHub stars as of 2026-06-10 | model-serving | source |
| 8 | Text Generation Inferencesoftware:text-generation-inference | software | open source | 10,860GitHub stars as of 2026-06-10 | model-serving | source |
| 9 | Triton Inference Serversoftware:triton-inference-server | software | open source | 10,742GitHub stars as of 2026-06-10 | model-serving | source |
| 10 | KServesoftware:kserve | software | open source | 5,561GitHub stars as of 2026-06-10 | model-serving | source |