Friends Don't Let Friends Use Ollama

Ollama is the most popular way to run local LLMs. It shouldn’t be. It gained that position by being first, the first tool that made llama.cpp accessible to people who didn’t want to compile C++ or write their own server configs. That was a real contribution, briefly. But the project has since spent years systematically obscuring where its actual technology comes from, misleading users about what they’re running, and drifting from the local-first mission that earned it trust in the first place. All while taking venture capital money.
Ollama 是运行本地 LLMs 最流行的方式。它本不该如此。它之所以获得这一地位，是因为它抢占了先机，成为了第一个让那些不想编译 C++ 或编写自己服务器配置的人也能轻松上手工具。这曾是一项真正的贡献，但仅限于很短的一段时间。此后，该项目多年来一直在系统性地掩盖其核心技术的真实来源，误导用户关于他们正在运行的内容，并背离了最初赢得信任的“本地优先”使命。而这一切，都是在吸纳风险投资的同时进行的。 Ollama 是运行本地大型语言模型（LLM）最流行的方式。但其实不应该如此。它之所以能占据这一位置，是因为它是第一个让那些不想编译 C++ 或编写自己服务器配置的人也能使用相关技术的工具。这在短期内确实是一项贡献。但此后，该项目多年来一直有系统地掩盖其实际技术的来源，误导用户他们运行的是什么，并偏离了最初赢得用户信任的本地优先使命。而这一切都是在获取风险投资资金的情况下发生的。

This isn’t a “both sides” piece. I’ve used Ollama. I’ve moved on. Here’s why you should too.
这不是一篇“各执一词”的文章。我用过 Ollama，但我已经弃用了。以下是为什么你也应该弃用的原因。 这并非一篇“中立”的文章。我曾使用过 Ollama，但现在已经不再用了。以下是您也应该放弃它的原因。

A llama.cpp Wrapper With Amnesia一个患有失忆症的 llama.cpp 封装器一个患有失忆症的 llama.cpp 封装器 #

Ollama’s entire inference capability comes from llama.cpp, the C++ inference engine created by Georgi Gerganov in March 2023. Gerganov’s project is what made it possible to run LLaMA models on consumer laptops at all, he hacked together the first version in an evening, and it kicked off the entire local LLM movement. Today llama.cpp has over 100,000 stars on GitHub, 450+ contributors, and is the foundation that nearly every GGUF-based tool depends on.
Ollama 的全部推理能力都源自 llama.cpp，这是 Georgi Gerganov 在 2023 年 3 月创建的 C++ 推理引擎。正是 Gerganov 的项目让在消费级笔记本电脑上运行 LLaMA 模型成为可能，他在一个晚上就拼凑出了第一个版本，并由此开启了整个本地 LLM 运动。如今，llama.cpp 在 GitHub 上拥有超过 100,000 个 star，450 多名贡献者，并且是几乎所有基于 GGUF 的工具所依赖的基础。 Ollama 的全部推理能力都源自 llama.cpp，这是 Georgi Gerganov 在 2023 年 3 月创建的 C++ 推理引擎。正是 Gerganov 的这个项目使得在消费级笔记本电脑上运行 LLaMA 模型成为可能，他在一个晚上就拼凑出了第一个版本，从而开启了整个本地 LLM 运动。如今，llama.cpp 在 GitHub 上拥有超过 10 万颗星标和 450 名贡献者，并且几乎成为所有基于 GGUF 的工具的基础。

Ollama was founded in 2021 by Jeffrey Morgan and Michael Chiang, both previously behind Kitematic, a Docker GUI that was acquired by Docker Inc. They went through Y Combinator’s Winter 2021 batch, raised pre-seed funding, and launched publicly in 2023. From day one, the pitch was “Docker for LLMs”, a convenient wrapper that downloads and runs models with a single command. Under the hood, it was llama.cpp doing all the work.
Ollama 由 Jeffrey Morgan 和 Michael Chiang 于 2021 年创立，两人此前都曾参与开发 Kitematic（一款被 Docker Inc. 收购的 Docker GUI）。他们通过了 Y Combinator 2021 年冬季批次的孵化，完成了种子前轮融资，并于 2023 年正式发布。从第一天起，他们的核心理念就是“LLMs 的 Docker”，即一个可以通过单条命令下载并运行模型的便捷封装工具。而在底层，所有的工作都是由 llama.cpp 完成的。 Ollama 由杰弗里·摩根（Jeffrey Morgan）和迈克尔·蒋（Michael Chiang）于 2021 年创立，两人此前曾参与开发 Docker GUI 工具 Kitematic，该工具已被 Docker 公司收购。他们参加了 Y Combinator 2021 年冬季批次的项目，获得了种子前融资，并于 2023 年公开发布。从一开始，其宣传口号就是“用于大语言模型的 Docker”，即一个只需一条命令就能下载并运行模型的便捷工具。其底层技术是 llama.cpp。

For over a year, Ollama’s README contained no mention of llama.cpp. Not in the README, not on the website, not in their marketing materials. The project’s binary distributions didn’t include the required MIT license notice for the llama.cpp code they were shipping. This isn’t a matter of open-source etiquette, the MIT license has exactly one major requirement: include the copyright notice. Ollama didn’t.
一年多以来，Ollama 的 README 中从未提及过 llama.cpp。无论是 README、官方网站，还是其营销材料中都没有。该项目的二进制发行版中，也没有包含其所分发的 llama.cpp 代码所要求的 MIT 许可声明。这不仅仅是开源礼仪的问题，MIT 许可只有一个核心要求：必须包含版权声明。而 Ollama 没有做到。 一年多来，Ollama 的 README 文件中从未提及 llama.cpp。无论是 README 文件、网站还是营销材料，都没有提及。该项目的二进制分发版也没有包含他们所分发的 llama.cpp 代码所需的 MIT 许可证声明。这并非开源礼仪的问题，MIT 许可证只有一个主要要求：包含版权声明。而 Ollama 没有做到。

The community noticed. GitHub issue #3185 was opened in early 2024 requesting license compliance. It went over 400 days without a response from maintainers. When issue #3697 was opened in April 2024 specifically requesting llama.cpp acknowledgment, community PR #3700 followed within hours. Ollama’s co-founder Michael Chiang eventually added a single line to the bottom of the README: “llama.cpp project founded by Georgi Gerganov.”
社区注意到了这一点。2024 年初，GitHub 提交了第 #3185 号 issue，要求遵守许可协议。在超过 400 天的时间里，维护者都没有做出回应。当 2024 年 4 月提交了第 #3697 号 issue，专门要求对 llama.cpp 进行致谢时，社区在几小时内便提交了第 #3700 号 PR。Ollama 的联合创始人 Michael Chiang 最终在 README 的底部添加了一行字：“llama.cpp 项目由 Georgi Gerganov 发起。” 社区注意到了。2024 年初，GitHub 上的第 3185 号问题被提出，要求遵守许可证。该问题在超过 400 天的时间里都没有得到维护者的回应。2024 年 4 月，当第 3697 号问题专门要求 Ollama 承认 llama.cpp 时，社区的第 3700 号 PR 在数小时内就提交了。Ollama 的联合创始人迈克尔·蒋最终在 README 的底部添加了一行字：“llama.cpp 项目由 Georgi Gerganov 创立。”

The response to the PR was revealing. Ollama’s team wrote: “We spend a large chunk of time fixing and patching it up to ensure a smooth experience for Ollama users… Overtime, we will be transitioning to more systematically built engines.” Translation: we’re not going to give llama.cpp prominent credit, and we plan to distance ourselves from it anyway.
对该 PR 的回复很有启发性。Ollama 团队写道：“我们花费了大量时间进行修复和打补丁，以确保 Ollama 用户的体验顺畅……随着时间的推移，我们将过渡到构建更系统化的引擎。” 翻译一下：我们不打算给 llama.cpp 显著的功劳，而且无论如何我们都计划与之保持距离。 对于公关回应的解读耐人寻味。奥拉马团队写道：“我们花费大量时间对其进行修复和修补，以确保奥拉马用户获得流畅的体验……随着时间的推移，我们将转向更系统构建的引擎。”翻译过来就是：我们不会给 llama.cpp 太多的突出赞誉，而且无论如何我们打算与它保持距离。

As one Hacker News commenter put it: “I’m continually puzzled by their approach, it’s such self-inflicted negative PR. Building on llama is perfectly valid and they’re adding value on ease of use here. Just give the llama team proper credit.” Another: “The fact that Ollama has been downplaying their reliance on llama.cpp has been known in the local LLM community for a long time.”
正如一位 Hacker News 评论者所言：“我一直对他们的做法感到困惑，这简直是在自讨苦吃，制造负面公关。基于 llama 进行开发完全没问题，而且他们在易用性方面确实增加了价值。只要给 llama 团队应有的认可就好了。”另一位评论者则表示：“Ollama 一直在淡化他们对 llama.cpp 的依赖，这一点在本地 LLM 社区中早已是公开的秘密。” 正如一位 Hacker News 的评论者所说：“我一直对他们这种做法感到困惑，这简直是自找负面公关。基于 llama 进行开发完全没问题，而且他们在易用性方面确实增加了价值。只要给 llama 团队应有的认可就行。”另一位评论者则表示：“Ollama 一直在淡化他们对 llama.cpp 的依赖，这一情况在本地的 LLM 社区早就人尽皆知。”

The Fork That Made Things Worse让事情变得更糟的分叉那把让情况变得更糟的叉子 #

In mid-2025, Ollama followed through on that distancing. They moved away from using llama.cpp as their inference backend and built a custom implementation directly on top of ggml, the lower-level tensor library that llama.cpp itself uses. Their stated reason was stability, llama.cpp moves fast and breaks things, and Ollama’s enterprise partners need reliability.
2025 年年中，Ollama 兑现了那种疏离。他们不再使用 llama.cpp 作为推理后端，而是直接基于 ggml 构建了一个自定义实现，而 ggml 正是 llama.cpp 本身所使用的底层张量库。他们给出的理由是稳定性：llama.cpp 更新迭代极快且容易导致功能损坏，而 Ollama 的企业合作伙伴需要的是可靠性。 2025 年年中，Ollama 实施了这一策略。他们不再使用 llama.cpp 作为推理后端，而是直接基于 llama.cpp 自身使用的底层张量库 ggml 构建了自定义实现。他们给出的理由是稳定性，llama.cpp 发展迅速但容易出问题，而 Ollama 的企业合作伙伴需要可靠性。

The result was the opposite. Ollama’s custom backend reintroduced bugs that llama.cpp had solved years ago. Community members flagged broken structured output support, vision model failures, and GGML assertion crashes across multiple versions. Models that worked fine in upstream llama.cpp failed in Ollama, including new releases like GPT-OSS 20B, where Ollama’s implementation lacked support for tensor types that the model required. Georgi Gerganov himself identified that Ollama had forked and made bad changes to GGML.
结果却适得其反。Ollama 的自定义后端重新引入了 llama.cpp 早在几年前就已解决的 bug。社区成员指出，在多个版本中都存在结构化输出支持损坏、视觉模型失效以及 GGML 断言崩溃等问题。在 upstream llama.cpp 中运行良好的模型在 Ollama 中却无法工作，包括像 GPT-OSS 20B 这样的新发布模型，因为 Ollama 的实现缺乏该模型所需的张量类型支持。Georgi Gerganov 本人也指出，Ollama 对 GGML 进行了分支开发并做出了错误的改动。 结果恰恰相反。Ollama 的自定义后端重新引入了 llama.cpp 多年前就已解决的错误。社区成员指出，多个版本中存在结构化输出支持故障、视觉模型故障以及 GGML 断言崩溃等问题。在上游 llama.cpp 中运行良好的模型在 Ollama 中却无法正常工作，包括像 GPT-OSS 20B 这样的新版本，Ollama 的实现缺少该模型所需的张量类型支持。Georgi Gerganov 本人也指出，Ollama 分支并修改了 GGML，造成了不良影响。

The irony is thick. They downplayed their dependence on llama.cpp for years, then when they finally tried to go it alone, they produced an inferior version of the thing they refused to credit.
这真是讽刺到了极点。多年来，他们一直淡化自己对 llama.cpp 的依赖；而当他们终于试图自立门户时，却做出了一款比他们拒绝承认功劳的原版更差的产品。 讽刺的是，多年来他们一直淡化对 llama.cpp 的依赖，可当他们最终决定单干时，却做出了一个比不上他们一直不愿承认的那款产品的劣质版本。

Benchmarks tell the story. Multiple community tests show llama.cpp running 1.8x faster than Ollama on the same hardware with the same model, 161 tokens per second versus 89. On CPU, the gap is 30-50%. A recent comparison on Qwen-3 Coder 32B showed ~70% higher throughput with llama.cpp. The performance overhead comes from Ollama’s daemon layer, poor GPU offloading heuristics, and a vendored backend that trails upstream.
基准测试说明了一切。多项社区测试表明，在相同硬件和相同模型的情况下，llama.cpp 的运行速度是 Ollama 的 1.8 倍，即每秒 161 个 token 对比 89 个 token。在 CPU 上，这一差距为 30-50%。最近针对 Qwen-3 Coder 32B 的对比显示，llama.cpp 的吞吐量高出约 70%。性能开销源于 Ollama 的守护进程层（daemon layer）、糟糕的 GPU 卸载启发式算法，以及落后于上游版本的内置后端。 基准测试说明了一切。多个社区测试表明，在相同硬件和相同模型的情况下，llama.cpp 的运行速度比 Ollama 快 1.8 倍，每秒处理 161 个标记，而 Ollama 为 89 个。在 CPU 上，差距为 30% 至 50%。最近在 Qwen-3 Coder 32B 上的比较显示，llama.cpp 的吞吐量高出约 70%。性能差距源于 Ollama 的守护进程层、糟糕的 GPU 卸载策略以及落后的第三方后端。

Misleading Model Naming 误导性的模型命名 误导性的模型命名 #

When DeepSeek released its R1 model family in January 2025, Ollama listed the smaller distilled versions, models like DeepSeek-R1-Distill-Qwen-32B, which are fine-tuned Qwen and Llama models, not the actual 671-billion-parameter R1, simply as “DeepSeek-R1” in their library and CLI. Running ollama run deepseek-r1 pulls an 8B Qwen-derived distillate that behaves nothing like the real model.
当 DeepSeek 在 2025 年 1 月发布其 R1 模型系列时，Ollama 在其库和 CLI 中仅将较小的蒸馏版本（例如 DeepSeek-R1-Distill-Qwen-32B，这些是经过微调的 Qwen 和 Llama 模型，而非实际拥有 6710 亿参数的 R1）简单地列为“DeepSeek-R1”。运行 ollama run deepseek-r1 会拉取一个基于 8B Qwen 的蒸馏模型，其表现与真正的模型完全不同。

This wasn’t an oversight. DeepSeek themselves named these models with the “R1-Distill” prefix. Hugging Face listed them correctly. Ollama stripped the distinction. The result was a flood of social media posts from people claiming they were running “DeepSeek-R1” on consumer hardware, followed by confusion about why it performed poorly, doing reputational damage to DeepSeek in the process.
这并非疏忽。DeepSeek 官方在命名这些模型时就使用了 “R1-Distill” 前缀。Hugging Face 的列举也是正确的。而 Ollama 抹去了这一区别。其结果是社交媒体上涌现出大量帖子，人们声称自己在消费级硬件上运行着 “DeepSeek-R1”，随后又因其性能不佳而感到困惑，在此过程中也损害了 DeepSeek 的声誉。

GitHub issues #8557 and #8698 requested separation of the models. Both were closed as duplicates with no fix. As of today, ollama run deepseek-r1 still launches a tiny distilled model. Ollama knew the difference and chose to obscure it, presumably because “DeepSeek-R1” drives more downloads than “DeepSeek-R1-Distill-Qwen-32B” does.
GitHub issue #8557 和 #8698 都曾请求将这些模型分开。但两者都作为重复问题被关闭，且未进行任何修复。截至今日， ollama run deepseek-r1 仍然启动的是一个微小的蒸馏模型。Ollama 清楚其中的区别，却选择将其掩盖，推测是因为 “DeepSeek-R1” 比 “DeepSeek-R1-Distill-Qwen-32B” 能带来更多的下载量。

The Closed-Source App 闭源应用 #

In July 2025, Ollama released a GUI desktop app for macOS and Windows. The app was developed in a private repository (github.com/ollama/app), shipped without a license, and the source code wasn’t publicly available. For a project that had built its reputation on being open-source, this was a jarring move.
2025 年 7 月，Ollama 发布了适用于 macOS 和 Windows 的 GUI 桌面应用。该应用是在一个私有仓库（ github.com/ollama/app ）中开发的，发布时未附带许可证，且源代码并未公开。对于一个建立在开源声誉之上的项目来说，这是一个令人震惊的举动。

Community members immediately raised concerns. The license issue received 40 upvotes. Developers found potential AGPL-3.0 dependencies in the binary. The website placed the download button next to a GitHub link, giving the impression users were downloading the MIT-licensed open-source tool when they were actually getting an unlicensed closed-source application. Maintainers were silent for months. The code was eventually merged into the main repo in November 2025, but the initial rollout revealed where the project’s instincts lie.
社区成员立即提出了质疑。关于许可证的问题获得了 40 个赞。开发者在二进制文件中发现了潜在的 AGPL-3.0 依赖项。网站将下载按钮放在 GitHub 链接旁边，给用户一种正在下载采用 MIT 许可证的开源工具的错觉，而实际上他们获取的是未经授权的闭源应用程序。维护者保持了数月的沉默。代码最终在 2025 年 11 月被合并到主仓库中，但最初的发布过程揭示了该项目的直觉导向。

As XDA put it: “If your project trades on being open source, you do not get to be vague about what is and is not open at launch.”
正如 XDA 所言：“如果你的项目是以开源为卖点，那么在发布时，你不能对什么是开源、什么不是开源含糊其辞。”

The Modelfile: Reinventing a Solved ProblemModelfile：重新发明一个已解决的问题 #

GGUF, the model format created by Georgi Gerganov, was designed with one core principle: single-file deployment. Bullet point #1 in the GGUF spec reads: “Full information: all information needed to load a model is contained in the model file, and no additional information needs to be provided by the user.” Chat templates, stop tokens, model metadata, it’s all embedded in the file. You point llama.cpp at a GGUF and it works.
GGUF 是由 Georgi Gerganov 创建的模型格式，其设计遵循一个核心原则：单文件部署。GGUF 规范中的第 1 条要点写道：“完整信息：加载模型所需的所有信息都包含在模型文件中，用户无需提供任何额外信息。”对话模板、停止标记、模型元数据，所有这些都嵌入在文件中。你只需将 llama.cpp 指向一个 GGUF 文件，它就能直接运行。

Ollama added the Modelfile on top of this. It’s a separate configuration file, inspired by Dockerfiles, naturally, that specifies the base model, chat template, system prompt, sampling parameters, and stop tokens. Most of this information already exists inside the GGUF file. As one Hacker News commenter put it: “We literally just got rid of that multi-file chaos only for Ollama to add it back.”
Ollama 在此基础上又增加了 Modelfile。这是一个独立的配置文件，灵感自然源自 Dockerfile，用于指定基础模型、聊天模板、系统提示词、采样参数以及停止标记（stop tokens）。其中大部分信息其实已经存在于 GGUF 文件中了。正如一位 Hacker News 评论者所言：“我们好不容易才摆脱了那种多文件带来的混乱，结果 Ollama 又把它加回来了。”

The problems with this approach compound quickly. Ollama only auto-detects chat templates it already knows about from a hardcoded list. If a GGUF file has a valid Jinja chat template embedded in its metadata but it doesn’t match one of Ollama’s known templates, Ollama falls back to a bare {{ .Prompt }} template, silently breaking the model’s instruction format. The user has to manually extract the chat template from the GGUF, translate it into Go template syntax (which is different from Jinja), and write it into a Modelfile. Meanwhile, llama.cpp reads the embedded template and just uses it.
这种方法产生的问题会迅速堆积。Ollama 只能从硬编码的列表中自动检测它已经识别的聊天模板。如果一个 GGUF 文件在其元数据中嵌入了有效的 Jinja 聊天模板，但该模板不符合 Ollama 已知的任何模板，Ollama 就会退回到原始的 {{ .Prompt }} 模板，从而在无形中破坏了模型的指令格式。用户必须手动从 GGUF 中提取聊天模板，将其转换为 Go 模板语法（这与 Jinja 不同），然后将其写入 Modelfile。与此同时，llama.cpp 则能直接读取嵌入的模板并直接使用。

Modifying parameters is worse. If you want to change the temperature or system prompt on a model you pulled from Ollama’s registry, the workflow is: export the Modelfile with ollama show --modelfile, edit it, then run ollama create to build a new model entry. Users have reported that this process copies the entire model, 30 to 60 GB, to change one parameter. As one user described it: “The ‘modelfile’ workflow is a pain in the booty. It’s a dogwater pattern and I hate it. Some of these models are 30 to 60GB and copying the entire thing to change one parameter is just dumb.”
修改参数则更糟糕。如果你想更改从 Ollama 注册表拉取的模型的 temperature（温度）或 system prompt（系统提示词），工作流程是：使用 ollama show --modelfile 导出 Modelfile，对其进行编辑，然后运行 ollama create 来构建一个新的模型条目。用户反映，为了更改一个参数，这个过程会复制整个模型，大小达 30 到 60 GB。正如一位用户所描述的那样：“‘modelfile’ 工作流简直麻烦透顶。这是一种烂透了的设计模式，我讨厌它。其中一些模型有 30 到 60GB，为了改一个参数就复制整个模型，简直是愚蠢至极。”

Compare this to llama.cpp, where parameters are command-line flags. Want a different temperature? Pass --temp 0.7. Different system prompt? Pass it in the API request. No files to create, no gigabytes to copy, no proprietary format to learn.

The Modelfile also locks users into Ollama’s Go template syntax, which is a different language from the Jinja templates that model creators actually publish. LM Studio accepts Jinja templates directly. llama.cpp reads them from the GGUF. Only Ollama requires you to translate between template languages, and gets it wrong often enough that entire GitHub issues are dedicated to mismatched templates between Ollama’s library and the upstream GGUF metadata.

The Registry Bottleneck #

When a new model drops, say a new Qwen, Gemma, or DeepSeek variant, GGUFs typically appear on Hugging Face within hours, quantized by community members like Unsloth or Bartowski. With llama.cpp, you can run them immediately: llama-server -hf unsloth/Qwen3.5-35B-A3B-GGUF:Q4_K_M. One command, straight from Hugging Face, no intermediary.

With Ollama, you wait. Someone at Ollama has to package the model for their registry, choose which quantizations to offer (typically just Q4_K_M and Q8_0, no Q5, Q6, or IQ quants), convert the chat template to Go format, and push it. Until then, the model doesn’t exist in Ollama’s world unless you do the Modelfile dance yourself.

This creates a recurring pattern on r/LocalLLaMA: new model launches, people try it through Ollama, it’s broken or slow or has botched chat templates, and the model gets blamed instead of the runtime. A recent PSA post titled “If you want to test new models, use llama.cpp/transformers/vLLM/SGLang” documented how Qwen models showed problems with tool calls and garbage responses that “only happen with Ollama” due to their vendored backend and broken template handling. As one commenter put it: “Friends don’t let friends use ollama.”

The quantization limitation is particularly frustrating. Ollama only supports creating Q4_K_S, Q4_K_M, Q8_0, F16, and F32 quantizations. If you need Q5_K_M, Q6_K, or any IQ quant, formats that llama.cpp has supported for years, you’re out of luck unless you do the quantization yourself outside of Ollama. When a user asked about Q2_K support, the response was effectively “use a different tool.” For a project that markets itself as the easy way to run models, telling users to go elsewhere for basic quantization options is telling.

Hugging Face eventually added support for ollama run hf.co/{repo}:{quant} by generating a Docker-style manifest on the fly, which partially addresses the availability problem. But even then, the file gets copied into Ollama’s hashed blob storage, you still can’t share the GGUF with other tools, and the template detection issues still apply. The fundamental architecture remains: Ollama inserts itself as a middleman between you and your models, and that middleman is slower, less capable, and less compatible than the tools it sits on top of.

The Cloud Pivot #

In late 2025, Ollama introduced cloud-hosted models alongside its local library. The tool that was synonymous with local, private inference started routing prompts to third-party cloud providers. Proprietary models like MiniMax appeared in the model list without clear disclosure that selecting them would send your data off-machine.

Users raised concerns about data routing, when you run a closed-source model like MiniMax-m2.7 through “Ollama Cloud,” your prompts may be forwarded to the external provider who actually hosts the model. Ollama’s own documentation says “we process your prompts and responses to provide the service but do not store or log that content,” but says nothing about what the third-party provider does with it. For models hosted by Alibaba Cloud, users noted there is no zero-data-retention guarantee.

This was compounded by CVE-2025-51471, a token exfiltration vulnerability that affects all Ollama versions. A malicious registry server can trick Ollama into sending its authentication token to an attacker-controlled endpoint during a normal model pull. The fix exists as a PR but took months to land. In a tool that built its brand on local privacy, a vulnerability that leaks credentials to arbitrary servers is not a minor issue, it’s an architectural philosophy problem.

The VC Pattern #

All of this makes more sense when you look at the incentive structure. Ollama is a Y Combinator-backed (W21) startup, founded by engineers who previously built a Docker GUI that was acquired by Docker Inc. The playbook is familiar: wrap an existing open-source project in a user-friendly interface, build a user base, raise money, then figure out monetization.

The progression follows the pattern cleanly:

1. Launch on open source, build on llama.cpp, gain community trust
2. Minimize attribution, make the product look self-sufficient to investors
3. Create lock-in, proprietary model registry format, hashed filenames that don’t work with other tools
4. Launch closed-source components, the GUI app
5. Add cloud services, the monetization vector

The model registry is worth examining. Ollama stores downloaded models using hashed filenames in its own format. If you’ve been pulling models through Ollama for months, you can’t just point llama.cpp or LM Studio at those files without extra work. You can bring your own GGUFs to Ollama via a Modelfile, but it’s deliberately friction-filled to take them out. This is a form of vendor lock-in that most users don’t notice until they try to leave.

What To Use Instead #

The tools Ollama wraps are directly accessible, and they’re not much harder to set up.

llama.cpp is the engine. It has an OpenAI-compatible API server (llama-server), a built-in web UI, full control over context windows and sampling parameters, and consistently better throughput than Ollama. In February 2026, Gerganov’s ggml.ai joined Hugging Face to ensure the long-term sustainability of the project. It’s truly community-driven, MIT-licensed, and under active development with 450+ contributors.

Mozilla’s llamafile takes the single-file idea further, it packages a model and the runtime into one executable that runs on six OSes with no install at all. Download, double-click, done.

llama-swap handles multi-model orchestration, loading, unloading, and hot-swapping models on demand behind a single API endpoint. Pair it with LiteLLM and you get a unified OpenAI-compatible proxy that routes across multiple backends with proper model aliasing.

If you want a desktop GUI, you have genuinely open-source options. Jan (AGPLv3) is a local-first chat app with a clean interface and full source code. koboldcpp (AGPL) is a llama.cpp fork with a built-in web UI and extensive configuration, fully open, fully auditable. Both are real FOSS projects, not wrappers hiding proprietary code behind an open-source engine.

Then there are the closed-source wrappers. LM Studio is proprietary software built on top of llama.cpp, and it’s the most popular alternative to Ollama for good reason: it offers the same one-click convenience with a proper GUI, accepts any GGUF, and exposes all the knobs. Crucially, LM Studio’s developers have acted in good faith toward the ecosystem. They maintain a proper acknowledgements page crediting llama.cpp and its license, and they don’t try to obscure what’s under the hood. It’s a closed-source product, but it’s not a parasitic one. Msty is another closed-source GUI sitting on top of open-source inference, with multi-model support and built-in RAG.

I’m not opposed to someone building a business by making FOSS convenient. That’s legitimate. But it’s worth being honest about what these tools are: commercial products that exist because of llama.cpp, not open-source alternatives to Ollama. The difference between a good-faith wrapper and a bad-faith one isn’t whether they charge money or ship proprietary code, it’s whether they respect the work they stand on. LM Studio does. Ollama didn’t.

Red Hat’s ramalama is worth a look too, a container-native model runner that explicitly credits its upstream dependencies front and center. Exactly what Ollama should have done from the start.

None of these tools require more than a few minutes to set up. The idea that Ollama is the only accessible option hasn’t been true for a long time.

The Bigger Picture #

Georgi Gerganov hacked together llama.cpp in an evening in March 2023 and kicked off a revolution in local AI. He and a community of hundreds of contributors have spent years making it possible to run increasingly powerful models on consumer hardware. That work is genuinely important, it’s the foundation that keeps local inference open and accessible.

Ollama wrapped that work in a nice CLI, raised VC money on the back of it, spent over a year refusing to credit it, forked it badly, shipped a closed-source app alongside it, and then pivoted the whole thing toward cloud services. At every decision point where they could have been good open-source citizens, they chose the path that made them look more self-sufficient to investors.

The local LLM ecosystem doesn’t need Ollama. It needs llama.cpp. The rest is packaging, and better packaging already exists.