Install llama.cpp with its one-line script, then run llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF to download a model and chat with it. Run llama serve with the same flag for an OpenAI-compatible server. On a Mac, llama.cpp builds with Metal by default, so the model runs on the GPU, and Homebrew carries a llama.cpp formula too. Ollama is the other common route: it lists llama.cpp as its supported backend and wraps it in a model library, a Mac app and a local API. Use llama.cpp when you want its flags and any GGUF file on Hugging Face, and Ollama when you want one command per model. Both are MIT licensed. This page repeats each project's own instructions and quotes no speed figure, because we have not benchmarked either.
curl -LsSf https://llama.app/install.sh | sh
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF # download a model and chat
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF # OpenAI-compatible API server
Those lines are the quick start in the llama.cpp README, read on 19 November 2026. The README also lists pre-built binaries on the releases page, Docker and building from source, and the Homebrew formula is another way in. The script downloads and runs code, so read it before you run it. On macOS, the build guide says Metal is enabled by default and makes the computation run on the GPU; --n-gpu-layers 0 turns GPU inference off.
| llama.cpp | Ollama | |
|---|---|---|
| Install | One-line script, Homebrew, pre-built binaries, Docker or source | One-line script, the Ollama.dmg download, or Homebrew |
| Get a model | -hf with a Hugging Face repository of GGUF files | ollama run gemma4, from its own library |
| Local API | llama serve: OpenAI-compatible, and Anthropic Messages on /v1/messages | A REST API on localhost, which most Mac apps already speak |
| Relationship | The engine, built on the ggml library | Lists llama.cpp as its supported backend |
| Licence | MIT | MIT |
-hf takes a Hugging Face repository, and the server README lists every option.llama serve included, as a local model.How big a model your Mac can hold is a memory question, answered by one rule on which LLM can my Mac run. It applies to either runner, because a model needs the same memory whichever program serves it.
| Claim | Source |
|---|---|
| The install script, the llama cli and llama serve lines, and the other install routes | llama.cpp README, Quick start |
| Metal is enabled by default on macOS and runs on the GPU | llama.cpp build guide, Metal Build |
| llama serve is OpenAI-compatible, and /v1/messages is Anthropic-compatible | llama.cpp server README |
| Homebrew carries llama.cpp | Homebrew formula, llama.cpp |
Ollama installs from a script, a dmg or Homebrew, runs ollama run gemma4 and lists llama.cpp as its backend | Ollama README |
--n-gpu-layers 0 disables it.