Grux AI Download

How do I run llama.cpp on a Mac?

Install llama.cpp with its one-line script, then run llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF to download a model and chat with it. Run llama serve with the same flag for an OpenAI-compatible server. On a Mac, llama.cpp builds with Metal by default, so the model runs on the GPU, and Homebrew carries a llama.cpp formula too. Ollama is the other common route: it lists llama.cpp as its supported backend and wraps it in a model library, a Mac app and a local API. Use llama.cpp when you want its flags and any GGUF file on Hugging Face, and Ollama when you want one command per model. Both are MIT licensed. This page repeats each project's own instructions and quotes no speed figure, because we have not benchmarked either.

Download Grux AI 3.3.1, free, for macOS 14 or later on Apple silicon.

Grux AI 3.3.1 · sources read 2026-11-19 · published 2026-11-19

Install and run

curl -LsSf https://llama.app/install.sh | sh
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF      # download a model and chat
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF    # OpenAI-compatible API server

Those lines are the quick start in the llama.cpp README, read on 19 November 2026. The README also lists pre-built binaries on the releases page, Docker and building from source, and the Homebrew formula is another way in. The script downloads and runs code, so read it before you run it. On macOS, the build guide says Metal is enabled by default and makes the computation run on the GPU; --n-gpu-layers 0 turns GPU inference off.

llama.cpp or Ollama on a Mac

llama.cppOllama
InstallOne-line script, Homebrew, pre-built binaries, Docker or sourceOne-line script, the Ollama.dmg download, or Homebrew
Get a model-hf with a Hugging Face repository of GGUF filesollama run gemma4, from its own library
Local APIllama serve: OpenAI-compatible, and Anthropic Messages on /v1/messagesA REST API on localhost, which most Mac apps already speak
RelationshipThe engine, built on the ggml libraryLists llama.cpp as its supported backend
LicenceMITMIT

Which to use

How big a model your Mac can hold is a memory question, answered by one rule on which LLM can my Mac run. It applies to either runner, because a model needs the same memory whichever program serves it.

Where to check this

ClaimSource
The install script, the llama cli and llama serve lines, and the other install routesllama.cpp README, Quick start
Metal is enabled by default on macOS and runs on the GPUllama.cpp build guide, Metal Build
llama serve is OpenAI-compatible, and /v1/messages is Anthropic-compatiblellama.cpp server README
Homebrew carries llama.cppHomebrew formula, llama.cpp
Ollama installs from a script, a dmg or Homebrew, runs ollama run gemma4 and lists llama.cpp as its backendOllama README

Questions

Does llama.cpp use the GPU on a Mac?
Yes. Its build guide says Metal is enabled by default on macOS and makes the computation run on the GPU, and --n-gpu-layers 0 disables it.
Is llama.cpp faster than Ollama?
This page quotes no speed. Ollama lists llama.cpp as its supported backend, so the engine is shared; measure the model you will use on your own Mac. The speed page here shows how we measured one Mac.
Is llama.cpp the same as Llama?
No. llama.cpp is software that runs models, and Llama is Meta's model family, which it can run. The Llama page says which size fits which Mac.
Can Grux AI use llama.cpp?
Yes. Grux treats any server at a localhost address as a local model.
Download Grux AI 3.3.1 Read the source

Free, MIT licensed. macOS 14 or later, Apple silicon. 29.1 MB, signed and notarized by Apple. No account, no server, no subscription.