MLX LM is a Python package for generating text and fine-tuning large language models on Apple silicon with MLX. Install it with pip install mlx-lm, then run mlx_lm.chat for a chat in the terminal or mlx_lm.server --model <a Hugging Face repo or a path> for an HTTP server on port 8080 of localhost, whose API its docs say is intended to be similar to the OpenAI chat API. Its README says the default model for generation and chat is mlx-community/Llama-3.2-3B-Instruct-4bit, and that you pick any other MLX-compatible model with --model. Its server page warns that the server is not recommended for production as it only implements basic security checks, and documents max_tokens as defaulting to 512. This page repeats MLX LM's and MLX's own documents and ran nothing on a Mac.
pip install mlx-lm
mlx_lm.chat
conda install -c conda-forge mlx-lm.mlx_lm.generate --prompt "How tall is Mt Everest?" generates text from one prompt; mlx_lm.chat opens a chat REPL whose context is preserved during the lifetime of the REPL.mlx-community/Llama-3.2-3B-Instruct-4bit. Pass any MLX-compatible model with --model; the README says thousands are in the MLX Community organization on Hugging Face.mlx_lm.generate --model mistralai/Mistral-7B-Instruct-v0.3 --prompt "hello" downloads that model from the Hugging Face Hub.-h after any command to see its options.mlx when installed on macOS.python -c "import platform; print(platform.processor())" should print arm.--trust-remote-code. The README says that without the flag you are prompted in the terminal to trust remote code when running the model.mlx_lm.server --model mlx-community/Mistral-7B-Instruct-v0.3-4bit
The server page says this starts a text generation server on port 8080 of the localhost, and that the model is downloaded from the Hugging Face repo if it is not already in the local cache. A request then looks like this:
curl localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"messages": [{"role": "user", "content": "Say this is a test!"}], "temperature": 0.7}'
max_tokens defaults to 512. temperature defaults to 0.0, and stream to false.model field takes a path or a Hugging Face repo id; a local path must be relative to the directory the server was started in.curl localhost:8080/v1/models lists the locally available models.--kv-bits quantizes the KV cache to save memory on long generations, and the page says the server then processes requests one at a time. It also says attention on a quantized cache keeps a score matrix of prefill_step_size x context_length, and to decrease --prefill-step-size or the matrix can be larger than the memory the quantized cache saves.Run mlx_lm.server --help for the full list of options; this page did not read that output.
--max-kv-size n uses a rotating fixed-size key-value cache: smaller values such as 512 use very little RAM but give worse quality, and larger values such as 4096 or higher use more RAM and give better quality.--prefill-step-size sets how many prompt tokens are processed at once; the default is 2048. Smaller steps use less peak memory at some cost to prompt processing speed.[WARNING] Generating with a model that requires ..., the model will likely be slow on that machine.sysctl that raises the system wired memory limit, with a value larger than the model's size in megabytes and smaller than the machine's memory:sudo sysctl iogpu.wired_limit_mb=N
The README does not say whether that setting survives a restart. The same macOS knob is described from a different source on Mac GPU memory limit for local LLMs.
MLX LM's management page says downloaded models are stored in the Hugging Face cache. mlx_lm.manage --scan lists them, --pattern narrows that to one or a set, and mlx_lm.manage --delete --pattern <repo> deletes the models the pattern matches.
| MLX LM | Ollama | |
|---|---|---|
| Models come from | Hugging Face; the README says thousands are in the MLX Community | Ollama's FAQ says it pulls models from the Internet |
| Where downloads live | The Hugging Face cache | ~/.ollama/models on macOS |
| Server address | Port 8080 of localhost | 127.0.0.1 port 11434 by default |
The two projects' own documents give no speed comparison, and this page makes none. For the formats each one runs, see MLX or GGUF on a Mac.
| Claim | Source |
|---|---|
| Install, chat, generate, the default model, --model, the Hub download, --trust-remote-code | MLX LM README |
| The long prompt options, the rotating cache, the macOS 15 line and the sysctl | MLX LM README |
| The server command, port 8080, the production note, request defaults, KV cache quantization, /v1/models | MLX LM server page |
| The Hugging Face cache and the scan and delete commands | MLX LM, managing models |
| Python 3.11 or newer, mlx installed on macOS | MLX LM on PyPI |
| Apple silicon, native Python 3.10 or newer, macOS 14.0 or later, the non-native Python check | MLX documentation, Build and Install |
Ollama's address, ~/.ollama/models | Ollama docs, Local FAQ |
pip install mlx-lm, or conda install -c conda-forge mlx-lm, says its README. MLX LM's PyPI metadata asks for Python 3.11 or newer, and MLX needs Apple silicon, a native Python and macOS 14.0 or later.mlx_lm.server --model <a Hugging Face repo or a path>. Its docs say it listens on port 8080 of localhost, with an API intended to be similar to the OpenAI chat API, and that it is not recommended for production as it only implements basic security checks.max_tokens request field as defaulting to 512, so a request that sets none can stop there. Send a larger max_tokens to ask for more.mlx_lm.manage --scan lists them and mlx_lm.manage --delete --pattern <repo> deletes the matching ones.