Grux AI Download

How do I run MLX LM on a Mac and serve a model over HTTP?

MLX LM is a Python package for generating text and fine-tuning large language models on Apple silicon with MLX. Install it with pip install mlx-lm, then run mlx_lm.chat for a chat in the terminal or mlx_lm.server --model <a Hugging Face repo or a path> for an HTTP server on port 8080 of localhost, whose API its docs say is intended to be similar to the OpenAI chat API. Its README says the default model for generation and chat is mlx-community/Llama-3.2-3B-Instruct-4bit, and that you pick any other MLX-compatible model with --model. Its server page warns that the server is not recommended for production as it only implements basic security checks, and documents max_tokens as defaulting to 512. This page repeats MLX LM's and MLX's own documents and ran nothing on a Mac.

Download Grux AI 3.3.1, free, for macOS 14 or later on Apple silicon.

Grux AI 3.3.1 · sources read 2026-11-19 · published 2026-11-19

Install and first run

pip install mlx-lm
mlx_lm.chat

What it needs

Serve a model over HTTP

mlx_lm.server --model mlx-community/Mistral-7B-Instruct-v0.3-4bit

The server page says this starts a text generation server on port 8080 of the localhost, and that the model is downloaded from the Hugging Face repo if it is not already in the local cache. A request then looks like this:

curl localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "Say this is a test!"}], "temperature": 0.7}'

Run mlx_lm.server --help for the full list of options; this page did not read that output.

Long prompts and large models

sudo sysctl iogpu.wired_limit_mb=N

The README does not say whether that setting survives a restart. The same macOS knob is described from a different source on Mac GPU memory limit for local LLMs.

Manage the models it downloads

MLX LM's management page says downloaded models are stored in the Hugging Face cache. mlx_lm.manage --scan lists them, --pattern narrows that to one or a set, and mlx_lm.manage --delete --pattern <repo> deletes the models the pattern matches.

Next to Ollama

MLX LMOllama
Models come fromHugging Face; the README says thousands are in the MLX CommunityOllama's FAQ says it pulls models from the Internet
Where downloads liveThe Hugging Face cache~/.ollama/models on macOS
Server addressPort 8080 of localhost127.0.0.1 port 11434 by default

The two projects' own documents give no speed comparison, and this page makes none. For the formats each one runs, see MLX or GGUF on a Mac.

What this page does not say

Where to check this

ClaimSource
Install, chat, generate, the default model, --model, the Hub download, --trust-remote-codeMLX LM README
The long prompt options, the rotating cache, the macOS 15 line and the sysctlMLX LM README
The server command, port 8080, the production note, request defaults, KV cache quantization, /v1/modelsMLX LM server page
The Hugging Face cache and the scan and delete commandsMLX LM, managing models
Python 3.11 or newer, mlx installed on macOSMLX LM on PyPI
Apple silicon, native Python 3.10 or newer, macOS 14.0 or later, the non-native Python checkMLX documentation, Build and Install
Ollama's address, ~/.ollama/modelsOllama docs, Local FAQ

Questions

How do I install MLX LM on a Mac?
Run pip install mlx-lm, or conda install -c conda-forge mlx-lm, says its README. MLX LM's PyPI metadata asks for Python 3.11 or newer, and MLX needs Apple silicon, a native Python and macOS 14.0 or later.
How do I start an OpenAI-style server with MLX LM?
Run mlx_lm.server --model <a Hugging Face repo or a path>. Its docs say it listens on port 8080 of localhost, with an API intended to be similar to the OpenAI chat API, and that it is not recommended for production as it only implements basic security checks.
Why do MLX LM server replies stop early?
Its server page documents the max_tokens request field as defaulting to 512, so a request that sets none can stop there. Send a larger max_tokens to ask for more.
Where does MLX LM keep downloaded models, and how do I delete one?
In the Hugging Face cache, says its management page. mlx_lm.manage --scan lists them and mlx_lm.manage --delete --pattern <repo> deletes the matching ones.
Download Grux AI 3.3.1 Read the source

Free, MIT licensed. macOS 14 or later, Apple silicon. 29.1 MB, signed and notarized by Apple. No account, no server, no subscription.