Run an LLM inference server on a Mac
Turns a Mac into an LLM inference server run from the menu bar.
Coding / product dossier
oMLX
Mac LLM server that cuts agent wait times from 90s to 5s
Product brief
What oMLX does.
oMLX turns your Mac into a full LLM inference server, run from the menu bar. It serves text, vision, OCR, embedding and reranker models with continuous batching, plus a RAM+SSD tiered KV cache that survives restarts, so Claude Code and Cursor respond in about 5s instead of 90s. OpenAI and Anthropic compatible APIs drop straight in. Native Swift, not Electron. Apache 2.0, open source.
Why we selected it
Editorially selected for practical value, novelty, and audience relevance.
- Best for
- AI builders and operators
- Category
- Coding
- Daily picks
- 1
- First selected
- 2026-08-30
Product preview saved with our daily selection
Capability scan
What it can help with.
Only capabilities supported by the product information we collected are listed here.
Serve several model types
Serves text, vision, OCR, embedding, and reranker models.
Use continuous batching
Includes continuous batching for model serving.
Retain a tiered KV cache across restarts
Uses a RAM-and-SSD tiered KV cache that survives restarts.
Expose compatible APIs
Provides APIs described as compatible with OpenAI and Anthropic APIs.
Best-fit use cases
Serving text models from a Mac.
Serving vision or OCR models from a Mac.
Running embedding or reranker models.
Connecting software that uses OpenAI- or Anthropic-compatible APIs to a local Mac server.
FAQ
Before you open it.
What model types can oMLX serve?
It is described as serving text, vision, OCR, embedding, and reranker models.
How is oMLX operated?
It runs from the Mac menu bar.
What happens to its KV cache after a restart?
The product description says its RAM-and-SSD tiered KV cache survives restarts.
Which API styles are supported?
Its APIs are described as compatible with OpenAI and Anthropic APIs.
Is oMLX open source?
Yes. It is described as open source under the Apache 2.0 license.
Selection history