Nous Research Adds One-Click Setup for Local AI Models in Hermes Desktop

Nous Research has added a one-click flow to Hermes Desktop that takes over model-sizing work: it reads the machine, selects a compatible open-weight model, downloads it and configures local inference.

A model that looks appealing on paper may not fit a particular computer’s graphics memory. Hermes presents every catalog option with a green, amber or red verdict. Green means it can run entirely in GPU memory; amber means it spills into system RAM and will be slower; red means it is incompatible with that machine.

Behind that interface, Hermes bundles and manages an official build of llama.cpp matched to the user’s hardware. Its supported inference backends span CUDA, Metal, Vulkan, HIP and CPU, without requiring users to configure the engine themselves.

Hermes chooses the highest-quality quantized build that fits fully in GPU memory, with 4-bit as its minimum supported quantization. A quantized model stores weights with fewer bits to reduce memory use; Hermes will not go below that floor, so some models remain unavailable rather than being reduced further.

The app starts models at a context window that fits in GPU memory and expands toward the model’s native maximum as a conversation needs more room. Recommended models are guaranteed at least a 64K context window. When a model must overflow into system RAM, Hermes prioritizes moving expert weights and avoids moving the attention cache, trading speed to preserve that context capacity.

How the local runtime behaves

• Users can start the local-model flow on first launch or return to it through Settings, then Providers, then Local Models.
• Idle models unload after 15 minutes and reload when the user sends another message.
• Hermes can also connect to OpenAI-compatible endpoints, including Unsloth and Ollama integrations.

The change removes configuration work, not the computational cost of a constrained system. On CPU-only or low-VRAM machines, Hermes may spend minutes on the first request processing its system prompt and tool schemas before producing output. That delay occurs during prefill, the stage in which the model processes input before generating a response.

Hermes supports a hybrid arrangement in which local inference handles routine requests and a configured cloud provider serves as a fallback. The desktop app is described as free and MIT-licensed, works on macOS 12 or later, Windows 10/11 and Linux, and does not require an account for local models.