PrismML Shrinks Qwen3.8 27B to 5.9 GB for Local AI

A model built from Qwen3.8 27B can now fit into a 5.9 GB weight package. PrismML has released Ternary Bonsai 2 27B, a compressed version that the company says retains 98.2% of the full-precision model’s aggregate benchmark performance in a footprint more than nine times smaller.

The release is designed for local deployment rather than a cloud-only workflow. Bonsai 2 accepts text and images, supports a 262K-token context window, and is available under the Apache 2.0 license. PrismML says it runs on NVIDIA GPUs through CUDA and on Apple devices through MLX, with a browser-based WebGPU demo.

The size reduction comes from how weights are stored

Bonsai 2 is not a smaller base model. PrismML represents Qwen3.8 27B’s weights with three values: negative one, zero, and positive one. Group-level FP16 scaling helps the compressed representation approximate a wider numerical range, producing an effective density of 1.76 bits per weight.

The average score conceals uneven losses

The headline retention number is a company-reported average across reasoning, math, coding, instruction following, vision, and agentic tool use. PrismML’s table puts coding at 81.58, compared with 82.17 for the full-precision model. Agentic and tool calling scored 77.57 versus 79.74, while vision had the largest listed drop: 78.59 versus 81.64.

Where those differences could matter

• Coding: PrismML’s reported score was 0.59 points below the full-precision model.
• Tool use: the reported gap was 2.17 points for agentic and tool-calling tasks.
• Vision: the reported gap was 3.05 points, the largest listed category difference.

A sharper second attempt at local workloads

PrismML says its first Bonsai 27B retained about 95% of its base model’s aggregate benchmark performance; Bonsai 2 raises that stated result above 98%. The company positions the new release for coding agents, computer-use systems, private document and image analysis, and hybrid arrangements that send selected work to remote models.

PrismML reports peak generation speeds of up to 143 tokens per second on an Nvidia GeForce RTX 5090 and 46.8 tokens per second on an M5 Max. Those hardware-specific figures do not answer how the model will perform in every local application. The practical test is whether its smaller task-level gaps remain acceptable when a workflow depends on interpreting images or taking repeated actions.