IBM adds broader model support and faster execution to its Spyre AI stack

IBM detailed software updates on October 26 that bring broader model support and faster execution to its Spyre AI accelerator. The company says a portable model-serving path has been validated with Gemma 4, Qwen3 and Granite 4.2. The updates aim to preserve familiar PyTorch workflows while moving computation onto different hardware.

Spyre is designed for inference—the work of running trained models—alongside enterprise applications and data on IBM Z, LinuxONE and Power systems. IBM describes the update as making Spyre a first-class PyTorch backend: an execution target within the framework.

Same model code, different machinery

Unlike a conventional GPU, Spyre triggers computation as data reaches its compute engines. Its compiler must plan data movement, local memory use and work allocation across cores before execution.

The torch-spyre backend translates ordinary PyTorch code into an executable plan. Most model logic passes through unchanged; adapters can replace unsupported operations with mathematically equivalent forms.

An earlier October 13 PyTorch technical account documented the switch from a separate compiler stack to PyTorch’s Inductor backend. Its developers reported a compiler roughly seven times smaller, though substantial Spyre-specific extensions remain. One integration point still uses a workaround they are seeking to formalize upstream.

Portability alongside GPU optimization

IBM says some newer vLLM model implementations increasingly target specific models and frontier GPUs. The community is developing hardware-agnostic layers that separate portable code from hardware-specific paths. Early support has landed in vLLM, including the pathway validated with Spyre.

IBM reports that these portable layers delivered token throughput within 3.4% of native implementations on NVIDIA H100 GPUs, measured as a geometric mean across three models. That tests portability on GPUs; it does not compare Spyre’s speed with an H100.

Faster dispatch, whole-model checks

In a Spyre demonstration using Granite 3.3 8B, batch size one and sequence length 1,024, IBM removed computation-graph construction from data transfer and compute dispatch. It reports 1.7 times faster prefill—processing the prompt—and 2.4 times faster decode—generating subsequent tokens.

IBM engineers are also testing whole models against trusted CPU or GPU references. Those checks catch failures involving combinations of operations, layouts and compiler transformations that isolated tests can miss. Continuous testing helps detect framework changes that break hardware support and identifies which model-and-accelerator combinations have been exercised.