Amazon opens SGLang serving tools to developers using its Trainium AI chips

Amazon is opening another route to its AI chips without asking developers to abandon their familiar model-serving interface. On November 12, 2026, it made SGLang-Neuron publicly available for AWS Trainium. The promise is continuity at the software interface; underneath, a separate plugin handles the hardware-specific work of running models and generating reinforcement-learning rollouts.

The release is accessible through an AWS Deep Learning Container, packaged by RadixArk in collaboration with Annapurna Labs. It follows the SGLang team’s October 22 limited beta, which admitted a small group of teams and supported both model serving and rollout generation on Trainium2 and Trainium3. That earlier announcement had put broader availability on the calendar for later in 2026.

A familiar front door, a different execution layer

Developers launch models through SGLang’s existing server interface and send requests through the same generation APIs. Installing the plugin registers Trainium support with SGLang; setting SGLANG_PLATFORM=neuron selects that support when the server starts. The familiar interface therefore sits above a specific backend rather than replacing the need for chip-specific implementation.

SGLang’s core retains the API server, request handling, RadixAttention and the bookkeeping for cached prompt prefixes. The plugin supplies Trainium-specific model implementations, attention, mixture-of-experts execution and the graph runner that prepares and executes compiled programs. AWS describes this as an out-of-tree design: the Trainium plugin and SGLang core can develop independently.

TorchNeuron registers Trainium as a native device in PyTorch, the framework used to build and run models. SGLang-Neuron uses standard PyTorch operations alongside custom chip-level routines written with the Neuron Kernel Interface. It then uses torch.compile to turn those operations and routines into executable programs for serving.

Two ways to separate the work

For serving, the plugin supports separating prompt processing from answer generation, known as prefill-decode disaggregation. One server processes the prompt and builds its key-value cache—the stored information needed to continue generation. It transfers that cache to a second server, which generates the remaining tokens, using NIXL over AWS’s Elastic Fabric Adapter network connection.

Both servers still use SGLang’s scheduler and request lifecycle. The plugin manages the Trainium execution and cache-transfer path. This divides a single serving job across two servers; it is distinct from splitting reinforcement learning across different kinds of hardware.

The earlier beta demonstrated that second arrangement. Reinforcement learning has a rollout phase, which generates model outputs, and a policy-update phase, which trains the model. The SGLang team said its first end-to-end run generated Qwen3-MoE-30B-A3B rollouts on Trainium3 while its Miles framework trained the policy on a single H200 node. It reported a stable training run, while describing performance work as ongoing.

The public starting point

AWS’s walkthrough uses a Trainium3 EC2 instance and launches openai/gpt-oss-120b inside the supplied container. The example configures a 10,240-token context and up to 64 running requests, then sends a chat-completions request to a local endpoint. Those are example deployment settings, not comparative performance results.

For model selection beyond that example, AWS directs developers to the project documentation for supported models, features and validated configurations. The October beta’s initial list included Qwen3, DeepSeek-V2-Lite and several GLM models, with further coverage in progress. That list describes the beta’s starting coverage, rather than a complete inventory of the public release.