Meta adds real-headset AI testing to its open-source VR web toolkit

AI coding agents can now test immersive web apps on a Quest headset connected to a computer, rather than only inside a desktop emulator. Meta announced Immersive Web SDK 1.0 on October 14, 2026, pairing that hardware-testing capability with a visual scene editor that developers and agents can use on the same project.

Meta first launched IWSDK at Connect 2025 as an open-source framework for immersive experiences delivered through the web. An app remains a website: it can run in a desktop browser, then offer an immersive view when a headset is available. People can open it through a link without first downloading it from a store or installing additional software.

Version 1.0 packages common building blocks behind a stable programming interface: grabbing objects, moving around, physics, three-dimensional menus and mixed-reality features. It uses Three.js for its graphics foundation. Havok physics runs in a separate background worker by default, reducing competition with the main rendering thread.

The agent tools go beyond generating code. Each new project includes a Model Context Protocol server—a connection through which an AI agent can use development tools—and agent skills. These let the agent inspect a scene, move the headset and simulated inputs, pause the engine and search the actual IWSDK programming interface rather than guess how it works.

Previously, that testing loop ran in the desktop emulator. With 1.0, an agent can use the same commands against an app running on a Quest plugged into the developer’s computer. Meta’s demonstration shows an agent playing every level of Hyperform on Quest 3, finding slowdowns, fixing them and replaying until the game holds 90 frames per second. That is Meta’s demonstration, not an independently verified performance result.

The Agentic Scene Composer gives manual and automated work a shared destination. Developers can switch between running the app and editing it in the same window, arrange real project assets and save changes. The agent works through code, adding models, materials and behaviors to that same editable scene, then rendering it to check the result.

Agents can also build individual 3D models and inspect them from multiple angles before placing them. Meta used the workflow to build Connect Island, a virtual venue with a pavilion, session stage and day-night cycle. The company offers the result as an IWSDK project developers can open and edit.

The release also prepares apps for Meta VR Glasses, which Meta says will ship in spring 2027. Gaze acts as a pointer: look at a target, then pinch with either hand to select it. Existing ray-based buttons, menus and grabbable objects can use that input without separate gaze-specific code, according to Meta.

Developers can try this on desktop with an emulated gaze and hand, or on Quest 3 and 3S. On those headsets, head direction stands in for eye tracking, while a cropped view previews the glasses’ narrower field of view. That can expose menus placed too close to the edges; it is not a test using real eye tracking.

The versioned release notes add an important input detail: direct touch and active grabs take priority over gaze. Hand and controller rays return automatically if tracked gaze becomes unavailable. Real eye-tracking input on the glasses remains the later step, after the hardware launches.

npm create @iwsdk@latest