Rochester and NVIDIA target precise AI image edits with PixelDiT2

The University of Rochester on October 28 described research with NVIDIA aimed at a frustrating image-generation problem: changing one detail without disturbing another. Their model, PixelDiT2, works directly with pixels rather than a compressed image representation. The university says it approaches leading models’ performance while offering finer control over individual pixels.

The PixelDiT2 paper was submitted in September, and the team plans to present it at the Neural Information Processing Systems conference, or NeurIPS, in December. Rochester’s announcement describes a research advance, not a consumer-tool rollout.

Rochester professor Jiebo Luo describes a tradeoff in latent-space models: compressing the image cuts computation, but makes it harder to control specific parts because the model is not adjusting individual pixels directly.

A guide for generating pixels

Working directly in pixels brings its own difficulty. The authors say pixel-space diffusion has narrowed the image-quality gap with latent-space approaches, but still learns more slowly and trails them in final quality. They argue that learning visual structure and pixel generation simultaneously helps explain that gap.

PixelDiT2 separates those jobs with what the authors call representation grounding. A pretrained vision model, kept fixed during the process, guides individual image patches as the diffusion model removes noise. That lets the generator focus more on producing pixels, rather than also learning useful visual structure from scratch.

The design does not introduce an autoencoder, the component used to compress and reconstruct images. The extra vision model supplies guidance without routing image generation through that compressed-image bottleneck.

From image quality to editing control

Those scores measure image-generation quality, not whether a requested edit leaves unrelated details untouched. That distinction separates the quantitative results from the project’s practical editing goal.

The control goal builds on the team’s earlier PixelDiT model. Rochester describes that work as allowing users to focus on a region, such as glasses on a person’s face or a car’s color. PixelDiT was presented at CVPR 2026 and was a best-paper finalist, according to the university.

Luo and his collaborators are now pursuing pixel-space video generation. That adds two challenges he identifies: keeping transitions between frames smooth and handling much more data. He says PixelDiT2’s progress puts the team in a strong position to tackle that work.