Weizmann researchers say AI can recreate viewed images with one hour of brain-scan data

Recreating what someone sees may require far less brain-scan data than before. Researchers at the Weizmann Institute of Science say their Brain-IT system can match comparison methods using one hour of recordings from a new participant, rather than 40. The finding concerns pictures viewed during scanning, not unrestricted access to a person’s thoughts.

Developed by Michal Irani and fellow researchers, Brain-IT reconstructs images from functional MRI, or fMRI, data. These scans measure brain activity through changes in blood flow and oxygen. CNET’s October 8 account describes the one-hour figure as the amount of data needed from a new subject—not the time required to generate a reconstruction.

Training beyond the scanner

The team trained the system on thousands of scans from eight volunteers in the publicly available Natural Scenes Dataset. Participants looked at specified pictures while being scanned, giving the model paired examples of an image and the brain activity associated with viewing it. Collecting more such examples would mean more time inside MRI machines.

Their workaround runs the relationship in both directions. An encoder predicts brain activity from an image. A decoder then uses that predicted activity to reconstruct the image. As Irani explained in a statement quoted by the New York Post, this loop lets the system train on pictures that were never shown during an actual fMRI scan.

The decoder separates two jobs: predicting an image’s structure and predicting its content. Those predictions feed a diffusion model, an AI image generator, to recreate the scene. Synthetic scans supply additional training examples without requiring each corresponding picture to be paired with a real scan from a volunteer.

Better detail, but a dog can become a goat

Irani says earlier approaches could preserve an image’s general character while getting composition and color wrong. Her team claims better reconstruction of both content and detail. The researchers also identified 128 functional regions whose activity helped connect scans to viewed content, including different responses to food and sports pictures.

The output still makes substantive mistakes. Irani told MIT Technology Review that the system had reconstructed a dog in a bathtub as a goat in a bathtub, according to Futurism’s account. That example illustrates an important limitation: a reconstruction can retain a scene’s setting while misidentifying what is in it.

Medical hopes and a conditional privacy risk

Brain-IT currently depends on high-resolution scans from large, expensive fMRI machines. Its demonstrated task is visual reconstruction, not decoding memories or language. Irani sees potential applications in studying the brain and helping people with paralysis communicate; those are proposed uses, not demonstrated clinical outcomes.

The team wants to explore auditory information, too. Reading dreams remains a more distant possibility: Irani says it would require overcoming the challenges of decoding video. Neither ambition changes what the current system has shown with still pictures viewed during scanning.

Privacy concerns hinge partly on whether such techniques could move to easier-to-use devices. Marcello Ienca, a neuroscientist and philosopher at the Technical University of Munich, warned MIT Technology Review of possible commercial misuse if similar decoding worked with wearable EEG devices. Futurism relayed his warning. That wearable application is hypothetical, not a demonstrated Brain-IT capability.