MIT researchers show a neural network can assemble its own learning machinery

A network of randomly connected, neuron-like units can organize not just its connections, but the machinery that helps it learn. MIT researchers describe that result in an October 9 PNAS paper: simple connection-change rules produced a network that recognized patterns and classified images, offering a way to investigate how learning circuits form.

The study, “How Biological Synapses Self-Assemble Gradient Learning,” was described by MIT News on October 13. Qianli Liao, Liu Ziyin and Yulu Gan are co-first authors. Working with Tomaso Poggio and Mark Harnett, they developed and validated the self-organizing network with a central question in mind: what makes an artificial learning system useful for understanding a biological one?

From local rules to learning circuits

The network begins with several layers of randomly interconnected units. Within each layer, every neuron-like component follows the same rules for adjusting its connections. The researchers supply those rules; the units then interact under them and develop behavior. Self-organization here therefore does not mean learning without any instructions. It describes what emerges from a relatively simple set of instructions.

One rule strengthens a connection when the two units it joins activate at the same time. This is the principle often expressed as “neurons that fire together wire together.” The team uses rules of the kind thought to reshape brain circuits as people and other animals learn through experience.

Those interactions create motifs: arrangements of connections that become building blocks for working circuits. The researchers say these motifs assemble themselves under the simple rules and enable learning. The important distinction is that the network develops the structures that adjust its connections, rather than merely changing connection strengths inside a fully prescribed learning system.

A familiar optimization strategy emerges

As the network learned, its motifs reorganized using a strategy similar to gradient descent. That is the process of gradually adjusting connections in response to errors to improve performance. Most modern AI carries out gradient descent through some form of back-propagation, an algorithm that neuroscientists consider unlikely to be used by the brain.

The team compared its network with networks designed to learn through that conventional approach and found similar performance. It also demonstrated pattern recognition and image classification. The result pairs an emerging learning mechanism with successful task performance; the researchers’ proposed neuroscience test goes further, asking whether the mechanism can organize itself rather than being specified by a designer.

The brain doesn't have an external designer or architect.

Qianli Liao, study co-first author, speaking to MIT News

A starting point, not a model of the whole brain

Liao cautions that a self-organizing network does not necessarily learn the way a brain does. He presents self-organization as a starting framework for judging biological plausibility: whether an artificial system is a credible model of biological learning. Poggio’s proposed connection is conditional, too. If the brain uses the team’s motifs to perform the work back-propagation does in artificial networks, that would link the two at the level of fundamental mechanisms.

Biology also presents variation the researchers want to understand. Rules governing neural connections may differ across brain regions or change as the brain matures. Neuroscientists are investigating how those rules work and how aging or disease might disrupt them, with the aim of developing therapies and prevention strategies.

Liao points to brain-computer interfaces as one potential application of that understanding. These systems aim to restore lost communication abilities or let paralyzed patients control neuroprosthetic limbs. He argues that knowing the rules behind neuron-to-neuron interactions is essential; designing a circuit that performs well is not enough to supply that knowledge.

Understanding the brain remains the group’s main objective. Liao says the knowledge could eventually help engineers build machine-learning systems that perform better or behave in more human-like ways. That is a possible longer-term benefit, not an outcome demonstrated by the image-classification experiment.