Sydney study finds tiny starting changes can alter what AI models learn

Tiny changes at the start of AI training can alter what a network learns, even when outside randomness is removed. University of Sydney researchers link that sensitivity to interwoven fractal patterns, suggesting that more precise starting settings may offer surprisingly little help in predicting which solution a trained model will reach.

Andrew Ly and Pulin Gong describe the findings in a paper published November 5 in Nature Machine Intelligence. They present the geometry as a candidate explanation for irreproducibility across the systems tested, not a proven rule governing every deep-learning model.

Training differences are often attributed to outside sources of randomness: the order of training examples, data augmentation or numerical effects from computing hardware. The study asks a different question: what happens when those factors are controlled, but a model begins with slightly different internal settings? The comparison is between nearby starting points, rather than identical runs somehow ceasing to be deterministic.

The researchers explain their proposed mechanism with a map. Each location represents a possible starting setup, colored according to the training outcome it eventually reaches. Locations leading to the same destination form a basin of attraction. In a “riddled basin,” locations leading to different destinations remain mixed together at arbitrarily fine scales. Zooming in does not reveal a clean separation between outcomes.

That differs from simply saying training is chaotic. Chaos concerns small starting errors growing along a system’s path. Riddling concerns which destination it ultimately reaches: shrinking the initial uncertainty can leave the final outcome almost as uncertain. The authors analytically establish the conditions for riddling in a minimal neural network; in deeper networks, they find measured signatures consistent with it.

According to the University of Sydney, the team trained approximately 500,000 neural networks on tasks including image recognition and text processing. The sensitivity appeared across tested architectures, datasets and training methods, and persisted in fully deterministic environments.

One larger image-recognition network provided a particularly sharp test of whether greater starting precision would help. The university’s account highlights two results from that network:

10 billion timesGreater starting precision

Making the starting settings this much more precise produced no measurable improvement in reproducibility.

About 48%Different internal configuration

A one-bit change to the starting settings led training to a different internal configuration in about 48% of cases.

An internal configuration is not the same thing as a prediction. The paper explicitly addresses that distinction: practical applications care about a network’s input-output behavior, not just its particular set of internal parameters. The authors therefore extend their measurements to learned functions and report that the persistent sensitivity also reaches that level.

Their diagnostic measures how much the chance of a changed outcome falls as the starting perturbation shrinks. A near-zero uncertainty exponent means increasingly precise initialization brings only marginal gains in predictability. The functional version applies that idea to differences in what networks learn, rather than differences in their weights alone.

The researchers also found a trade-off between learning effectiveness and reproducibility. Lowering the learning rate—the size of each adjustment during training—made results reproducible, but the networks learned less effectively. Settings that supported the most effective learning made outcomes harder to predict and reproduce.

The work introduces tools for measuring this sensitivity and points toward redesigning network architectures to reduce the conditions that produce instability. That remains a direction for future work, rather than a demonstrated general fix. The authors have also made simulation and analysis code available without restrictions through Code Ocean and Zenodo.