This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility: Researchers at the University of Sydney have identified a fundamental mathematical reason why artificial intelligence systems can learn different behaviors even when trained under almost identical conditions, a discovery that could help improve the reliability and trustworthiness of AI technologies. The study by Professor Pulin Gong from the School of Physics and his Ph.D. student Andrew Ly has been published in Nature Machine Intelligence.
As AI systems become increasingly embedded in health care, business, research and everyday life, ensuring they behave consistently and predictably has become a major challenge. The new research suggests that some forms of AI unpredictability are not due to engineering flaws or inconsistencies in computer hardware, but arise from the underlying mathematics of how neural networks learn. "Researchers have often attributed differences in AI behavior to external sources of randomness, such as the order in which training data is presented or tiny numerical variations introduced by hardware," Gong said.
"Our findings point to something deeper. Even when these factors are removed, extremely small differences in a model's starting conditions can still lead to different outcomes." Neural networks, the technology behind today's most advanced AI systems, learn by adjusting millions or even billions of internal parameters during training. The Sydney team investigated why networks sometimes arrive at different solutions despite using the same data, training procedure and deterministic computing environment.
Ly said, "Imagine two AI systems trained to detect cancer using the same medical images and the same training procedure. Tiny differences in their initial parameters could lead them to learn different behaviors. Our research helps explain how such differences arise and might inform methods for making these systems more consistent." Understanding the source of these differences could guide the design of more dependable systems and reduce the cost and uncertainty associated with developing large-scale models.
To explain the phenomenon, the researchers use the analogy of a map. Imagine every possible starting point for training as a location on that map, colored according to the outcome it eventually produces. Regions leading to the same outcome form what mathematicians call a "basin of attraction." The team found evidence that these basins are organized as highly intricate fractals known as "riddled basins." In such a landscape, every point lies arbitrarily close to points leading somewhere else.
No matter how far you zoom in, the different regions remain interwoven. The implication is striking. Increasing the precision of a network's starting conditions may do little to improve confidence about where training will end up.
Extract — continue reading at the source.
