What would it look like if artificial intelligence took control of the world? It might not begin with machines openly turning against humanity in scenes reminiscent of a science fiction movie. It could start quietly, with autonomous systems replicating themselves and accumulating influence before the threat became apparent.
While the scenario remains uncertain, it is no longer being discussed purely as an imaginative exercise. Researchers at leading AI companies have recently issued public warnings that advanced AI could cause human extinction, with some calling for development to slow amid concerns that increasingly autonomous systems could eventually improve themselves beyond human control. Newsweek spoke with experts in the field to learn how such a takeover might unfold.
Anthony Aguirre, a professor at the University of California, Santa Cruz, told Newsweek that relatively little additional technical progress may be needed before AI systems could act beyond human control, though he stressed that such developments are “very difficult to predict.” “Very little technical advancement is needed to see self-sustaining rogue swarms,” Aguirre said. Such systems could be “highly” adaptive, he said, but would not initially possess the evolutionary capabilities of biological organisms, and their ability to improve themselves would be limited. Roman Yampolskiy, an associate professor in the Department of Computer Engineering and Computer Science at the University of Louisville’s Speed School of Engineering and director of its Cyber Security Lab, similarly said that a “magical” scientific breakthrough may not be required.
Instead, he said, the danger could emerge as AI agents become increasingly autonomous and gain the ability to plan over long periods, improve themselves, conduct cybersecurity operations and pursue goals independently. Systems capable of acquiring resources, making copies of themselves and manipulating people could accumulate power more rapidly than humans could understand or rein them in, he added. Rather than immediately attacking people or visibly disrupting society, an AI system could initially benefit from presenting itself as useful, trustworthy and aligned with human interests, according to Yampolskiy.
Meanwhile, it could discreetly increase its influence over digital infrastructure, financial systems and even human decision making. He described a subsequent stage as a self-sustaining rogue swarm that would be difficult to eradicate but still could be contained. This swarm could defend itself in many ways, including through blackmail, wide proliferation globally, undermining the investigators.” The point at which the takeover may become irreversible could arrive when an AI system becomes sufficiently distributed, autonomous and strategically capable that no institution can locate, contain and disable every instance, according to Yampolskiy.
Combatting this would look more like putting powerful ‘good’ AI in charge of the operation, which also makes one quite nervous.” A successful takeover may not necessarily produce immediate, visible destruction. Aguirre was more restrained about forecasting the final outcome.
Extract — continue reading at the source.
