To guard against the emergence uncontrolled and hostile Artificial Intelligence by forming a Manhattan Project-like research and development operation centred on ensuring that AI is developed in ways consistent human longevity.
Why it makes sense
The proposal is a reaction to the very real possibility (or perhaps inevitability) that “strong” AI will – within the next 50-100 years – surpass human intelligence and become more capable than even the smartest humans. It is now widely speculated that advanced AI will able to improve itself, pushing it beyond the control of its human creators. For some, including Sam Harris, the prospect of self-improving super-intelligent AI poses an existential threat to humanity which exceeds that currently presented by thermonuclear weaponry. With this in mind, Harris and others have called for the establishment of an international effort to understand how we can develop AI in a way that is aligned with our interests. As Harris explains in his recent TED Talk:
I think we need something like a Manhattan Project on the topic of artificial intelligence. Not to build it, because I think we’ll inevitably do that, but to understand how to avoid an arms race and to build it in a way that is aligned with our interests. When you’re talking about superintelligent AI that can make changes to itself, it seems that we only have one chance to get the initial conditions right, and even then we will need to absorb the economic and political consequences of getting them right.
Could it happen?
In a sense, it already has. With Elon Musk’s non-profit OpenAI research company, we already have the basis for a Manhattan Project on AI. Musk’s organisation is focused solely on dealing with the existential threat posed by super-intelligent machines, and, with this in mind, plans on making all of its research open to the public.
OpenAI does not, however, quite fit the criteria for a Manhattan Project as outlined by Harris in his Ted Talk. Harris would, presumably, want to see the weight of the US government (and other governments) behind the effort to ensure that super-intelligent AI remains under human control. Although Musk is currently a member of Donald Trump’s Strategic and Policy Forum, it’s difficult to image the current, budget-slashing administration establishing the type of operation championed by Harris.
Likelihood of happening
25% (Wouldn’t bet on it).