Try to imagine the last 11 minutes of Lion Air Flight 610 in October.
The plane is a new machine, Boeing’s sleek and intelligent 737 Max 8, fitted with an advanced electronic brain. After takeoff, this cyberpilot senses that something is wrong with the angle of ascent and starts to force the jetliner down.
A tug of war follows between men and computer, at 450 miles an hour – the human pilots trying to right the downward plunge, the automatic pilot taking it back. The bot wins. The jetliner crashes into the Java Sea. All 189 onboard are killed.
Here’s the most agonizing part: The killer was a smart computer designed to protect a gravity-defiance machine from error. It lacked judgment and intuition, precisely because those traits can be fatal in guiding an aerodynamic tube through the sky.
We still don’t know why the pilots of that flight couldn’t return to manual control. A report by the Federal Aviation Administration in 2013 found that 60 percent of accidents over a decade were linked to confusion between pilots and automated systems.
Flight 610 represents the hinge in history we’ve arrived at – with the bots, the artificial intelligence and the social media algorithms now shaping the fate of humanity at a startling pace.
Like the correction system in the 737, these inventions are designed to make life easier and safer – or at least more profitable for the owners. And they do, for the most part. The overall idea is to outsource certain human functions, the things prone to faulty judgment, while
At what point is control lost and the creations take over? How about now?
It was 200 years ago that Mary Shelley published a story of a monster who is still very much with us. Her book “Frankenstein” is about the consequences of man playing God.
Shelley’s concerns were raised at the peak of the Industrial Revolution, when the Western world was transformed from sleepy agricultural societies into a frenetic age of factories, machines and overcrowded cities. All the helpful inventions also produced mass dislocation, life-killing pollution, child labor and an expansion of human enslavement.
Today we are close to creating a human brain inside a computer – an entirely new species.
Facebook, once all baby pictures and high school reunions, is a monster of misinformation. And Facebook’s creator is more clueless than Dr. Frankenstein about the dangers of what he has unleashed on the world.
Mark Zuckerberg, its CEO, has glibly assured us that building advanced artificial intelligence systems will root out the hate, lies and propaganda passed among the 2 billion active users. But fake news is the mother’s milk of Facebook. The AI may only make it easier for mass manipulation.
Driverless cars will soon be available for ride-sharing in the United States. If they can reduce the carnage on the roads this will be a good thing. Except that this year a bot-car killed a woman in Arizona, and others have been slower than humans to react. There shouldn’t be any rush to hand over the steering wheel to a driver without a heartbeat.
It’s not Luddite to see the lesson from Mary Shelley’s era to our own, at the cusp of an age of technological totalitarianism. Nor is it Luddite to ask for more screening, more ethical considerations, more projections of what can go wrong, as we surrender judgment, reason and oversight to our soulless creations.
As haunting as those final moments inside the cockpit of Flight 610 were, it’s equally haunting to grasp the full meaning of what happened: The system overrode the humans and killed everyone. Our invention. Our folly.
Tim Egan is a columnist for The New York Times.
