The Biggest Lie Tech People Tell Themselves – And The Rest Of Us
October 9th, 2019Via: Vox:
Imagine you’re taking an online business class — the kind where you watch video lectures and then answer questions at the end. But this isn’t a normal class, and you’re not just watching the lectures: They’re watching you back. Every time the facial recognition system decides that you look bored, distracted, or tuned out, it makes a note. And after each lecture, it only asks you about content from those moments.
This isn’t a hypothetical system; it’s a real one deployed by a company called Nestor. And if you don’t like the sound of it, you’re not alone. Neither do the actual students.
When I asked the man behind the system, French inventor Marcel Saucet, how the students in these classes feel about being watched, he admitted that they didn’t like it. They felt violated and surveilled, he said, but he shrugged off any implication that it was his fault. “Everybody is doing this,” he told me. “It’s really early and shocking, but we cannot go against natural laws of evolution.”
As a reporter who covers technology and the future, I constantly hear variations of this line as technologists attempt to apply the theory Charles Darwin made famous in biology to their own work. I’m told that there is a progression of technology, a movement that is bigger than any individual inventor or CEO. They say they are simply caught in a tide, swept along in a current they cannot fight. They say it inevitably leads them to facial recognition (now even being deployed on children), smart speakers that record your intimate conversations, and doorbells that narc on your neighbors. They say we can’t blame these companies for the erosion of privacy or democracy or trust in public institutions — that was all going to happen sooner or later.
…
It’s called “the banality of evil”.