Don’t be evil. Avoid unintended consequences. Get the Ethical OS Toolkit.
- Summary:
- The Ethical OS Toolkit helps firms avoid creating software that has unexpected and potentially destructive consequences.
In recent weeks, employees at Google, Salesforce, Amazon, and Microsoft have all urged their employers to stop providing technology services to local and federal government agencies that engage in police surveillance, military work, and immigration enforcement and deportation. In May, Google stopped building artificial intelligence services for a Pentagon drone program after thousands of employees signed a petition asking the company to end its Department of Defense contract.
Google’s original “Don’t be Evil” motto, an aspiration shared by many idealists in the industry, has been quietly deprecated yet has run smack into the reality that there is no way to guarantee the software you’re building will not be used in unanticipated and, potentially evil ways. And that's not to mention there is an enormous amount of money to be made in providing advanced technologies to governments and companies whose motives are rarely, if ever, pristine.
What’s a fast-growing ethically-inclined company to do? The Institute for the Future (IFTF) and Omidyar Network, a self-described “philanthropic investment firm” founded in 2004 by eBay founder Pierre Omidyar, have created a toolkit designed to help technologists envision the potential risks and worst-case scenarios of how their technologies may be used in the future so they can anticipate issues and design and implement ethical solutions from the outset.
The Ethical OS Toolkit Or: How Not to Regret the Things You Will Build is already being piloted by nearly 20 tech companies, schools, and start-ups, including Mozilla, Techstars, and others.
The toolkit’s goal is to provide technologists with information about the new risks they should be paying attention to as well as choices they can make to safeguard users, communities, society and their own companies. It was designed by IFTF’s Jane McGonigal, a renowned expert in collaborative foresight and human interaction and Samuel Woolley, director of IFTF’s Digital Intelligence Lab. Raina Kumra, entrepreneur in residence at Omidyar Network’s Tech and Society Solutions Lab, provided insight during the toolkit’s development ensuring it’s useful to the kinds of technology companies Omidyar Network invests in and advises.
The introduction to the Toolkit begins by laying out the premise:
As technologists, it’s only natural that we spend most of our time focusing on how our tech will change the world for the better. Which is great. Everyone loves a sunny disposition. But perhaps it’s more useful, in some ways, to consider the glass half empty. What if, in addition to fantasizing about how our tech will save the world, we spent some time dreading all the ways it might, possibly, perhaps, just maybe, screw everything up? No one can predict exactly what tomorrow will bring (though somewhere in the tech world, someone is no doubt working on it). So until we get that crystal ball app, the best we can hope to do is anticipate the long-term social impact and unexpected uses of the tech we create today.
What’s in the toolkit?
- The toolkit lists fourteen areas where unintended consequences bearing major social impact may emerge.
- Technologists are presented with 14 “risky futures” scenarios where they can imagine various risky situations that could arise because of new technologies. They include scenarios where video-faking algorithms are so advanced that faked videos are impossible to distinguish from real footage (allowing anyone to manufacture video “proof” to back up any claim); or where data companies offer free health insurance to anyone who agrees to install a smart toilet in their home and submit its data—which can include detection of stress hormones, infectious disease, alcohol and drug use and more—to the company.
- A checklist of eight risk zones where hard-to-anticipate and unwelcome consequences are most likely to emerge. Most tech is designed with the best intentions. But once a product is released and reaches scale, all bets are off.
- Seven future-proofing strategies that help technologists prioritize identified risks, determine what the biggest and hardest to address threats are, and where to begin to develop strategies that will help mitigate those risks.
Said IFTF’s Samuel Woolley:
Most people create technology with the hope that it will make the world a better place, but our best intentions must incorporate actionable safeguards to ensure others don’t corrupt our ideas along the way. This toolkit helps anyone creating new technology contemplate worst-case scenarios and familiarize themselves with major areas of potential risk. As the unexpected high-jacking by Russian hackers of Facebook and Twitter during the 2016 election showed, considering unexpected risks is a must for the entire tech community. Our collective future will be the better for it.
My take
As computing technologies like AI, machine learning, facial recognition and so on became more accurate and powerful so does their potential for abuse by “bad actors” in ways the companies that created them didn’t anticipate.
The notion that Facebook could be weaponized to spread disinformation and political chaos almost certainly didn’t occur to young Mark Zuckerberg in his Harvard dorm room when he was nurturing his big idea and hiding out from the Winklevoss twins.
In retrospect, it should have occurred to somebody at some point in the Facebook growth spurt that there would be unintended consequences and not all of them would be positive. But, even Dr. Frankenstein didn’t grasp the full danger of his creation until too late. This from the book:
There is something at work in my soul, which I do not understand.
We have reached a point in the development of cognitive computing software where it would be foolhardy not to have processes that actively safeguard users, communities, society, and your company from future risk. At some point in the not-so-distant future it is certain to become not only an unavoidable moral and ethical issue, but a matter of legal due diligence.
It is easy to criticize this type of initiative as a 'holier than thou' effort from a player who benefitted mightily from commericalizing the internet. I prefer that history be the judge.