You Are Now Remotely Controlled
Surveillance capitalists control the science and the scientists, the secrets and the truth.
By Shoshana Zuboff
Jan 24 2020
The debate on privacy and law at the Federal Trade Commission was unusually heated that day. Tech industry executives “argued that they were capable of regulating themselves and that government intervention would be costly and counterproductive.” Civil libertarians warned that the companies’ data capabilities posed “an unprecedented threat to individual freedom.” One observed, “We have to decide what human beings are in the electronic age. Are we just going to be chattel for commerce?” A commissioner asked, ‘‘Where should we draw the line?” The year was 1997.
The line was never drawn, and the executives got their way. Twenty-three years later the evidence is in. The fruit of that victory was a new economic logic that I call “surveillance capitalism.” Its success depends upon one-way-mirror operations engineered for our ignorance and wrapped in a fog of misdirection, euphemism and mendacity. It rooted and flourished in the new spaces of the internet, once celebrated by surveillance capitalists as “the world’s largest ungoverned space.” But power fills a void, and those once wild spaces are no longer ungoverned. Instead, they are owned and operated by private surveillance capital and governed by its iron laws.
And like our forebears who named the automobile “horseless carriage” because they could not reckon with its true dimension, we regarded the internet platforms as “bulletin boards” where anyone could pin a note. Congress cemented this delusion in a statute, Section 230 of the 1996 Communications Decency Act, absolving those companies of the obligations that adhere to “publishers” or even to “speakers.”
Only repeated crises have taught us that these platforms are not bulletin boards but hyper-velocity global bloodstreams into which anyone may introduce a dangerous virus without a vaccine. This is how Facebook’s chief executive, Mark Zuckerberg, could legally refuse to remove a faked video of Speaker of the House Nancy Pelosi and later double down on this decision, announcing that political advertising would not be subject to fact-checking.
All of these delusions rest on the most treacherous hallucination of them all: the belief that privacy is private. We have imagined that we can choose our degree of privacy with an individual calculation in which a bit of personal information is traded for valued services — a reasonable quid pro quo. For example, when Delta Air Lines piloted a biometric data system at the Atlanta airport, the company reported that of nearly 25,000 customers who traveled there each week, 98 percent opted into the process, noting that “the facial recognition option is saving an average of two seconds for each customer at boarding, or nine minutes when boarding a wide body aircraft.”
In fact the rapid development of facial recognition systems reveals the public consequences of this supposedly private choice. Surveillance capitalists have demanded the right to take our faces wherever they appear — on a city street or a Facebook page. The Financial Times reported that a Microsoft facial recognition training database of 10 million images plucked from the internet without anyone’s knowledge and supposedly limited to academic research was employed by companies like IBM and state agencies that included the United States and Chinese military. Among these were two Chinese suppliers of equipment to officials in Xinjiang, where members of the Uighur community live in open-air prisons under perpetual surveillance by facial recognition systems.
Privacy is not private, because the effectiveness of these and other private or public surveillance and control systems depends upon the pieces of ourselves that we give up — or that are secretly stolen from us.
Our digital century was to have been democracy’s Golden Age. Instead, we enter its third decade marked by a stark new form of social inequality best understood as “epistemic inequality.” It recalls a pre-Gutenberg era of extreme asymmetries of knowledge and the power that accrues to such knowledge, as the tech giants seize control of information and learning itself. The delusion of “privacy as private” was crafted to breed and feed this unanticipated social divide. Surveillance capitalists exploit the widening inequity of knowledge for the sake of profits. They manipulate the economy, our society and even our lives with impunity, endangering not just individual privacy but democracy itself. Distracted by our delusions, we failed to notice this bloodless coup from above.
The belief that privacy is private has left us careening toward a future that we did not choose, because it failed to reckon with the profound distinction between a society that insists upon sovereign individual rights and one that lives by the social relations of the one-way mirror. The lesson is that privacy is public — it is a collective good that is logically and morally inseparable from the values of human autonomy and self-determination upon which privacy depends and without which a democratic society is unimaginable.
Still, the winds appear to have finally shifted. A fragile new awareness is dawning as we claw our way back up the rabbit hole toward home. Surveillance capitalists are fast because they seek neither genuine consent nor consensus. They rely on psychic numbing and messages of inevitability to conjure the helplessness, resignation and confusion that paralyze their prey. Democracy is slow, and that’s a good thing. Its pace reflects the tens of millions of conversations that occur in families, among neighbors, co-workers and friends, within communities, cities and states, gradually stirring the sleeping giant of democracy to action.
These conversations are occurring now, and there are many indications that lawmakers are ready to join and to lead. This third decade is likely to decide our fate. Will we make the digital future better, or will it make us worse? Will it be a place that we can call home?
Epistemic inequality is not based on what we can earn but rather on what we can learn. It is defined as unequal access to learning imposed by private commercial mechanisms of information capture, production, analysis and sales. It is best exemplified in the fast-growing abyss between what we know and what is known about us.
Twentieth-century industrial society was organized around the “division of labor,” and it followed that the struggle for economic equality would shape the politics of that time. Our digital century shifts society’s coordinates from a division of labor to a “division of learning,” and it follows that the struggle over access to knowledge and the power conferred by such knowledge will shape the politics of our time.
The new centrality of epistemic inequality signals a power shift from the ownership of the means of production, which defined the politics of the 20th century, to the ownership of the production of meaning. The challenges of epistemic justice and epistemic rights in this new era are summarized in three essential questions about knowledge, authority and power: Who knows? Who decides who knows? Who decides who decides who knows?
During the last two decades, the leading surveillance capitalists — Google, later followed by Facebook, Amazon and Microsoft — helped to drive this societal transformation while simultaneously ensuring their ascendance to the pinnacle of the epistemic hierarchy. They operated in the shadows to amass huge knowledge monopolies by taking without asking, a maneuver that every child recognizes as theft. Surveillance capitalism begins by unilaterally staking a claim to private human experience as free raw material for translation into behavioral data. Our lives are rendered as data flows.
Early on, it was discovered that, unknown to users, even data freely given harbors rich predictive signals, a surplus that is more than what is required for service improvement. It isn’t only what you post online, but whether you use exclamation points or the color saturation of your photos; not just where you walk but the stoop of your shoulders; not just the identity of your face but the emotional states conveyed by your “microexpressions”; not just what you like but the pattern of likes across engagements. Soon this behavioral surplus was secretly hunted and captured, claimed as proprietary data.
The data are conveyed through complex supply chains of devices, tracking and monitoring software, and ecosystems of apps and companies that specialize in niche data flows captured in secret. For example, testing by The Wall Street Journal showed that Facebook receives heart rate data from the Instant Heart Rate: HR Monitor, menstrual cycle data from the Flo Period & Ovulation Tracker, and data that reveal interest in real estate properties from Realtor.com — all of it without the user’s knowledge.
These data flows empty into surveillance capitalists’ computational factories, called “artificial intelligence,” where they are manufactured into behavioral predictions that are about us, but they are not for us. Instead, they are sold to business customers in a new kind of market that trades exclusively in human futures. Certainty in human affairs is the lifeblood of these markets, where surveillance capitalists compete on the quality of their predictions. This is a new form of trade that birthed some of the richest and most powerful companies in history.