A ‘Coup des Gens’ Is Underway – and We’re Increasingly Living Under the Regime of the Algorithm

I recently attended a large meeting of faculty to discuss graduate students’ evaluation, recruitment and retention.

“Let the data drive your goals,” one of the speakers repeated, mantra-like — with genuine enthusiasm and conviction — and I couldn’t help but wince.

The slogan struck me as symptomatic of what social psychologist Shoshana Zuboff has dubbed a “coup des gens.”

If a coup d’état denotes the illegal replacement of one political system by another, a coup des gens — “gens” meaning “people” in French — is characterized by the forced replacement of human beings by abstract information systems.

In my recent book, “The Terminal Self,” I show how we are increasingly coerced to interact with computer technology in all aspects of our daily lives. Among many other effects, this coerced interaction forces us to sync our cognitive functions to the logic of the computer and feed it endless streams of data, rendering us ripe for constant surveillance and exploitation.

Managing the flows of ‘the new oil’

Data is now used by companies to help them evaluate, rank, select or dismiss potential customers. It can be used to measure medical risk, credit-worthiness, psychological health, job performance, spending habits, food preferences, moods, dating preferences and political views.

As European Consumer Commissioner Meglena Kuneva put it, “personal data is the new oil of the Internet and the new currency of the digital world.”

The high value placed on data has ushered in what French sociologist Paul Virilio calls an “informational fundamentalism” — an ideology that exalts digital information as the ultimate good and supreme power to which all must surrender their will, time and common sense.

In order to function efficiently, most institutions must intelligently manage these swelling flows of data. However, the technological capacity to manage vast quantities of data doesn’t necessarily lead to intelligent analysis.

On the contrary, notes media studies scholar Mark Andrejevic, “We have become intelligence analysts sorting through more data than we can absorb.” Individuals simply don’t have the brainpower to sift through the constantly growing flows of information they need to process and mine to make intelligent decisions.

An increasing part of our day-to-day lives requires not just processing the “information bombs” — the emails, messages, breaking news and announcements — that randomly detonate in our daily lives. It also demands that we constantly input information into the system.

 

Think of the growing number of knowledge workers and professionals in different sectors of the economy who essentially have become data entry assistants. Their jobs primarily consist of feeding always more information to an ever-hungrier digital organism, for reasons that seem obvious but are rarely questioned.

We see it in the granular-level data many public school teachers are now expected to enter about every aspect of their students’ learning. We see it in the endless, and often flawed, surveys professionals of all stripes are asked to complete about their daily tasks, progress and “satisfaction.”

This mind-numbing data entry work deskills knowledge workers, alienates them and betrays a tragic mismanagement of human capital.

The digital Trojan horse

If the duty to enter information is openly enforced at work, it’s covertly induced at home. Whenever we’re leisurely browsing the internet, consulting websites, clicking on links or sharing our pictures, politics and preferences, we’re also — unwillingly and unwittingly — producing gigantic volumes of information. Invisible others then greedily harvest, store, organize and sell this data for the purposes of social control, persuasion and behavior modification.

“On Google, you are what you click. On Facebook, you are what you share,” writes Upworthy CEO Eli Pariser.

This extraction of digital information can influence real human lives. In his extensive analysis of corporate surveillance in everyday life, digital culture scholar Wolfe Christl warns that data-mining companies are wielding personal data to automatically make decisions about people that may worsen existing inequalities.

For example, Christl shows how health analytics companies like GNS Healthcare gather massive amounts of data from genomics, medical records, lab data and mobile health devices to provide information about users to health insurance companies. These companies can then significantly manipulate the costs of health insurance for different categories of applicants.

Similarly, he points out that data collected about someone’s emotional stability, happiness and likelihood of having a baby can influence whether they’re hired, retained or promoted.

With algorithms increasingly displacing humans in this kind of decision making, citizens’ fates are increasingly determined by information systems. As Zuboff explains, it’s a trend that reduces people to sources of data extraction and targets for exploitation.

The most baffling aspect of this situation is not the surveillance capacities themselves, as daunting as they are. It’s our tacit submission to a regime of constant and remote surveillance that — except in spy movies or paranoid delusions — would have been considered preposterous a few decades ago.

More worrisome, in a 2017 interview on “60 Minutes,” former Google marketing manager Tristan Harris explained the insidious motives of data harvesting: Tech companies aren’t just trying to find information about users to sell it to companies that want to push merchandise. They’re competing to create the best “brain-hacking” programs that collect the most accurate information about what makes users tick emotionally and what seizes their attention.

The objective is to manipulate users’ emotional reactions — remotely and at the neural level. In the wrong hands, this sort of manipulation can be used not just to sell people stuff, but to also influence their political decisions. According to political scientist Collin Bennett, this project is well underway.

Access denied

To add insult to injury, it’s almost impossible for users to detect which information about them is being collected, who’s collecting it and what they do with it.

Every swipe on a smartphone screen and every message that’s typed is recorded as “signals” that help gain insight into a user’s personality, interests, mood and financial health. The credit scores of a user’s Facebook friends can even be used to calculate their own credit-worthiness. Because of the complete anonymity of the agencies that collect and use this data, it’s unrealistic to hope that one could correct — and avert the consequences of — the errors that inevitably emerge in someone’s “profile.”

It’s distressing to feel unilaterally monitored, powerless and uninformed. According to one study, those who realize that their privacy has been exposed without their consent experience “emotional, psychoanalytic and corporeal responses which are sometimes stultifyingly profound.”

But this surveillance is also a form of exploitation.

After all, we’re the ones producing this information. We’re the ones increasingly sacrificing our time to read website guidelines, watch instructional videos, fill out digital forms, forward information, sustain digital social networks and generate valuable information about ourselves for others to use.

Shouldn’t those who generate this value have more control over the conditions of its production, sale, dissemination and use? And if this right cannot be granted due to inextricable technological difficulties, shouldn’t the same level of transparency be imposed on those who collect, sell and use it?

The regime of the algorithm

The growing dependence on information, the normalization of surveillance and the increasing use of information systems to make important decisions about citizens’ lives amounts to a “coup des gens.”

However, democracy requires privacy, and freedom entails protection from invasion, whether it’s physical or digital. In his famous treatise on the Fourth Amendment, Supreme Court Justice Brandeis argued that “the right to be let alone” is “the right most valued by civilized men” and is “fundamental to a free and civil society.”

Adult citizens of a democratic society should have the right to decide which sort of information is being gathered about them, who can use that information and how they can use it. They should have the right to verify that this information is accurate and, when it isn’t, to correct it. They should have the right — and the responsibility — to question this bulimic addiction for data and its ultimate purpose.

As Dave Eggers writes in his novel “The Circle,” “The ceaseless pursuit of data to quantify the value of any endeavor is catastrophic to true understanding.”

This article is republished from The Conversation under a Creative Commons license. Read the original article.