[D66] The Big AI Risk Not Enough People Are Seeing

[René Oudeweg]

theatlantic.com
The Big AI Risk Not Enough People Are Seeing
Tyler Austin Harper
16–20 minutes

Beware technology that makes us less human.

“Our focus with AI is to help create more healthy and equitable 
relationships.” Whitney Wolfe Herd, the founder and executive chair of 
the dating app Bumble, leans in toward her Bloomberg Live interviewer. 
“How can we actually teach you how to date?”

When her interviewer, apparently bemused, asks for an example of what 
this means, Herd launches into a mind-bending disquisition on the future 
of AI-abetted dating: “Okay, so for example, you could in the near 
future be talking to your AI dating concierge, and you could share your 
insecurities. ‘I just came out of a breakup. I have commitment issues.’ 
And it could help you train yourself into a better way of thinking about 
yourself. And then it could give you productive tips for communicating 
with other people. If you want to get really out there, there is a world 
where your dating concierge could go and date for you with other dating 
concierges.” When her audience lets out a peal of uneasy laughter, the 
CEO continues undeterred, heart-shape earrings bouncing with each sweep 
of her hands. “No, no, truly. And then you don’t have to talk to 600 
people. It will then scan all of San Francisco for you and say, These 
are the three people you really ought to meet.”

What Herd provides here is much more than a darkly whimsical peek into a 
dystopian future of online dating. It’s a window into a future in which 
people require layer upon layer of algorithmic mediation between them in 
order to carry out the most basic of human interactions: those involving 
romance, sex, friendship, comfort, food. Implicit in Herd’s 
proclamation—that her app will “teach you how to date”—is the assumption 
that AI will soon understand proper human behavior in ways that human 
beings do not. Despite Herd’s insistence that such a service would 
empower us, what she’s actually describing is the replacement of human 
courtship rituals: Your digital proxy will go on innumerable dates for 
you, so you don’t have to practice anything so pesky as flirting and 
socializing.

Read: America is sick of swiping

Hypothetical AI dating concierges sound silly, and they are not exactly 
humanity’s greatest threat. But we might do well to think of the Bumble 
founder’s bubbly sales pitch as a canary in the coal mine, a harbinger 
of a world of algorithms that leave people struggling to be people 
without assistance. The new AI products coming to market are 
gate-crashing spheres of activity that were previously the sole province 
of human beings. Responding to these often disturbing developments 
requires a principled way of disentangling uses of AI that are 
legitimately beneficial and prosocial from those that threaten to 
atrophy our life skills and independence. And that requires us to have a 
clear idea of what makes human beings human in the first place.

In 1977, Ivan Illich, an Austrian-born philosopher, vagabond priest, and 
ruthless critic of metastatic bureaucracies, declared that we had 
entered “the age of Disabling Professions.” Modernity was characterized, 
in Illich’s view, by the standardization and professionalization of 
everyday life. Activities that were once understood to be within the 
competencies of laypeople—say, raising children or bandaging the 
wounded—were suddenly brought under the purview of technical experts who 
claimed to possess “secret knowledge,” bestowed by training and elite 
education, that was beyond the ken of the untutored masses. The licensed 
physician displaced the local healer. Child psychologists and their 
“cutting edge” research superseded parents and their instincts. 
Data-grubbing nutritionists replaced the culinary wisdom of grandmothers.

Illich’s singular insight was that the march of professional reason—the 
transformation of Western civilization into a technocratic enterprise 
ruled by what we now call “best practices”—promised to empower us but 
actually made us incompetent, dependent on certified experts to make 
decisions that were once the jurisdiction of the common man. “In any 
area where a human need can be imagined,” Illich wrote, “these new 
professions, dominant, authoritative, monopolistic, legalized—and, at 
the same time, debilitating and effectively disabling the 
individual—have become exclusive experts of the public good.” Modern 
professions inculcate the belief not only that their credentialed 
representatives can solve your problems for you, but also that you are 
incapable of solving said problems for yourself. In the case of some 
industries, like medicine, this is plainly a positive development. Other 
examples, like the ballooning wellness industry, are far more dubious.

If the entrenchment of specialists in science, schooling, child-rearing, 
and so on is among the pivotal developments of the 20th century, the 
rise of online dating is among the most significant of the 21st. But one 
key difference between this more recent advancement and those of 
yesteryear is that websites such as Tinder and Hinge are defined not by 
disabling professionals with fancy degrees, but by disabling algorithms. 
The white-coated expert has been replaced by digital services that cut 
out the human middleman and replace him with an (allegedly) even smarter 
machine, one that promises to know you better than you know yourself.

Faith Hill: ‘Nostalgia for a dating experience they’ve never had’

And it’s not just dating apps. Supposed innovations including 
machine-learning-enhanced meal-kit companies such as HelloFresh, Spotify 
recommendations, and ChatGPT suggest that we have entered the Age of 
Disabling Algorithms as tech companies simultaneously sell us on our 
existing anxieties and help nurture new ones. At the heart of it all is 
the kind of AI bait-and-switch peddled by the Bumble CEO. Algorithms are 
now tooled to help you develop basic life skills that decades ago might 
have been taken as a given: How to date. How to cook a meal. How to 
appreciate new music. How to write and reflect. Like an episode out of 
Black Mirror, the machines have arrived to teach us how to be human even 
as they strip us of our humanity. We have reason to be worried.

As conversations over the dangers of artificial intelligence have heated 
up over the past 18 months—largely thanks to the meteoric rise of large 
language models like ChatGPT—the focus of both the media and Silicon 
Valley has been on Skynet scenarios. The primary fear is that chat 
models may experience an “intelligence explosion” as they are scaled up, 
meaning that LLMs might proceed rapidly from artificial intelligence to 
artificial general intelligence to artificial superintelligence (ASI) 
that is both smarter and more powerful than even the smartest human 
beings. This is often called the “fast takeoff” scenario, and the 
concern is that if ASI slips out of humanity’s control—and how could it 
not—it might choose to wipe out our species, or even enslave us.

These AI “existential risk” debates—at least the ones being waged in 
public—have taken on a zero-sum quality: They are almost exclusively 
between those who believe that the aforementioned Terminator-style 
dangers are real, and others who believe that these are Hollywood-esque 
fantasies that distract the public from more sublunar AI-related 
problems, like algorithmic discrimination, autonomous weapons systems, 
or ChatGPT-facilitated cheating. But this is a false binary, one that 
excludes another possibility: Artificial intelligence could 
significantly diminish humanity, even if machines never ascend to 
superintelligence, by sapping the ability of human beings to do human 
things.

The epochal impact of online dating is there for all to see in a simple 
line graph from a 2019 study. It shows the explosive growth of online 
dating since 1995, the year that Match.com, the world’s first 
online-dating site, was launched. That year, only 2 percent of 
heterosexual couples reported meeting online. By 2017, that figure had 
jumped to 39 percent as other ways of meeting—through friends or family, 
at work or in church—declined precipitously.

Besides online dating, the only way of meeting that increased during 
this period was meeting at a bar or restaurant. However, the authors of 
the study noted that this ostensible increase was a mirage: The 
“apparent post-2010 rise in meeting through bars and restaurants for 
heterosexual couples is due entirely to couples who met online and 
subsequently had a first in-person meeting at a bar or restaurant or 
other establishment where people gather and socialize. If we exclude the 
couples who first met online from the bar/restaurant category, the 
bar/restaurant category was significantly declining after 1995 as a 
venue for heterosexual couples to meet.” In other words, online dating 
has become hegemonic. The wingman is out. Digital matchmaking is in.

But even those selling online-dating services seem to know there’s 
something unsettling about the idea that algorithms, rather than human 
beings, are now spearheading human romance. A bizarre Tinder ad from 
last fall featured the rapper Coi Leray playing the role of Cupid, 
perched on an ominously pink stage, tasked with finding a date for a 
young woman. A coterie of associates, dressed in Hunger Games chic, 
grilled a series of potential suitors as Cupid swiped left until the 
perfect match was found. These characters put human faces on an inhuman 
process.

Leif Weatherby, an expert on the history of AI development and the 
author of a forthcoming book on large language models, told me that ads 
like this are a neat distillation of Silicon Valley’s marketing 
playbook. “We’re seeing a general trend of selling AI as ‘empowering,’ a 
way to extend your ability to do something, whether that’s writing, 
making investments, or dating,” Weatherby explained. “But what really 
happens is that we become so reliant on algorithmic decisions that we 
lose oversight over our own thought processes and even social 
relationships. The rhetoric of AI empowerment is sheep’s clothing for 
Silicon Valley wolves who are deliberately nurturing the public’s 
dependence on their platforms.” Curtailing human independence, then, is 
not a bug, but a feature of the AI gold rush.

Of course, there is an extent to which this nurtured dependence isn’t 
unique to AI, but is an inevitable by-product of innovation. The broad 
uptake of any new technology generally atrophies the human skills for 
the processes that said technology makes more efficient or replaces 
outright. The advent of the vacuum was no doubt accompanied by a 
corresponding decline in the average American’s deftness with a broom. 
The difference between technologies of convenience, like the vacuum or 
the washing machine, and platforms like Tinder or ChatGPT is that the 
latter are concerned with atrophying competencies, like romantic 
socializing or thinking and reflection, that are fundamental to what it 
is to be a human being.

Read: AI has lost its magic

The response to our algorithmically remade world can’t simply be that 
algorithms are bad, sensu stricto. Such a stance isn’t just untenable at 
a practical level—algorithms aren’t going anywhere—but it also 
undermines unimpeachably positive use cases, such as the employment of 
AI in cancer diagnosis. Instead, we need to adopt a more sophisticated 
approach to artificial intelligence, one that allows us to distinguish 
between uses of AI that legitimately empower human beings and those—like 
hypothetical AI dating concierges—that wrest core human activities from 
human control. But making these distinctions requires us to re-embrace 
an old idea that tends to leave those of us on the left rather 
squeamish: human nature.

Both Western intellectuals and the progressive public tend to be hostile 
to the idea that there is a universal “human nature,” a phrase that now 
has right-wing echoes. Instead, those on the left prefer to emphasize 
the diversity, and equality, of varying human cultural traditions. But 
this discomfort with adopting a strong definition of human nature 
compromises our ability to draw red lines in a world where AI encroaches 
on human territory. If human nature doesn’t exist, and if there is no 
core set of fundamental human activities, desires, or traits, on what 
basis can we argue against the outsourcing of those once-human endeavors 
to machines? We can’t take a stand against the infiltration of 
algorithms into the human estate if we don’t have a well-developed sense 
of which activities make humans human, and which activities—like 
sweeping the floor or detecting pancreatic cancer—can be outsourced to 
nonhuman surrogates without diminishing our agency.

One potential way out of this impasse is offered by the so-called 
capability approach to human flourishing developed by the philosopher 
Martha Nussbaum and others. In rejection of the kind of knee-jerk 
cultural relativism that often prevails in progressive political 
thought, Nussbaum’s work insists that advocating for the poor or 
marginalized, at home or abroad, requires us to agree on universal 
“basic human capabilities” that citizens should be able to develop. 
Nussbaum includes among these basic capabilities “being able to imagine, 
to think, and to reason” and “to engage in various forms of familial and 
social interaction.” A good society, according to the capability 
approach, is one in which human beings are not just theoretically free 
to engage in these basic human endeavors, but are actually capable of 
doing so.

As AI is built into an ever-expanding roster of products and services, 
covering dating, essay writing, and music and recipe recommendations, we 
need to be able to make granular, rational decisions about which uses of 
artificial intelligence expand our basic human capabilities, and which 
cultivate incompetence and incapacity under the guise of empowerment. 
Disabling algorithms are disabling precisely because they leave us less 
capable of, and more anxious about, carrying out essential human behaviors.

Of course, some will object to the idea that there is any such thing as 
fundamental human activities. They may even argue that describing 
behaviors like dating and making friends, critical thinking, or cooking 
as central to the human condition is ableist or otherwise bigoted. After 
all, some people are asexual or introverted. Others with mental 
disabilities might not be adept at reflection, or written or oral 
communication. Some folks simply do not want to cook, an activity which 
is historically gendered besides. But this objection relies on a sleight 
of hand. Identifying certain activities as fundamental to the human 
enterprise does not require you to believe that those who don’t or can’t 
engage in them are inhuman, just as embracing the idea that the human 
species is bipedal does not require you to believe that people born 
without legs lack full personhood. It only asks that you acknowledge 
that there are some endeavors that are vital aspects of the human 
condition, taken in the aggregate, and that a society where people 
broadly lack these capacities is not a good one.

Without some minimal agreement as to what those basic human capabilities 
are—what activities belong to the jurisdiction of our species, not to be 
usurped by machines—it becomes difficult to pin down why some uses of 
artificial intelligence delight and excite, while others leave many of 
us feeling queasy.

What makes many applications of artificial intelligence so disturbing is 
that they don’t expand our mind’s capacity to think, but outsource it. 
AI dating concierges would not enhance our ability to make romantic 
connections with other humans, but obviate it. In this case, technology 
diminishes us, and that diminishment may well become permanent if left 
unchecked. Over the long term, human beings in a world suffused with 
AI-enablers will likely prove less capable of engaging in fundamental 
human activities: analyzing ideas and communicating them, forging 
spontaneous connections with others, and the like. While this may not be 
the terrifying, robot-warring future imagined by the Terminator movies, 
it would represent another kind of existential catastrophe for humanity.

Whether or not the Bumble founder’s dream of 
artificial-intelligence-induced dalliances ever comes to fruition is an 
open question, but it is also somewhat beside the point. What should 
give us real pause is the understanding of AI, now ubiquitous in Big 
Tech, that underlies her dystopian prognostications. Silicon Valley 
leaders have helped make a world in which people feel that everyday 
social interactions, whether dating or making simple phone calls, 
require expert advice and algorithmic assistance. AI threatens to 
turbocharge this process. Even if your personalized dating concierge is 
not here yet, the sales pitch for them has already arrived, and that 
sales pitch is almost as dangerous as the technology itself: AI will 
teach you how to be a human.

About the Author

Tyler Austin Harper is an assistant professor of environmental studies 
at Bates College and a contributing writer at The Atlantic.