The AI Con is an important demolition of the hype around AI, and points to how we need to take control of the direction and purpose of new technology, finds Kevin Crane

It’s been a couple of years now since so-called Artificial Intelligence (AI) has been the buzzword of choice for capitalism, but the noise around it hasn’t let up. The official line from both Western governments and the Big Tech companies continues to be that AI is the future, and to this end they are continuing to commit untold (and potentially unrealisable) quantities of funding, resources and energy to expand greatly its development and use. This is at the same time, of course, as real anxieties about AI products and their impact on both people and planet Earth are significantly rising. Many people who are critical, or simply apprehensive, of AI still are not confident to argue about it. For that reason alone, it is worthwhile for people on the left to continue to arm and motivate our own side with books on the topic.

The AI Con is a title that lets the reader know exactly what they’re getting, both in regard to what the two authors’ opinions are and the tone in which they’ll receive those opinions. Bender and Hanna both have strong backgrounds in the subject – being, respectively, a professor of linguistics and a former Google research scientist – but they’ve also developed a reputation for publicly confronting Big Tech’s overrated bosses and ‘thought leaders’ in debate. The writing reflects this experience of having to take on the constant propaganda noise that comes off the AI industry. They favour a snarky, wry vibe over a strictly academic one.

Just in case the title didn’t do it for you, Bender and Hanna set out their thoughts about the use of AI in our society very clearly: it is currently being used for fraud, racist policing, harassment of women and the Palestinian genocide. They don’t reject the idea that the technologies behind AI can, and should, be used for positive things, but that’s not what’s happening now so there ought to be resistance.

False promises everywhere

The authors’ critique of the technology involved isn’t necessarily one you haven’t heard before, but it is well explained and usefully argued. Like many critics, me included, they strongly dislike the term ‘AI’ because it is so fundamentally misleading. They do feel that they have to use it because the marketing behind the term is so pervasive, but they draw the line at other terms that encourage bad analogies. They explicitly reject, for instance, using the word ‘hallucination’ to describe AI generating statements about things that aren’t true: the term implies that AI should know what truth and untruth are, which is fully impossible. It is a misunderstanding of how the tech works to believe otherwise.

The negative effects of AI as described in the book are broken down broadly into those that harm people doing work, and those that harm people trying to use the products and services. On the production side, the critical thing is that AI tools are used by capitalists for increasing exploitation, which is achieved at multiple levels of the work process. At one end, people who were previously paid to produce original work are having that work stolen from them by AIs, to be repackaged as if they were original products, which is simple theft. At the opposite end, a service worker is expected to use AI tools to churn out increasingly crappy output that they have less and less ability to control. There’s also a layer in between that is intentionally kept obscure: low-paid workers in the Global South (or Majority World, in the authors’ preferred terminology) who must toil unseen to smooth off the defects of AI functionality and suffer the horror of cleaning obscene and unacceptable content out of them.

On the service-user end, Bender and Hanna identify a range of scenarios in which AI is entrusted with making decisions that can cause active harm. In some cases, the machines have simply been given something directly dangerous to do: a ‘self-driving car’ crashing is an example, as is a chatbot encouraging a depressed person to take their own life. More insidious is the use of AIs to implement policies that appear to be purely ‘data-driven’ but actually reinforce prejudice and oppression.

In the USA, the racial dimension of AI-based surveillance has been both inescapable and shocking, not least because it is simply a new vector for a very old social ill. It has long been the case that the marginalisation and poverty of non-white people in Western societies create a ‘data-driven’ justification for heavy policing, heavy incarceration and the breaking up of families. These things all exacerbate their marginalisation, and now the AI can do it even more efficiently and with the blessing of liberal mayors in cities like New York and Los Angeles.

An example of worker exploitation and oppressive surveillance coming together, of course, has been the use of ‘self-driving’ taxis to obtain camera footage from city streets and pass it to racist police. This book came out just too soon to see the citizens of Los Angeles burning out such taxis as part of their rebellion against Donald Trump’s thuggish immigration enforcers. I doubt they are surprised about it, though, as they do report on some novel resistance to such vehicles in cities across the West Coast.

Back to school

An area where I was quite glad to see the authors use a bit of nuance was in education. A lot of the more critical coverage about AI that we’ve seen in the last few months has been focused on somewhat alarmist headlines like ‘AI is making young people stupider!’, which frankly start to look more like moral panic than serious analysis to me.

Bender and Hanna do agree that large-language models have hit education like ‘a bombshell’, but they qualify this by saying that serious research doesn’t support a vision of Generation Alpha all being lazy ne’er-do-wells. They are far more critical of educators getting extremely paranoid about cheating, and squandering resources on highly dubious ‘AI detection’ products which are, inevitably, supplied by the same Big Tech companies as those models. At the same time as crying about students over-using AI, education institutions are also amongst the many sectors trying to use the exact same products to force staff to do more with less.

Although this is a book about technology, not education, I do think during this part of the discussion, the authors are edging towards an interesting debate about education and schooling which I would like to see expanded further. They refer British student protests in 2020 – over the use of ‘algorithms’ to simulate the results of exams that never took place because of Covid-19 – and cite them an example of resistance to institutionalised discrimination through technology. I think there’s more than a little insight in there, but it needs us also to debate the reality of exams and other features of education which have always been discriminatory.

A history of debating the future

Something this book does, that you probably haven’t seen elsewhere, is analyse the ideological rhetoric surrounding AI and the historical roots that these ideologies draw upon. Up front, the writers assert that they believe that the racial bias problem in much of AI isn’t simply unthinking prejudice, but that an inherent racism informs a lot of the thinking in Big Tech. They further argue that racism is a component of wider reaction in the whole sector. The evidence for this is that the tech boosters’ definition of intelligence (and their tools for supposedly measuring intelligence) are deeply rooted in the discredited thinking that surrounds ‘intelligence quotient’ (IQ), which is in-turn strongly influenced by the pseudoscience of eugenics. Eugenicism was a supposed quest to create superior human beings through directed breeding and technocracy, that justified both apartheid and genocide. The fact that AI advocates have normally (until they recently needed to cosy up to Donald Trump’s election victory) tended to present themselves as modern and liberal, rather than racist, isn’t something we get to treat as a surprise: eugenicists often presented a modernist and progressive face.

The belief systems that tech bosses advocate are normally either ignored or taken seriously in the media, despite the fact that you don’t need to scrutinise them for very long to see that they are bizarre. Studying the weirdness yourself can be enough to make your head hurt, so The AI Con is doing a real service by digesting the chaotic blur of propaganda and buzz phrases for you. Critics often summarise the Silicon Valley ideology as ‘Transhumanism, Extropianism, Singularism, Cosmism, Rationalism, Effective Accelerationism and Longtermism’ (TESCREAL). Bender and Hanna do perform a bit of jargon-busting on what those words mean but emphasise that the supposed definitions aren’t terribly important: what matters is that the first four terms are extraordinary claims (without the support of extraordinary evidence) and the last three are outrageous conclusions from those claims, which serve only to justify criminal behaviour.

Unpacking the absurd dogmas of Big Tech can be quite funny: their claims that generative AI can further science, for example, betray a distinct lack of clarity about what science even is! However, it is also very worrying to see how morally blank a lot of the decision making that is informed by TESCREAL thought is. Convincing yourself that your scheme for obtaining a billion-dollar fortune is actually a mission to usher future generations of enlightened human beings into a glorious future is a great way to justify to yourself such acts as driving people into poverty and inflicting ecological damage upon the world.

On the more positive side, however, the authors also explore the much lesser-known pedigree of left-wing critiques of high technology and its scope for abuse under capitalism. I must confess, that prior to reading this book, I was not aware of the work of Joseph Weizenbaum. An advanced mathematician active in the post-World-War-Two years, Weizenbaum helped to create one of the first machines that could identify language from printed writing. From here, he made one of the earliest examples of a ‘chatbot’, by applying pattern recognition rules to language that was inputted to the machine. It could then respond with a suitably generated word pattern. To him, this was a nifty demonstration of complex data processing.

He soon realised, however, that other people – including well educated people – found the device to be so persuasive that they were convinced that it was sincerely responding to their words, even that it was alive. This led Weizenbaum to write extensively about how this was an illusion, and that there were deep risks inherent in believing that illusion. He effectively became the first true critic of AI and was strongly opposed to the idea offloading consequential decision-making onto any sort of automated system. Additionally, he was a staunch anti-militarist, correctly predicting the dark future that would be brought about by automating machines that kill.

Don’t Panic, Organise

Bender and Hanna strongly believe that it is both possible and necessary for ordinary people to change the direction in which technology is travelling. They write enthusiastically about workers having successful industrial struggles over questions of automation and high technology. Obviously, you start with the crucial 2023 Hollywood actors’ and writers’ strike – a breakthrough mass fightback against AI in the workplace – but they also bring to the reader’s attention that there is a long history here. American car workers and miners had major industrial battles against exploitation through automation back in the 1950s. The book goes back even further to industrial revolution-era England to argue, much as Jathan Sadowski does in his recent work, that the Luddites should be understood as a working-class resistance movement rather than a reaction against technology.

Bringing things back to the present day, there are some really intriguing examples of workers coming up with innovative ways to organise. These include the development of content moderators’ unions in the Global South, organising to protect the sanity of the people who are put through the psychological wringer of keeping unacceptable material out of the public internet. Elsewhere, graphic artists – a group who historically lack trade-union organisation – have been teaming up with engineers to come up with means to prevent generative visual models from ripping off their work to make slop for which they do not get paid.

The final chapter of the book is dedicated to very practical things that activists and organisers should think about and seek to do. I honestly think a lot of people – particularly if you happen to work in education or public services – will find this part quite useful. There is a personal appeal to the readers: the authors really don’t want you to use Chat-GPT or the other language models to produce pieces of work if you could simply write them yourselves. I feel they manage to avoid a moralistic argument around this, and I agree: you aren’t really gaining very much by avoiding producing your own work.

The majority of the action points, however, are political and not personal. It is a set of questions to ask and demands to raise, all of which are focused on using technology to make life better rather than worse, from the level of the workplace or classroom right up to that of state legislation. If you are looking to clarify your own thoughts about this, it’s likely that you can find good advice here.

In the book’s conclusion, Bender and Hanna, like many others, fully predict that AI is a capitalist bubble that must burst one way or another. Far from this being a determinist outlook, they seek to persuade the left and the labour movement that what will follow the collapse of Big Tech in its current form has not been determined, and that we have a job on our hands to shape that outcome.