AI and the Future of Work

Introduction to A Multi-part Series

A few weeks ago in Singapore, at an in-person meetup for my Coursera data science online learners (as well as some keen Duke alumni), I gave a talk on AI and the Future of Work.

I presented an informal overview of the whole history of AI, focusing on fundamental theoretical limits on what digital computers can do, and how this constrains their ability to replace us.

I explained why I believe that these unbridgeable limits will leave the vast majority of human work untouched.

Depending on your point of view, my realism may have been reassuring or deflating, but in any case the current hype around AI, and in particular Artificial General Intelligence, or AGI, is at an all-time high and my counter-message clearly resonated with the audience.

I repeated the talk online the next week for people not in Singapore who had expressed interest, and then several of my former students asked me to post it or publish it, as they wanted to share it with others.

Granting their wish has caused me some trepidation. Anyone who has ever reviewed the written transcript of a spoken talk will know what I'm talking about. I've been studying, building, and teaching AI systems for thirty-two years, so naturally I have developed many opinions that are easy to state, but harder to justify. Filling in reasons and evidence, sourcing academic authorities, anticipating and rebutting superficially appealing objections and so on, to convert this informal talk into persuasive argument would require rewriting an already 70-page transcript as a book-length document.

So, apologizing in advance for the lack of rigor, I will just lay out my conclusions and my basic reasons for them, and write the book later.

I will be posting the written version of AI and the Future of Work as a series over the next couple weeks.

Here's what to expect:

(1) "Experts" Versus "Networks"

There is much insight to be gained into the history and future prospects of AI by studying it as a rivalry between two largely incompatible schools of thought, here called "Expert Systems" and "Neural Networks." I explain what I mean by these contrasting engineering and theoretical approaches. Each has strengths, blind spots, and well-understood theoretical limits. Neither can achieve AGI on its own. An intentional rapprochement - a synthesis of the two - is needed to achieve further progress.

(2) Mathematical Limits of Large Language Models

Large Language Models (LLMs) have shown amazing capabilities in the last three years, but they deal in probabilities, not certainties. The methods they use to store the results of their training prevent them from ever giving answers guaranteed correct. As a result, nothing they generate on their own will be considered final and definitive, and no one will trust them.

Neural Networks can be built that give guaranteed answers to certain questions within narrower domains, but not when applied to generic language tasks through the LLM architecture.

(3) Today's Commercial LLMs are Swaddled in Band-Aids

The spectacular success of Chat-GPT and similar generative transformer systems, starting in 2022, inevitably led to attempts to extend their range and use to tasks for which they are fundamentally unsuited.

Commercial ventures driving the field have no incentive to acknowledge limits to LLM architecture. Instead, companies began tacking on more and more ad hoc solutions to address various failings of their LLMs, while obscuring their non-LLM structure, as if they were merely fixing bugs in an otherwise omniscient system. But the problems with LLMs are fundamental. The Band-Aid approach is nearly certain not to lead to further performance breakthroughs.

(4) Achieving Artificial General Intelligence (AGI)

AGI can be a useful goal and a meaningful concept only if it the term is more carefully defined. I put forward a definition that is limited to the potential capabilities of binary digital computers, while excluding theoretical future machines with wholly different substrates and architectures.

(5) Universal Turing Machines (UTMs).

All digital binary computers, past, present, and future, are Universal Turing Machines or UTMs. UTM are defined in a mathematically rigorous way. If we clarify what UTMs can and cannot do, we can demystify much of the discussion around AGI and its future impact on humanity.

(6) Human Work

Most of the meaningful work that human beings do can never be done by any Universal Turing Machine. Therefore, the overall economic impact and social disruption that AGI can bring is much less dramatic than people fear.

The tendency to equate work with production ignores most of what real work actually is.

(7) Human Freedom

Free people will never voluntarily accept the direct and final authority of AGI in a very large number of human domains:

allocation of scarce resources (such as acceptances to competitive Universities or graduate programs);

issuing governmental permissions like licenses and permits, or determining who is eligible for special privileges like social security disability payments;

calculating what an individual owes in taxes, fees, fines, or in compensation for liability;

choosing who will be drafted to fight in wars and who exempted; or

granting or withholding bail and sentencing in criminal cases.

Nor should they. Absent coercion, insistence on human accountability will limit AI's ability to replace human decision-makers.

(8) Big Changes in Work and Employment

For about the past three hundred years, rapid adoption of new technologies has frequently changed the nature of work.

The main lesson of past episodes is that while both skilled and unskilled workers are displaced, both types of workers will be re-employed in other roles, often of greater value, and relatively quickly. Technology that makes large amounts of work unnecessary does not lead to mass unemployment. The resulting economic changes are far less destructive and more beneficial than one might expect.

And, as discussed above in Parts 6 and 7, the work any AGI can legitimately displace is quite a small part of current human work.

Ok, that’s it. Part (1) later this week.