I genuinely, sincerely believe this is an incredibly important tweet in understanding how some people think about AI.
cool use case of chatgpt work i heard last night:
connect your family calendars and explain your kids’ interests.
every morning for the drive to school, have it make a podcast that talks about one kid’s soccer game that afternoon, one kid’s upcoming birthday, some news, etc.
The top reply says it better than I could:
What if you just talked to your children
Now, Sam Altman is the CEO of an AI company. It is very much his job to spruik the benefits of AI, and to downplay the areas where AI is irrelevant. I don’t want to pick on Sam Altman in particular here.
But there’s an attitude encoded within this tweet that is widely prevalent. You can see it in the comments of any randomly chosen Hackernews thread about AI. It goes like this:
Just consider all problems as inputs and outputs. Everything in between is process, and process is irrelevant.
In Altman’s tweet, the latent idea is that you want to know what your kids are up to. But that’s not the point. You want your kids to tell you how they are doing. The information you gain at the end isn’t anywhere near as important as the bonding experience of gaining it.
What no AI CEO would ever say is that AI cannot successfully be applied to any category of problems where process matters. And that’s a pretty big category.
It’s like thinking that the point of a holiday is to end up with pictures of yourself in front of the Eiffel Tower. And if you can generate the pictures then you don’t need the holiday. This is the “disregard-the-process” model of solving problems with AI.
Process is not always irrelevant. Sometimes it really is about the friends we make along the way.
The point of writing is not to produce written words
AI-generated writing falls for this trap. You can put a few half-baked dot points into a prompt and it will spit out a polished essay. Inputs, outputs, ignore the process.
Paul Graham venerates the writing process: “Writing about something, even something you know well, usually shows you that you didn’t know it as well as you thought. Putting ideas into words is a severe test.”
When I write, I have a vague soup of ideas in my head that I don’t truly understand. Each word I put down makes those ideas a little clearer. I’m usually targeting an imagined audience in my head, and that little bit of empathy affects every word choice.
My heuristic is this: you don’t know something until you can write about it, and you haven’t mastered something until you can teach it.
Afterwards I have words on a page, but the words are not the point. The process to get there is the point. AI writing skips this, because the disregard-the-process model only cares about the words on a page.
And oh boy, we can tell when we’re reading AI slop. The prose is like butter scraped over too much bread. Getting rid of the emdashes doesn’t help. The process isn’t optional, and AI writing skips the process.
There’s actually one case where I think AI writing is a superb fit. Bureaucratic systems love forms, and sometimes these forms are just boxes to be ticked — you just have to submit something vaguely sensible. It’s not even clear whether the output will be read. In these cases, outputs matter but process doesn’t. But use your own judgement here. I am absolutely not recommending that people use AI to fill out government forms.
Mathematics research isn’t just proofs
OpenAI recently published hundreds of ostensible proofs for open mathematics problems. This was a terrific marketing exercise that has irrevocably damaged the mathematics community and negatively impacted mathematics research.
I can hear someone asking, “How can solving open problems damage mathematics research?” Because proofs are mere outputs of mathematics. Everything that goes into making a proof is what sustains a community of mathematics research. Again, the disregard-the-process mindset fails.
One might reasonably claim that the proof of the Four Colour Theorem is an example of a proof that skipped the proof-generating process. Computer-validated proofs are not new, and I used them myself in my PhD thesis. But AI represents a wholesale disconnect of proofs from process.
Terence Tao describes the use of AI in this manner as mining open problems, in the sense that a resource can only ever be mined once. Terence Tao’s views on AI, as well as those of his guest posters, are nuanced and deserve to be read directly.
The gist is this: whenever an open problem is solved, a proof is produced. As part of the process of generating that proof, new ideas and techniques are discovered, sometimes even new fields of mathematics. For substantial contributions, authors will be asked to present seminars and lectures on their discoveries. New problems are discovered. Some problems are on the easier side, and assigned to students. The proof is the output. The process of creating that proof is more important. It generates new mathematics and sustains the mathematical community.
There is a core idea here that is almost certainly true in any scientific field: the individual is nothing without the community. The mathematics community solves problems. Proofs by themselves do not build community, and strip mining open problems harms it.
Andrew Wiles famously worked in near-secrecy to prove Fermat’s Last Theorem, but he did not work in isolation. The technique of lifting was built on existing work. And the flaw in the original proof was corrected in collaboration with Richard Taylor.
OpenAI’s contributions skipped the process and went straight to proof. There is no author who will give seminars on how they discovered those solutions. Those open problems are gone forever.
I took my PhD in mathematics and while I quickly moved to industry, I have a lot of love for the field. Anecdotally, I can say that research mathematicians have started withholding open problems from their publications for fear of being “scooped”. Journal editors have had their submissions multiplied, and most of it is incomprehensible AI slop.
That is to say, OpenAI (and AI more broadly) have had a net negative impact on mathematics research. It is not yet clear if AI can produce enough positives to offset this. Perhaps LLMs will be able to generate new, interesting problems in mathematics and solve them without the involvement of humans. Perhaps “centaur” mathematicians using AI will create a new wave of mathematics research. Predicting the future capabilities of AI is difficult. Currently, the score is negative.
The “spikiness” of AI applications
Salesforce released a bizarre advertisement for its AI tools, starring Matthew McConaughey and Woody Harrelson. Matthew McConaughey dines outside in a storm, eating a meal he doesn’t want to eat, because the booking app he used didn’t have Salesforce’s AI platform.
Speaking with all the experience of someone who has been to at least one restaurant in his life, I would like to inform the Salesforce marketing team that this is not how restaurants work. They don’t seat you outside in a storm. Most will present you with a piece of paper containing a list of meal options, colloquially known as a “menu”. If Matthew McConaughey doesn’t like prawns, he has every opportunity to order something else. If he insists on ordering prawns, blame him, don’t blame the lack of AI.
Incidentally, this isn’t the only time I’ve seen people propose using AI to solve the “problem” of à la carte, and it’s another case of the disregard-the-process mindset. Choosing what to order from a menu is genuinely a fun experience! I don’t want to skip that.
Let’s assume, given the star power involved, that significant marketing resources went into this ad. Probably an experienced, expensive marketing firm or in-house team. Now, it’s not uncommon for advertisers to invent problems for their products to solve (see, for example, shame around underarm sweat and deodorant). But why couldn’t this high-powered team at least come up with vaguely coherent problems for AI to solve?
The applicability of AI is spiky. It’s incomprehensible that anyone could deny that AI is good at writing code. But outside of tech, it’s not showing a whole lot of promise.
A simple explanation could be that it’s still early days, and the tech industry is the best-positioned to take advantage of its own technology. Or perhaps coding, being about formally defined languages with some verification possible, falls to AI first.
But I propose the following contributing factor: AI can only solve problems where:
- The inputs are tokenisable
- The outputs are tokenisable
- The process is either not important, or consciously discarded
My own most useful applications of AI fit this model. I’ve used it to generate metadata, for example. Very useful (and very tedious), and the output matters more than the process.
The process does matter for coding — hand-writing code gives you an intuition and understanding of your codebase that a coding assistant never will. But we can consciously decide to abandon that in favour of the outputs.
I’ll say this here and now: I’m a better code with AI than I was before.
But the disregard-the-process mindset fails for a large number of problems. One of the promised use cases for Microsoft’s Copilot was AI-generated emails, and now we treat AI-generated corporate communication with revulsion. It turns out that communicating with your colleagues requires thinking through what you want to say and how you want to say it, and AI can’t help you skip that process.
AI-generated communication worsens the situation because it displaces what could have been a well-thought-out email, Slack message, or presentation. I’ve never once worked in an organisation that had a problem with “too much quality communication”. It sounds like a wonderful problem to have, but I suspect no such company exists. AI is solving this problem that no one has.
Don’t disregard the process
If you’re a software engineer like me, you’re probably used to the concept of black-boxing. The API endpoint returns what it returns, and our job is to make sure that the user doesn’t need to think about how it returns that. It’s part of the job, and I can see how that mindset would leak into aspects of life outside of an IDE. But every time you come up with a new application for AI, just take a moment to ask yourself: is there something more to this than just the outputs?
And if you’ve made it this far, here’s my final plea: take a moment today to ask someone you care about how their day went, and truly listen to the answer.