TL;DR:
- AI is excellent at generating output, but that does not mean it understands what it is doing.
- When humans accept AI output without understanding it, they accumulate understanding debt.
- Thinking can be outsourced as labor, but for real judgment you need understanding of the underlying concepts.
- The practical rule is simple: do not ship, approve, or build systems whose concepts you don’t understand.
AI automates & speeds up labor
Nowadays, AI can draft code, summarize information, and produce reports quickly. Agents built on powerful AI models make parts of knowledge work cheaper and faster, especially the repetitive parts. Step by step, this will change how we do white-collar work.
Output is not understanding
That convenience creates a risk. AI systems can produce plausible output that seems correct on the surface, but actually does not make sense in the broader context of the system you want to build. When we hand off a problem to a model and accept the result without checking the reasoning, we accumulate understanding debt.
AI tools do not understand a problem in the way a person with context does. They can head in the wrong direction, follow a flawed strategy, or invent details. If the human operator cannot explain why an output makes sense, the system becomes hard to steer. You may still move fast, but with less control over where you are going.
How do we stay in control?
The clearest answer I have seen comes from a mental model popularized by Andrej Karpathy: “You can outsource your thinking, but you can’t outsource your understanding.”
To work well with AI, we need to draw a sharp line between the two:
- Thinking is labor: drafting, summarizing, calculating, listing options, and writing boilerplate. AI can help with that.
- Understanding is the mental model behind that labor: judgment and strategic direction. That part still has to stay with the human.
In practice, this changes the role of the user. The new job shifts toward directing, evaluating, and verifying what gets produced instead of producing every artifact manually. AI can generate a first pass quickly, but conceptual understanding and strategic direction still has to stay in human hands. At the same time, it stays true that the potential productivity of a human has increased by a lot.
But there is a difference between using AI as a tool and relying on it blindly. If you keep hold of an understanding of the underlying concepts, AI can save time. If you give that up, you are handing off judgment along with the labor.
Andrej Karpathy discusses the shift to agentic workflows
This conversation explores how people can use increasingly capable systems without giving up control over the reasoning behind the work.