Andrej Karpathy says you should learn AI depthwise, not breadthwise.
Most education is breadthwise: watch lectures, memorize formulas, and trust you'll need it later.
Karpathy flips this by learning "depthwise, on demand."
What this means:
Pick a project, start building, and learn exactly when you hit a wall.
When he created a tutorial on transformers (the architecture behind ChatGPT), he didn't start by explaining attention mechanisms or complex architectures.
Instead, he started with the simplest possible thing: a lookup table that predicts the next word.
You build that first. Then you try to make it handle more complex patterns.
And it breaks.
Only then, when you've felt the limitation, does he introduce the next concept. Each piece solves a problem you've actually encountered.
As he puts it:
"It's a dick move to present the solution before I give you a shot to try it yourself."
When you attempt the problem first, the solution actually makes sense.
Teaching forces you to learn. "If I don't really understand something, I can't explain it."
When you try to explain and stumble, you've found the gaps in your understanding.
...
Build a project that gives you a reward.
Hit a wall. Learn just enough to solve it. Then explain it to someone else.
Don't consume content. Build the code.
That's how you actually learn.