After a scientist tries to explain that high-dimensional matrix multiplication is beautiful in some ways, New Yorker writer doubles down -- that's interesting, but recall that matrices have TWO dimensions. Can't make it up. Open the schools!!!!
Stephen Witt
 @stephenwitt
Nov 4
Replying to @boazbaraktcs
I agree, although I believe most matrix multiplies for AI are done in two dimensions. Anyway, I stand by my statement. Matmuls are an effective piece of mathematical machinery, but the mechanics of calculating them are headache-inducing. Indeed, it's exactly their computationally cumbersome nature that's propelling the data center boom. That point seems beyond argument.

Nov 4, 2025 · 7:15 PM UTC

25
20
5
761
Respect !
Stephen Witt
 @stephenwitt
Nov 4
Replying to @stephenwitt @ben_golub
Update: I'm wrong about this.
1
2
1
72
Replying to @ben_golub
I'm right about this. Tensors are downscaled to 2-D when the operation is executed in the computer. Read about it here: developer.nvidia.com/blog/pr…

Programming Tensor Cores in CUDA 9 | NVIDIA Technical Blog

A defining feature of the new NVIDIA Volta GPU architecture is Tensor Cores, which give the NVIDIA V100 accelerator a peak throughput that is 12x the 32-bit floating point throughput of the previous…

developer.nvidia.com
6
10
I think the point of confusion is you think "high-dimensional" refers to the box/hypercube of numbers, whereas almost universally (and in particular in this conversation) a matrix is seen as a mapping between very high-dimensional spaces - and those mappings are very beautiful
1
1
1
132
Replying to @ben_golub
Big reading comp fail here. Matrix multiplies for AI are indeed executed in two-d, regardless of the number of dimensions in the software framework. The operation below is the cogwheel of capitalism.
15
2
11
Replying to @ben_golub
I don't think there's anything particularly beautiful about matrix multiplication, or regular multiplication. But matrix multiplication is used to represent complex things, and those things can be beautiful. It's like saying that typing on keyboards isn't beautiful without looking at what things you can write with them.
3
Replying to @ben_golub
To be very generous…most local properties of matrices are exhibited by 2x2 matrices.
Replying to @ben_golub
We need someone to step forward:
Lon()
 @Lon
Nov 4
Replying to @boazbaraktcs @emollick @EpochAIResearch
Who will volunteer as Tribute, go to the Capitol, and sacrifice themselves at the altar to teach the libero-corpo-journalists and politico-sock-puppets the beauty of abstract mathematical structures?
2
Replying to @ben_golub
He starts by complaining it's non-commutative, but then shifts to saying it's tedious to compute. But lots of things are tedious to compute! (square roots, large primes...) That doesn't mean the concept of primes or square roots are inelegant.
31
Replying to @ben_golub
Tensors are way too hard for jornos.
5
Replying to @ben_golub
I think they are worried that we will replace beauty-obsessed journos with truth-seeking AI writers.
Replying to @ben_golub
The only path forward is to stop this nonsense that matrices are tables of numbers. Teach linear algebra from Serge Lang’s book!
4
3
45
Replying to @ben_golub
Ben, it's 1-D. Did Newton get brainwashed by higher-order thinking that can obfuscate the Narrow Path? Or was he an Alchemist you'd reject with you proudly formalized ignorance that shortcircuits bearing direct witness to Truth as best we can through G*d? github.com/bestape/squareRoo… chatgpt.com/share/690b6730-0…
1
Replying to @ben_golub
I think there is some truth to the idea that most mathematicians would not find the deep learning particularly pleasing theoretically
1
3
Replying to @ben_golub
He's likely generating some controversy to market his book.
1
Replying to @ben_golub
Goodness, just wait until he finds out about convolutions, Fourier transforms, statistical analysis, curvefitting, multimedia compression... I have many criticisms of the AI bubble and overhyped LLMs being trained & used unethically, but "matmul is ugly" is not one of them lol
Replying to @ben_golub
I think even if he was right about this it is still a silly argument. Calculating solutions to differential equations is quite headache-inducing too, but describing a dynamical system exhibiting quite complex behavior with just a few diff. equations is quite beautiful!
8
Replying to @ben_golub
Open the asylum institutions and the reeducation camps!!
Replying to @ben_golub
Funny thing is that I agree with him in some orthogonal sense: I dislike the algorithmic part of ML and the math side is not interesting. There is beautiful software engineering, and hardware design, but at least as a matter of personal taste I don't like the ideas part.
3
Replying to @ben_golub
WTF x.com/stephenwitt/status/198…
1
10
Replying to @ben_golub
Is his nvidia book going the way of Naomi Wolf's Outrages?
Replying to @ben_golub
I'll blame Pytorch for confusing a lot of people with `matrix.ndims() == 2`
4
Replying to @ben_golub
Maybe the problem was they were thinking of the “table” view of a matrix. you can always reduce higher dimensions to 2d. (This fact has become important in the description of black holes, for ex, as well as in hardware matmul: x.com/stephenwitt/status/198… )
Stephen Witt
 @stephenwitt
Nov 4
Replying to @aidanprattewart @boazbaraktcs
Again, tensor cores do everything in 2-D read about it here: developer.nvidia.com/blog/pr…
1
Replying to @ben_golub
“Composition of permutations is ugly because it is non-commutative in general and cycle notation is kind of annoying.” —The Mathematics Dunce
1
Replying to @ben_golub
Every scalar is a matrix!