Went searching for this after seeing the post:

It's a matrix of differentiable functions that, eventually, you ultimately approximate via linear algebra. Some of these operations are multiplications.

"Just activate my function" brutal

It has almost 2x more likes than account followers, people posted lots of interesting opinions and links

it's been kind of fun watching numerical linear algebra suddenly become a hot topic, after working in the field for 20 years. check the comments, every moron is suddenly an expert in it and just regurgitating random word-associated statements they've heard elsewhere

No la ven

i don't think non-linear activation functions rebut the actual point being made

Was too lazy to post this yesterday, but universal approximation theorem relies on non-linear activation functions i.e. NNs are not *just* linear algebra.

Awkshually

matmuls or non-linearities aside, the lesson that complexity emerges from simplicity is the underlying lesson both Turing machines and deep neural networks. The lessons of emergence aren't in physics or bio, it's always been CS

the intuition that matmuls rotate and scale the space but nonlinearities cut and fold it was really illuminating for me no amount of rotation gets you to anything but a different rotation. but once you can cut and fold, hyperdimensional origami becomes possible