Digital Geometer, Assoc. Prof. of Computer Science & Robotics @CarnegieMellon @SCSatCMU and member of the @GeomCollective. There are four lights.
Pittsburgh, PA
Joined January 2008
- Tweets 5,049
- Following 487
- Followers 37,900
- Likes 10,178
Pinned Tweet
I work on fundamental algorithms for geometric and visual computing. Here's a taste of our group's work, as a list of "explainer" threads posted on Twitter/X over the years. (🧵)
For a whole lot more, check out:
cs.cmu.edu/~kmcrane/
geometry.cs.cmu.edu/
Fantastic video from @SciShow about our work that turns any shape into fair dice:
piped.video/-gp7AbYD9NI?si=4pA6…
Get all the details on Hossein Baktash’s page here: hbaktash.github.io/
“Fair dice” might make you think of perfect cubes with equal frequencies (say, 1/6 on all sides) 🎲
But “fair” really just means you get the frequencies you expect (say, 1/4, 1/4 & 1/2)
We can now design fair dice with any frequencies—and any shape! 🐉
hbaktash.github.io/projects/…
Nicely produced clip by Matt Wein and Marylee Williams about our recent dice design project at @SCSatCMU and @AdobeResearch
piped.video/shorts/jD0agpC2V…
🎲 🎥 🐉 🪙
I got tired of mashing together tools to write long threads with 𝐫𝐢𝐜𝐡 𝑓𝑜𝑟𝑚𝑎𝑡𝑡𝑖𝑛𝑔 and ℳα†ℏ—so I wrote La𝑇𝑤𝑒𝑒𝑡!
It converts Markdown and LaTeX to Unicode that can be used in “tweets”, and automatically splits long threads. Try it out!
keenancrane.github.io/LaTwee…
P.S. I should also mention that these diagrams were significantly improved via feedback from many folks from here and elsewhere.
Hopefully they account for some of the gripes—if not, I'm ready for the next batch! 😉
Of course, there will be those who say that the representation diagram is “obvious,” and “that's what everyone has in their head anyway.”
If so… good for you! If not, I hope this alternative picture provides some useful insight as you hack in this space.
[End 🧵]
If you want to use or repurpose these diagrams, the source files (as PDF) can be found at
cs.cmu.edu/~kmcrane/Autoenco…
Likewise, here's a simpler “implementation” diagram, that still retains the most important idea of an *auto*-encoder, namely, that you're comparing the output against *itself*.
Personally, I find both of these diagrams a little bit crowded—here's a simpler “representation” diagram, with fewer annotations (that might anyway be better explained in accompanying text).
Finally, a natural question raised by this picture is: how do I sample/generate new latents z? For a “vanilla” autoencoder, there's no simple a priori description of the high-density regions.
This situation motivates *variational* autoencoders (which are a whole other story…).
It should also be clear that, unless the reconstruction loss is exactly zero, the learned manifold M only approximates (rather than interpolates) the given data. For instance, x does not sit on M, even though x̂ does.
(If M does interpolate all xᵢ, you're probably overfitting)
Another thing made clear by this picture is that, no matter what the true dimension of the data might be, the manifold M predicted by the decoder generically has the same dimension as the latent space: it's the image of R^k under g.
So, the latent dimension is itself a prior.
In regions where we don't have many samples, the decoder g isn't reliable: we're basically extrapolating (i.e., guessing) what the true data manifold looks like.
The diagram suggests this idea by “cutting off” the manifold—but in reality there’s no clear, hard cutoff.
Here's a way of visualizing the maps *defined by* an autoencoder.
The encoder f maps high-dimensional data x to low-dimensional latents z. The decoder tries to map z back to x. We *always* learn a k-dimensional submanifold M, which is reliable only where we have many samples z.
This picture is great if you want to simply close your eyes and implement something.
But suppose your implementation doesn't work—or you want to squeeze more performance out of your data.
Is there another picture that helps you think about what's going on?
(Obviously: yes!)
With autoencoders, the first (and last) picture we see often looks like this one: a network architecture diagram, where inputs get “compressed”, then decoded.
If we're lucky, someone bothers to draw arrows that illustrate the main point: we want the output to look like the input!
A similar thing happens when (many) people learn linear algebra:
They confuse the representation (matrices) with the objects represented by those matrices (linear maps… or is it a quadratic form?)
“Everyone knows” what an autoencoder is… but there's an important complementary picture missing from most introductory material.
In short: we emphasize how autoencoders are implemented—but not always what they represent (and some of the implications of that representation).🧵
P.S. I'm sure there will also be people who comment, "But that's not the standard diagram for an autoencoder, it's actually…" Yes, please show me the better diagrams people are using!
I chose this as the "standard" one because it appears on Wikipedia: en.wikipedia.org/wiki/Autoen…
I'm sure there are people who will comment, "Well actually, I prefer the top image because…" (and I'm very interested to hear those reasons).
But for anyone interested in using the bottom image, I've put a PDF version here (CC0 1.0 Universal): cs.cmu.edu/~kmcrane/Autoenco…
Enjoy!