The AI compute market will all boil down to T/s (tokens per second).
So if you’re trying to figure out whether to buy $NVDA or not, consider the below…
When Web2.0 exploded in the late 2000s, AWS was principally responsible for arming the rebels - they offered startups compute (EC2) and storage (S3) in bite sized pieces we could pay for with a credit card. As a result a thousand flowers bloomed.
This will happen for AI compute clusters. And as it does a few things should happen:
1) startups won’t care what chips are used - a100, h100, Groq etc.
2) CUDA lockin will be broken
3) foundational model diversity will emerge
As all of this gets figured out, it’s generally good for incumbent revenues but generally bad for incumbent enterprise values (see Cisco c2000s). It’s also very good for emerging disrupters like @GroqInc (disclaimer: I helped get this company off the ground 8 years ago and am one of their largest external shareholders).
I expect startups like @GroqInc to push the market aggressively towards super cheap T/s: they should show that they can drive massive throughout at the lowest relative cost.
The cloud providers will then do the rest…
“Both companies will field very interesting offerings available in early 2024...upstarts like @GroqInc have shown advantages on certain models and workflows for LLM inference, today,” Moorhead added.
#Groq outperforms NVDA in most #AI
finance.yahoo.com/news/nvidi… via @YahooFinance
Aug 31, 2023 · 9:56 PM UTC

































