The AI compute market will all boil down to T/s (tokens per second). So if you’re trying to figure out whether to buy $NVDA or not, consider the below… When Web2.0 exploded in the late 2000s, AWS was principally responsible for arming the rebels - they offered startups compute (EC2) and storage (S3) in bite sized pieces we could pay for with a credit card. As a result a thousand flowers bloomed. This will happen for AI compute clusters. And as it does a few things should happen: 1) startups won’t care what chips are used - a100, h100, Groq etc. 2) CUDA lockin will be broken 3) foundational model diversity will emerge As all of this gets figured out, it’s generally good for incumbent revenues but generally bad for incumbent enterprise values (see Cisco c2000s). It’s also very good for emerging disrupters like @GroqInc (disclaimer: I helped get this company off the ground 8 years ago and am one of their largest external shareholders). I expect startups like @GroqInc to push the market aggressively towards super cheap T/s: they should show that they can drive massive throughout at the lowest relative cost. The cloud providers will then do the rest…
“Both companies will field very interesting offerings available in early 2024...upstarts like @GroqInc have shown advantages on certain models and workflows for LLM inference, today,” Moorhead added. #Groq outperforms NVDA in most #AI finance.yahoo.com/news/nvidi… via @YahooFinance

Aug 31, 2023 · 9:56 PM UTC

Replying to @chamath
Can someone explain this to me like I’m 5
8
2
102
So right now people are all excited about buying AI chips. But when AI chips get too expensive, someone decides to buy the AI chips and then sell a timeshare to the chips to others. This way more people can use the chips because it’s not as if the chip is used 100% of the time if only one person owns it. That is what selling in tokens/s means. $NVDA sells to Amazon and they then sell a “clock cycle” to you.
Replying to @chamath
Comparing incumbents now, to how incumbents operated back then, isn’t fair
2
1
15
Replying to @chamath
Nice job running an ad for one your sh*tcos but dressing it up as broad AI commentary 👏
9
Replying to @chamath
I hear "cuda lock in broken" every five years, yet it's still in the lead. I'm not sure I'll believe it even if it happens at this point...
2
7
Replying to @chamath
Nvidia is almost certainly overpriced at these levels, the risk to reward doesn't favor buying in the near term But Nvidia remains in many regards highly underrated with technology like Omniverse & their self driving I fully expect them to be over a $2 trillion company in time
1
4
Replying to @chamath
The difference between the AWS & Nvidia analogy is AWS was pretty commoditized hardware it was mostly business model innovation that couldn't be replicated & has been by competitors We've seen for decades that leading in chip design is extraordinarily difficult & Nvidia has managed It's unclear any competitors can push them enough to be really profitable in this. What they do is super hard & they are building their own data centers
4
Replying to @chamath
agree completely. If we believe the use of AI/LLMs will continue to explode there are a lot of companies that stand to benefit that aren't named Nvidia. Amazon and Google feel like no-brainers at the point. Boring? yes... but sometimes boring is the right answer. Then some small bets on high-rest/high-reward disrupters along the edges.
4
Replying to @chamath
I think this has much to do with what they are trying to accomplish with AI, instead of just choosing whatever chip is available. Application specific chips differ from general purpose.
1
2
Replying to @chamath
NVDAs dividend history. iow near null. NVDA stock buy decision shouldn’t provene from sales forecast as you’ll never see a dividend. The meme is the only thing that matters.
1
Replying to @chamath
Worth flagging Groq
1
Replying to @chamath
Agree with the shift towards T/s efficiency and allocation. As the AI landscape evolves, the key will be a mix of hardware diversity and software innovation,
1
Replying to @chamath
makes sense as the next logical step
1
Replying to @chamath
We can’t ignore energy efficiency of the hardware. TOPS per Watt (or Tera Operations Per Second per Watt) is better (not saying perfect) when comparing AI compute
1
Replying to @chamath
T/s is what it's all about! With the global shortage of AI chips, it's been especially difficult for startup founders to get access to them. So its super important to have companies driving down price of T/s, so we can then help with distribution to startups at a more affordable price.
Replying to @chamath
What about if you bought $NVDA 5 years ago? Sell or hold?
RNDR / OTOY fits so well here, isn’t it?
Replying to @chamath
@chamath what did you mean when you referred to “cheaper abstracted hardware wrappings” in the last episode?
This tweet is unavailable
Replying to @chamath
You have zero credibility left. Thanks for playing.
1
9
Replying to @chamath
Chamath, We know how familiar you are with @rendernetwork . Decentralized Distributed Compute is the only way #RNDR
8
Replying to @chamath
You guys spoke about this on the pod and I think you’re correct, competition will drive the price down, nvidia is the king for now, but soon ARM and others will probably challenge them for market share
3
5
Replying to @chamath
TLDR for those with no patience or ability to read... Chamath is saying the time to buy $NVDA has long since come and gone. There is money to be made in AI obviously. But the obvious play (NVDA) ain't it..
3
3
Replying to @chamath
💯 1) sell work not software (inspired by @sarahtavel) 2) @Modular_AI will break CUDA, a swarm of Pythonian is coming to Mojo 🔥 3) foundational model is the new Linux, not to say there won't be a place for windows (@OpenAI)
2
Replying to @chamath
This is what @rendernetwork is doing the “airbnb” of GPUs, right?
2
Replying to @chamath
Okay, but also.. The unit of work and workload are very different this time. I wonder what Instagram, at say 1,000,000 users, spent (dollars) on compute per month (back then) vs what a million users of <whizBang>GPT will cost per month (next month, 9/24). Selecting Instagram because pictures are between webpage text and video (size-wise).
1
2
Replying to @chamath
This is basically what @akashnet_ has created and has been doing for over 4 years. GPU leasing marketplace, but I'm sure you already know that @chamath
2
Replying to @chamath
I looked at the specs, 230 MB SRAM is not enough for LLMs and the DRAM bandwidth isn't mentioned.
1
Replying to @chamath
how about betting on the actual innovator, rather than the chip provider; tesla (and perhaps zm), about to have its chatGPT moment, is the safe (and most rewarding) bet, imho
2
1
Replying to @chamath
Let me guess, another spac. 🤪
1
Replying to @chamath
The democratization of technology, whether it was web services in the 2000s or potential AI compute in the future, often leads to rapid innovation and reduced reliance on incumbent providers. If AI compute follows a similar path to web hosting, we might see a surge in AI startups and a shift in market dynamics. Breaking the lock-in of proprietary technologies, like NVIDIA's CUDA, can foster competition, driving both innovation and cost-efficiency. This pattern has been observed in various tech sectors over the years.
1
Replying to @chamath
sounds like someone’s in the arena
1
Replying to @chamath
Trying to push your own bags up again?
1