PTSD!

i'm scarred from the gpt-3 days

Aspect-oriented context weaves new intelligence

close your windows

Context becomes RAM when reasoning becomes real-time.

Yeah, that level of details is very helpful. @OpenAIDevs should have that in Codex CLI too.

Literally 🎯

yessir

Cool viz @giorgio_dmrch

beauty

@jicapal need this

I mean Sonnet 4.5 has a 1m context window so you’re probably fine🙃

Except that it's not uniform as opposed to RAM 😇 LLMs don't uniformly process context and it's U-shaped, plus perf degrades the more high similarity or even ambiguous needles are present.

Did you install defrag.exe yet? it can defragment your context and create extra space

💯 This is so true. Providing that context for every code review costs thousands in reasoning tokens and manual bug checks. Kluster.ai automates this in real-time, within your IDE. We handle the context so you save the tokens. 😉 Try it free: kluster.ai

Some might argue it’s always been.

so bullish on ctx engineering, on exa as well!

Great way to visualize it

Defragging context is the new defragging RAM

Or registers?

You’re genius

So what's the new solid storage hard drive for long term memory 😉 Asking cause rag ain't cutting it for me

Quick question: You posted a few weeks ago about how the h1b changes wouldn't affect American citizens chances of being hired at your company. Was that because you knew your HR department is a bunch of cat ladies who do nothing but mash the "reject without Email" button all day?

w/ memory (ai memory, not ram) and all the optimization tricks (e.g. sleep-time compute, compression) and the issues they brings (like context rot), context feels more like an hdd than ram to me.

The need for a context window and that what goes and stays in has to be managed are the most obvious evidence that LLMs are not actually intelligent

needs swap

I wish we had more control about the shape of that context. ESPECIALLY after a "summarization event". That process is rough and feels like the model (regardless of origin lab) was lobotomized. A knowledge graph, some post its, or even a sort of parallel/separate mini exchange about what is important and what should persist. Not sure if an onion, or a tree is the better paradigm, but AI RAM is in short supply.

Benchmarking models at mongodb hackathon yesterday was fun. I filled ups each model from 0 to 10 M token then stress tested them. Very fun

how does one visualize context usage like that?

great reference to the new trends