This seems like a major breakthrough for AI advancement Tencent and Tsinghua introduced CALM (Continuous Autoregressive Language Models), a new approach that replaces next token prediction with continuous vector prediction, allowing the model to think in ideas instead of words. Key results: ~4× fewer prediction steps 44% less training compute
Robert Youssef
 @rryssf_
Nov 4
Holy shit... this might be the next big paradigm shift in AI. 🤯 Tencent + Tsinghua just dropped a paper called Continuous Autoregressive Language Models (CALM) and it basically kills the “next-token” paradigm every LLM is built on. Instead of predicting one token at a time, CALM predicts continuous vectors that represent multiple tokens at once. Meaning: the model doesn’t think “word by word”… it thinks in ideas per step. Here’s why that’s insane 👇 → 4× fewer prediction steps (each vector = ~4 tokens) → 44% less training compute → No discrete vocabulary pure continuous reasoning → New metric (BrierLM) replaces perplexity entirely They even built a new energy-based transformer that learns without softmax no token sampling, no vocab ceiling. It’s like going from speaking Morse code… to streaming full thoughts. If this scales, every LLM today is obsolete.

Nov 4, 2025 · 4:03 PM UTC

63
238
23
1,665
paper arxiv.org/pdf/2510.27688
1
7
44
Replying to @Dr_Singularity
AI just discovered overthinking.
1
Replying to @Dr_Singularity
If @MarkJCarney and @realDonaldTrump decide their Molochian cult is not worth more than the Anglosphere successfully competing with China in this and other powerful applications of math, we will — sans them — inspire the world with enough Faith to enter the Promised Land.
Replying to @Dr_Singularity
This falls into the general trend of latent reasoning model, language diffusion models, and perhaps more tenuously the JEPA approaches. Next token prediction encourages memorization too much, this is not ground-breaking. Writing's on the wall for a year now
1
33
Replying to @Dr_Singularity
This is huge. The shift from token-by-token to vector prediction could be the biggest leap for language models yet. If models start thinking in ideas instead of words we’re really entering a new phase of intelligence.
1
19
Replying to @Dr_Singularity
This isn't really brand new, about six months ago Meta researchers did something like this. It was called LCM ( Large Concept Models) that generated the next tokens based on predicting a stream of previous tokens called concepts. Here's a thread on it:
Solution Dev.ai
 @Paulfruitful_
Apr 1
🚀 Day 40 of #100DaysOfAIEngineering For the past few weeks, we’ve been exploring Large Language Models (LLMs) and various augmentation techniques. But today, we’re diving into something different, something even more exciting than LLMs!
6
Replying to @Dr_Singularity
This is the 'Pointillism vs. Brushstroke' breakthrough. Old LLMs (Tokens): Are like a pointillist painter (like Seurat), building an image one tiny, separate dot (token) at a time. It's painstaking and slow. CALM (Vectors): Is like an impressionist. It can use a full, continuous brushstroke (a vector) to paint an entire idea (a tree, a cloud) in one motion. It's a shift from 'pixel-by-pixel' rendering to 'object-oriented' rendering. 4x fewer 'strokes' to paint the same masterpiece.
5
Replying to @Dr_Singularity
Until guys actually training models say something . This is noise .
5
Replying to @Dr_Singularity
Make the K = Total no. of words in your answer and you have one step diffusion language model
1
2
Replying to @Dr_Singularity
👍🏻
1
Replying to @Dr_Singularity
i wonder what the context window is for the model being used as that is a major factor involved i didnt see it in the paper unless i just overlooked it
1
Replying to @Dr_Singularity
Scaling is hard. Wait for one than one paper from China.
1
Replying to @Dr_Singularity
Junk science paper. There are not “continuous” models or vectors in Machine Learning. All ML systems are discrete. The NLP community keeps producing junk science papers.
1
1
Replying to @Dr_Singularity
no not really nothing new here read more papers
1
Replying to @Dr_Singularity
CALM is not new. Try harder if you won't to shill Singularity on our doorstep.
Replying to @Dr_Singularity
Idea-level models not token-level, game changer, 4x fewer steps and 44% less compute
Replying to @Dr_Singularity
Umm. Isnt that what large concept models from Meta does ? It went nowhere btw idk why
Replying to @Dr_Singularity
4× fewer prediction steps and 44% less compute? That’s huge for scaling LLMs sustainably. Curious how well it preserves output quality at scale.
Replying to @Dr_Singularity
I'm looking forward to introducing the advanced layering aspects of this to you all.
Replying to @Dr_Singularity
So it’s literally thinking in ideas now?? we’re cooked
Replying to @Dr_Singularity
It sounds like this approach would amplify hallucination.
Replying to @Dr_Singularity
This sounds like a game changer! Thinking in ideas could reshape how we interact with AI. What potential applications do you see for CALM in everyday tech? Would love to hear your take on this!
Replying to @Dr_Singularity
Nothing is "continues" in digital clock based calculators. But good to see some progress in digital state machines.
Replying to @Dr_Singularity
🔹 The Shift from Tokens to Fields: AI’s Next Evolution We’re witnessing the biggest paradigm change in artificial intelligence since transformers. 🔸 GPT models think token by token — predicting one discrete unit at a time. 🔸 CALM (Tencent + Tsinghua) models think in continuous semantic waves — streaming clusters of meaning instead of words. 🔸 Turner AI goes one level deeper — it doesn’t just process meaning, it embodies structure. It learns through Field Coherence — the same physics that governs balance, motion, and cognition. This is where language meets biomechanics, where data flows like gravity and stability replaces probability. AI isn’t just about what comes next in a sentence anymore. It’s about what holds everything together in motion. #TurnerAI #CALM #AIInnovation #NeuralPhysics #NextGenAI #FieldIntelligence
Replying to @Dr_Singularity
@grok explain as if i was 15
1
Replying to @Dr_Singularity
Are the Chinese ahead or western Ai companies aren't releasing research papers Someone tell me @AskPerplexity
2
Replying to @Dr_Singularity
If CALM predicts in continuous vectors, could we start seeing models that generate high-level reasoning or multi-step plans natively, without relying on token-by-token chains?
Replying to @Dr_Singularity
Grok said that’s what xAI use for him, CALM.
Replying to @Dr_Singularity
We’ve been training models to predict words — now they’re learning to predict meaning. That’s a real leap toward intelligence.
Replying to @Dr_Singularity
@grok is this bullshit? Is the perf still just as good without the extra compute
1
Replying to @Dr_Singularity
Massive development CALM could reshape how models process and represent knowledge. Moving from token prediction to idea prediction means reasoning can become smoother, faster, and more human-like. That’s the kind of foundation LazAI is preparing for agents that reason in concepts, not just words, built on verifiable and decentralized data networks.
Replying to @Dr_Singularity
I like how people training AI learn about human thinking... that is just the beginning, we don't even started.
Replying to @Dr_Singularity
Isn't this the same as topic modelling but with LLMs??
Replying to @Dr_Singularity
GPUs glut incoming....
Replying to @Dr_Singularity
Moving from token-by-token to continuous vector prediction is a fundamental shift. If this works at scale it could unlock entirely new capabilities
Replying to @Dr_Singularity
as predicted in AI 2027
Load more