That actually did happen...

make this post go viral

Coffee With Scott Adams 11/5/25 x.com/i/broadcasts/1dRKZaXgv…

·˚ ༘ ☾ ⋆｡˚ ☄︎·˚ ༘ ☾ ⋆｡˚ ☄︎ T𝔥𝔢 L𝔞𝔰𝔱 Q𝔲𝔢𝔰𝔱𝔦𝔬𝔫 - I𝔰𝔞𝔞𝔠 A𝔰𝔦𝔪𝔬𝔳 ·˚ ༘ ☾ ⋆｡˚ ☄︎·˚ ༘ ☾ ⋆｡˚ ☄︎

CALM Turbocharges Language Models: 4x Fewer Autoregressive Steps, 34% Inference FLOPs Slash, and 44% Training Compute Savings While Matching Discrete Baselines! Large language models have long been shackled by the grind of predicting one discrete token at a time, A new paper by Tencent WeChat AI and Tsinghua University is new way to see the process. Continuous Autoregressive Language Models (CALM), is a revolutionary framework that ditches the time and energy draining next-token paradigm in favor of predicting dense, continuous vectors—each encapsulating the essence of multiple tokens in a single, fluid stroke. This shift not only compresses language into a higher-dimensional continuum but also unlocks unprecedented efficiency, slashing autoregressive generation steps by up to fourfold through a high-fidelity autoencoder that maps chunks of tokens into vectors with over 99.9% reconstruction accuracy, all while demanding a just 75 million parameters (very small). At the heart of CALM's magic lies its likelihood-free architecture, which boldly eliminates the softmax bottleneck and traditional perplexity metrics, replacing them with an energy-based transformer that trains via a strictly proper Energy Score for seamless, single-step vector forecasting. This technical leap outperforms iterative alternatives like diffusion or flow matching, delivering crisp generations without the computational drag, and introduces BrierLM, a novel, sample-based evaluation metric that correlates almost perfectly with cross-entropy at a Pearson coefficient of -0.966, ensuring unbiased assessments in the continuous domain. By harnessing semantic bandwidth as a fresh scaling lever, where increasing the chunk size K from one to four boosts information density far beyond discrete vocabularies, CALM achieves superior performance-compute trade-offs, matching the prowess of a 281-million-parameter discrete baseline with 34% fewer inference FLOPs and 44% less training compute. CALM's mastery over temperature-controlled sampling in implicit models, employing exact rejection algorithms and batch approximations to fine-tune creativity, say, at a temperature of one-third: using only black-box samplers, all while sidestepping error accumulation that plagues lesser continuous approaches. Trained on datasets like The Pile and validated on WikiText-103, it paves the way for semantically richer autoencoders and integrated architectures that could redefine AI scaling laws. I can apply this to Analog AI in memory dot product rails and turbo charge this. I need to find the time and money. I will do it soon. Paper: arxiv.org/abs/2510.27688.

Coffee With Scott Adams 11/3/25 x.com/i/broadcasts/1vOGwdgDB…