it’s over turns out the rl victory lap was premature. new tsinghua paper quietly shows the fancy reward loops just squeeze the same tired reasoning paths the base model already knew. pass@1 goes up, sure, but the model’s world actually shrinks. feels like teaching a kid to ace flash cards and calling it wisdom. so the grand “self-improving llm” dream? basically crib notes plus a roulette wheel: keep sampling long enough and the base spits the same proofs the rl champ brags about, minus the entropy tax. it’s compression, not discovery. maybe the endgame isn’t better agents, just sharper funnels. we’ve been coaching silicon parrots to clear increasingly useless olympiad hurdles while mistaking overfit for insight. hard not to wonder if we’re half a decade into the world’s most expensive curve-fitting demo.

7 steps to launch your own vibe business (from scratch):

HUGE Immigration News! We have the first EVER look at H-1B lottery data. Did you also suspect the lottery wasn't truly random? They're not. Certain companies like Tiktok and Bytedance have 50% higher odds than average. I broke it down by nationality, company and age... 1/5

Researchers at the Beihang University in Beijing have invented a 2-centimeter-long microbot with ultrafast untethered running speeds. The research results were published in a study titled "A Wireless Controlled Robotic Insect with Ultrafast Untethered Running Speeds" in journal Nature Communications. Resembling a bug, this innovative robot has the potential for future applications in emergency rescue and mechanical inspection. Video source: Beihang University / CGTN #microrobot #locomotion #biomimicry #robotics #technology #engineering #innovation

I gave a talk at GPU MODE workshop last week on llm.c - the origin story of llm.c - being naked in the world without PyTorch and having to re-invent Array, Autograd, Device, Dtype, Compile, Distributed - how to port a PyTorch layer to 1) explicit PyTorch - and then to 2) write the backward pass - 3) port forward & backward pass to C - 4) string all the layers together - achieving one file of C with no dependencies that compiles and runs ~instantly, where all memory is pre-planned and allocated a single time, fully deterministic, portable code that can run on a potato or a von Neumann probe - how most of llm.c was built at 1am-7am in a water villa porch in Maldives and why this is the recommended way to develop software - convert all of it to run in CUDA on GPU in fp32 - port matmul to cuBLAS - port attention to cuDNN flash-attention - introduce bfloat16 mixed precision - introduce many more optimizations and features like kernel fusions, Packed128, stochastic rounding, full determinism - add multi-GPU training, NCCL, sharded optimizer - add multi-node with MPI or file system or socket - reproduce GPT-2 (1.6B) on one 8XH100 node in 24 hours for $672 in llm.c, achieving (at the time) 29% less memory, 19% faster training that PyTorch nightly, and much faster compile & run - how open source development attracts Avengers from the internet - port to training Llama 3 imminent (branch exists) - many other notable forks - last thought: how software abstractions like Python/PyTorch and everything else really exist only because humans are finite in knowledge, IQ and attention, and how with increasing AI capability LLMs may export custom binaries like llm.c for any application directly, tearing apart and refactoring all abstractions as needed. <|endoftext|> More links in reply

best song ever composed

All you need is Kolmogorov–Arnold Network! 🔥🔥🔥 complete with GitHub repo 🚀🚀🚀🚀🚀 'KAN: Kolmogorov–Arnold Networks' from @MIT and @Caltech h/t @illumattnati

Build a LLM app with RAG to chat with your Gmail Inbox in just 30 lines of Python Code (step-by-step instructions):

Never underestimate the open source AI community. These cracked engineers are here to break the limits of what’s possible with local LLMs. We just witnessed some nutty inventions. Here’s what we saw at the @ollama Open Source and Local AI meetup at @cerebral_valley (🧵):

Have you ever wanted to train LLMs in pure C without 245MB of PyTorch and 107MB of cPython? No? Well now you can! With llm.c: github.com/karpathy/llm.c To start, implements GPT-2 training on CPU/fp32 in only ~1,000 lines of clean code. It compiles and runs instantly, and exactly matches the PyTorch reference implementation. I chose GPT-2 to start because it is the grand-daddy of LLMs, the first time the LLM stack was put together in a recognizably modern form, and with model weights available.

> Locally running Perplexity clone > The big secret about perplexity is they havent done much beyond using off the shelf models github.com/nilsherzig/LLocal… from hn

Early results from my AI training runs. I've trained my $200 robot arm on a simple picking task using imitation learning. It has learned to control the robot arm using only camera images and joint states.

Introducing a “nonlinear equivalent” of LQR controllers! ⚙️ CSAIL researchers synthesize approximate optimal controllers for nonlinear dynamical systems w/optimality & stability guarantees. To achieve this, their work finds both under-and over-approximations of the value function via sums-of-squares. A thread 🧵

Paper: rb.gy/9zg635 Code: rb.gy/ew58oh Video: rb.gy/29w26b

Save Bandipur Tiger Reserve: Stop the Relaxation of Night Traffic Ban and Railway Project - Sign the Petition! chng.it/fbvN6ffn via @ChangeOrg_India

My second hand redmi note 9 pro running flowpilot is driving my alto k10 😂Can it get more desi than this ? #flowpilot #openpilot #ai #robotics #autonomous #cars #Android

We invented Q* first Glad openai is building on top of our idea

My supervisor Rich Turner has written an introduction to transformers. I highly recommend anyone wanting to learn (more) about transformers to take a look, it is precise and concise at the same time. arxiv.org/abs/2304.10557

Conformal Prediction With Conditional Guarantees Excited to share my work with John Cherian (@jjcherian) and Emmanuel Candès arxiv.org/abs/2305.12616. It is well-known that pointwise conditional coverage is impossible without strong assumptions. But how close can we get? (1/6)

So I set up BakLLaVA-1 in the llama.cpp, and now it can provide real-time descriptions of the live feed from my camera check it out! open source? cc: @nisten @thursdai_pod @willdepue #llama