So proud of our team! Math and coding remain critical to our mission. Especially proud of our work to land new SOTA in Humanity's Last Exam: 34.8%. A +13% boost: no tools, just an intelligent base model and reasoning capabilities. blog.google/products/gemini/…

A cheekier version

I departed Google DeepMind after 8 years. So many fond memories—from early foundational papers in Google Brain (w/ @noamshazeer @ashvaswani @lukaszkaiser on Image Transformer, Tensor2Tensor, Mesh TensorFlow) to lead Gemini posttraining evals to catch up & launch in 100 days, then leading the team to leapfrog to LMArena #1 (and stay there for over a year!), and finally working on the incredible reasoning innovations for Gemini’s IMO & ICPC gold medals (w/ @HengTze @quocleix). Gemini has been a wild journey from one paradigm to another: first, revamping our LaMDA model (the first instruction-like chatbot!) from an actual chatbot to long contentful responses with RLHF; then, reasoning and deep thinking by training over long thinking chains, novel environments, and reward heads. When we first started, public sentiment was bad. Everyone thought Google was doomed to fail due to its search legacy and organizational politics. Now, Gemini is consistently #1 in user preference and spearheading new scientific accomplishments, and everyone thinks Google winning is obvious. 😂 (It also used to be the case that OpenAI would jump the AI newscycle by announcing before us from a backlog of ideas for every new Google release; safe to say that backlog is empty.) I have since joined xAI. The recipe is well-known. Compute, data, and O(100) brilliant, hard-working people are all that’s needed to obtain a frontier-level LLM. xAI *really* believes in this. For compute, even at Google I have never experienced this # of chips per capita (& 100K+ GB200/300K’s are incoming with Colossus 2). For data, Grok 4 made the biggest bet in scaling RL & posttraining. xAI is making new bets to scale data, deep thinking, and the training recipe. And the team is quick. No company has gotten to where xAI is today in AI capabilities in as little as time. As @elonmusk says, a company’s first- and second-order derivatives are the most important: xAI’s acceleration is the highest. I’m excited to announce that in my first few weeks, we launched Grok 4 Fast. Grok 4 is an amazing reasoning model, still the top on ARC-AGI and new benchmarks like FinSearchComp. But it’s slow and was never really targeted for general-purpose user needs. Grok 4 Fast is the best mini-class model—on LMArena, it is #8 (Gemini 2.5 Flash is #18!), and on core reasoning evals like AIME, it is on par with Grok 4 while 15x cheaper. S/o to @LiTianleli @jinyilll @ag_i_2211 @s_tworkowski @keirp1 @yuhu_ai_

Following its IMO gold-level win, @GoogleDeepMind is sharing Gemini Deep Think with mathematicians for feedback. Excited to see what they discover! 🧠 Plus, an updated Gemini 2.5 Deep Think is now rolling out for Google AI Ultra subscribers. Learn more: bit.ly/3IWcWq0

Our latest and greatest coding model! We've made some big strides for web app and visual development. And it continues dominating in user preference: #1 with a 37 Elo gap from #2.

This is so good. Love meta-analyses. From a benchmark it's much harder to optimize the test set (implicitly or otherwise).

2.5 Pro Exp is a model we're so proud of: #1 on LMArena, #1 on benchmarks like AIME, Aider, MMMU, and MRCR, & significant gains across coding, reasoning, multimodal, and so much more. Try it now! aistudio.google.com gemini.google.com

Here is what Gemini can do on *Flash*. My favorite perk: Gemini 2.0 Flash Thinking has significant gains in core capabilities while also excellent in user preferences (co-#1 with gemini-exp-1206 on @lmarena_ai). The best of both worlds.

We’ve been able to ship models in less than 24 hours. I’ve heard multiple VPs state they’ve never seen Google able to ship so quickly before.

I love the team’s shipping speed: today, not only the base model but also our update to Astra for real-time multimodal interactions, our Jules coding assistant, & Colab with Gemini 2.0.

Try out Gemini 2.0 Flash today. We made significant improvements across all domains, especially code, math, and multimodal reasoning. And 2.0 newly has native audio and vision generation! aistudio.google.com/prompts/…

gemini-exp-1206, out now. #1 everywhere. A 1 year anniversary for Gemini! aistudio.google.com/app/prom…

The team says hi again

Massive News from Chatbot Arena🔥 @GoogleDeepMind's latest Gemini (Exp 1114), tested with 6K+ community votes over the past week, now ranks joint #1 overall with an impressive 40+ score leap — matching 4o-latest in and surpassing o1-preview! It also claims #1 on Vision leaderboard. Gemini-Exp-1114 excels across technical and creative domains: - Overall #3 -> #1 - Math: #3 -> #1 - Hard Prompts: #4 -> #1 - Creative Writing #2 -> #1 - Vision: #2 -> #1 - Coding: #5 -> #3 - Overall (StyleCtrl): #4 -> #4 Huge congrats to @GoogleDeepMind on this remarkable milestone! Come try the new Gemini and share your feedback!

Nice work on controlling style biases! In this view, many models are no longer inflated (e.g., response length, formatting). Gemini 1.5 Flash also outperforms gpt-4o-mini overall and across all categories except for coding.

Welcome back @NoamShazeer to Google! It'll be a great time working together again since 2018. Let's take Gemini which is #1 and continue expanding the limits of its capabilities. techcrunch.com/2024/08/02/ch…

Our latest version of Gemini 1.5 Pro in AI Studio is #1 on the LMSys leaderboard. 🚀 This is the result of various advances in post-training and we have more lined up. Congrats to the Gemini team.

Gemini is #1 overall on both text and vision arena, and Gemini is #1 on a staggering total of 20 out of 22 leaderboard categories. It's been a journey attaining such a powerful posttrained model. Proud to have co-lead the team!

On results: * Spanish is where I expect models at. Gemini is within CI of #1 and should be #1 (it is so good at multilingual and also #1 on LMSYS non-English). * Coding as well. * Math focuses on grade school math which can be saturated. I expect the ranking to change on more complex problems. * Instruction following is surprising. Would be great to iron out whether it's a quirk from eval or generally consistent.

New public leaderboard from Scale! It looks like a solid set of evals. Mitigates two of the biggest problems in evals today: eval sets contaminated in model training, and rater quality for human evaluation.