LoCoDiff: Natural Long Context Code Bench can models reconstruct the current state of code files, given the diff history? - naturally interconnected content. no padding or junk context - simple generation and eval - reasoning models do the worst - except Sonnet 3.7!

Mentat's first fan art

Mentat also iterates on GitHub Actions. If you want to get the most out of Mentat, you ought to setup GitHub Actions. Here, it iterated three times before passing. iirc some tests were failing, then there was a typing issue. Important note: you can set a 'Credit Limit' on your agents, which is the number of credits it can spend without any human interaction. By default this is $5. So if it's spent $5 and still hasn't passed CI, it will stop until you poke it again.

I gave both Claude Code and Mentat the same task and had them race, side by side see below for the highlights (and the unexpected twist at 0:33), full video in next tweet

Listen. You don't know shit about vibe coding. Cursor is only half of the equation. Less than half. You think the 100x engineer single-threads his workflow? (or her workflow) You need to be using ASYNC agents. You need to be maximizing your token consumption. Mentat works INSIDE OF GITHUB. Github is already optimized for async workflow. It goes like this. 1) Write an issue and tag at-mentatbot 2) Write another issue and tag at-mentatbot 3) Check for a PR on the first one 3a) Pull it into cursor 3b) Run it, read it 3c) Leave a review 4) Check for a PR on the second one Now scale this up to N. This is how you get the 100x engineer. This is how to vibe code. Mentat Dot AI.

> honestly it fixed it almost as fast as we were able to see that CI had failed > I strongly believe you need to make it easier for your agents to get feedback on their own, instead of getting feedback from you

how to use mentat on a new project - basics of working with mentat - agent pages - mentat iterating on CI failures - setting up mentat scripts - giving mentat your api keys first in a series of videos using mentat to build "gpt generals"

I wrote a blog post about how LLMs are really not like junior developers at all.

Our official response @granawkins

i have become the rubber duck

Mentat AI Agent did like 80% of this. I basically: - Write a GH issue for a feature/fix - Tag @mentatbot in it - 3-5 minutes later it submits a PR - I review the PR, maybe run locally, give feedback - I merge the PR and deploy Here were some of the tasks it nailed 👇 (#4 is 🤯)

Mentat is now powered by a newsonnet agent and is shockingly good Log in with Github and we give you $30 of credits to try it out! - generate PRs - review your PRs - iterate on CI - run arbitrary code

I wrote a blog post about our decision to write a GitHub bot instead of an editor. Leads to better: - Async work - Collaboration

Singularity next year fyi

the model can respond with draft issue artifacts, which you can post to github with a single click, where MentatBot will begin working on them I've found it's much faster to iterate on a plan in chat mode before sending it to be converted to a PR!

MENTATBOT FREE TIER Try MentatBot for free, starting today! 1) Sign in with Github on mentat dot ai 2) Tag MentatBot on any issue 3) Get best-in-the-world AI PR generation in minutes

1/ Thrilled to announce that our team has created the most advanced coding AI in the world, smashing the previous State-of-the-Art by solving 38.33% of SWE-bench Lite! MentatBot is not only the most accurate, but runs extremely quickly and is available for you to use today!

Today we launch MentatBot: a Github-native, SOTA coding agent. - Scored 38% on SWE-bench lite (previous SOTA was 33% 🤯) - Writes PRs based on issues - Reviews your PRs, responds to "@MentatBot" - Available *right now* 1/7

Abante Intelligence