New blog post (link in next tweet) - "Standards, Incentives, and Evidence: The Frontier AI Governance Triad." I explain the basic building blocks of making sure that frontier AI systems are safe and secure.

Trying not to think about how Vince Gilligan said some wrong things about AI + just focus on the fact that all his work is great and his new show is supposed to be great *fingers in ears* la la la la

The actual phasing out part is hard in part due to that earlier misalignment, but there is an info asymmetry btwn companies + users here as well as abundant evidence of harmful uses such that deferring entirely to users’ preferences is, again, predictably causing harm

Not sure exactly what he meant ofc but basically my take here is - reasoning models blow away earlier models on basically every safety + capability dimension, and generally we just get better at alignment over time, so old models predictably increase various harms

Roon was right btw

Some wild content over at at @Gossip_Goblin

“In reality, it is not so” is Ilya-coded (complimentary)

(Antitrust, sensitive IP etc. are sometimes cited as reasons not to do this but IMO these are not the real barriers. Solvable if there is a desire to do do, easy excuses to disengage otherwise)

AI companies sometimes cooperate on safety but this is slowed by people disliking each other + some companies’ whole identity being “we’re better than them in XYZ ways.” May be more promising long term for companies to trust their AIs to negotiate with the other companies’ AIs.

How can we give defenders of critical infrastructure an advantage over attackers? @Miles_Brundage proposes Operation Patchlight: a national effort to use each jump in frontier AI capabilities to find and patch critical open-source code vulnerabilities before these AI capabilities diffuse to attackers. Attacks on critical infrastructure are already a big problem. 61% of US hospitals report ransomware affecting clinical care, with 17% saying it caused serious patient harm. It takes 491 days on average to apply critical security updates to hospital equipment, and cyber incidents in healthcare cost over $15 billion in 2023 alone. But it could get much worse. Future agentic AI systems may constitute advanced persistent threats, scaling the effective workforce of cyberattackers by orders of magnitude. The proposal addresses a market failure. Attackers can make millions from a single ransomware campaign, while open-source code maintainers (whose libraries underpin ~70% of commercially used software) remain chronically under-resourced. The plan: for the US government and AI labs to jointly fund AI-powered vulnerability discovery, plus more funds to give every hospital and power plant administrator an always-on AI security assistant that helps implement patches and security improvements. As AI systems become more capable, both the need for this project will become more acute and its feasibility more apparent, as future AI systems become capable of both cyberattacks and protecting infrastructure against them. Even large investments into AI-powered cyberdefense could prove to be extremely cost-effective for the US government. Read the online essay: ifp.org/operation-patchlight…

If you’re a hotel manager and you decide to take down the sign in the bathroom that says “we care about oceans and stuff. Don’t make us wash your towels too much,” do you get kicked out of the Hotel Illuminati or something?

I made a list of forecasts of the impact of AI on economic growth over the next decade. A few observations... (🧵): tecunningham.github.io/posts…

The UK’s @AISecurityInst is one of the rare successes of government. I wrote a piece for @Samfr substack on why and what lessons it gives us for how to make government work. There are five key lessons. A thread ⬇️

🚀 Hello, Kimi K2 Thinking! The Open-Source Thinking Agent Model is here. 🔹 SOTA on HLE (44.9%) and BrowseComp (60.2%) 🔹 Executes up to 200 – 300 sequential tool calls without human interference 🔹 Excels in reasoning, agentic search, and coding 🔹 256K context window Built as a thinking agent, K2 Thinking marks our latest efforts in test-time scaling — scaling both thinking tokens and tool-calling turns. K2 Thinking is now live on kimi.com in chat mode, with full agentic mode coming soon. It is also accessible via API. 🔌 API is live: platform.moonshot.ai 🔗 Tech blog: moonshotai.github.io/Kimi-K2… 🔗 Weights & code: huggingface.co/moonshotai

Gov't gets 10%, Trump himself gets least 200 tokens in the Model Spec

That too

(also wasn't the restructure supposed to mean no more complaining from OpenAI about fundraising being hard?)

The American AI industry has many problems but a lack of eager investors is not one of them

😬