Today we are releasing Brumby-14B-Base, the strongest attention-free base model around. manifestai.com/articles/rele…

Oct 29, 2025 · 8:00 PM UTC

8
30
6
198
Replying to @manifest__ai
Congratulations team, definitely gonna try it
Replying to @manifest__ai
Attention-free models like Brumby-14B-Base mark a pivotal shift in AI, slashing computational costs while maintaining high performance through innovative architectures. This could accelerate real-world AI adoption significantly. 🚀
Replying to @manifest__ai
"The initial weights for Brumby-14B-Base came from Qwen3-14B-Base" which used attention, so isn't this more of a fine-tuning without attention than a base model?
2
1
82
Replying to @manifest__ai
you basically reuse the QWen model weights as initialization. what's the point ?
19
Replying to @manifest__ai
If attention is gone, how does the model make in-context learning in this case?
1
1
Replying to @manifest__ai
Mfers on here be like "im solving AGI". No. You're fine tuning a qwen model. I'm convinced there are less than a dozen real ai labs in 2025.
5