Congratulations team, definitely gonna try it

Attention-free models like Brumby-14B-Base mark a pivotal shift in AI, slashing computational costs while maintaining high performance through innovative architectures. This could accelerate real-world AI adoption significantly. 🚀

"The initial weights for Brumby-14B-Base came from Qwen3-14B-Base" which used attention, so isn't this more of a fine-tuning without attention than a base model?

you basically reuse the QWen model weights as initialization. what's the point ?

cool but is it stronger than huggingface.co/featherless-a… though?

If attention is gone, how does the model make in-context learning in this case?