All the posts tagged "swiglu".
This blog recounts my testing on the impact of modern LLM optimizations on ViT-based models, such as SwiGLU, removing bias, and the Muon optimizer.
Harrison Chojnowski · February 19, 2026