Tag: llm

Projection Splitting for Muon Optimizer

Muon and MoE are becoming standard in frontier LLMs, but neither is plug and play at smaller scale. Part I covers getting Muon to work at 280M parameters and why splitting fused projections is the key.

Kirill Luka · March 26, 2026

Evaluating Modern LLM Optimizations on ViT

This blog recounts my testing on the impact of modern LLM optimizations on ViT-based models, such as SwiGLU, removing bias, and the Muon optimizer.

Harrison Chojnowski · February 19, 2026