-
Projection Splitting for Muon Optimizer
Muon and MoE are becoming standard in frontier LLMs, but neither is plug and play at smaller scale. Part I covers getting Muon to work at 280M parameters and why splitting fused projections is the key.
Kirill Luka · 2026년 3월 26일
-
Evaluating Modern LLM Optimizations on ViT
This blog recounts my testing on the impact of modern LLM optimizations on ViT-based models, such as SwiGLU, removing bias, and the Muon optimizer.
Harrison Chojnowski · 2026년 2월 19일
-
Rec-Flow Results: Stage 5 Highlights
Selected Stage 5 rec-flow generations with shared settings and short observations across style categories.
Harrison Chojnowski · 2026년 2월 14일