Appearance
Writing
Technical notes, course reflections, and engineering writeups.
Sparse Linear Attention: Viewing SWA from the Perspective of Linear AttentionTechnicalK3's KDA Precision Problem: Why Lower-Bounded Decay Is NecessaryTechnicalFrom Heuristic Learning to Auto-Architecture SearchReflectionAfter DSA: Two Routes to a Better IndexerTechnicalDeepSeek DSA FLOPsTechnicalLinear Attention: From DeltaNet to KDATechnicalSingular Values and Spectral DecompositionTechnicalNotes on MLA FLOPs, RoPE, Query Compression, and MTPTechnicalAttention Sink SurveyTechnicalGlitch Token and MiniMax-M2TechnicalAn Open Problem in RL: Beyond Verifiable RewardsReflectionTraining and Inference Mismatch: IcePopTechnicalGLM-5 NotesTechnicalGEMM: SIMD Implement on CPU and Triton Implemetn on GPUTechnical