Attention Flow

Attention Sink

Early tokens can absorb persistent attention mass, shaping how later queries distribute context.

attentioncontextsurvey

01BOSanchor

02Sinkmass

03Windowlocal

04Queryretrieve

sink weightstable

context drifttracked

mitigationdesign