Post
fry69
fry69.dev
did:web:fry69.dev
A combination of tokens inside those embeddings plus earlier attention can easily overshadow any later tokens when LLMs calculate that those earlier ones are highly relevant.
This all happens inside the embeddings and later the QKV matrices etc, completely independent from the actual tokens.
2026-02-19T13:17:58.551Z