This is a heavily interactive web application, and JavaScript is required. Simple HTML interfaces are possible, but that is not what this is.
Post
mr. TIM
timkellogg.me
did:plc:ckaz32jwl6t2cno6fmuw2nhn
new 3-token attention reduces pre-training data requirements
the pre-training scaling laws dictated that you have to scale up model size, data and compute in tandem. But this new method means you can double the model size without doubling the data
arxiv.org/abs/2507.02754
https://arxiv.org/abs/2507.02754
2025-07-06T12:37:27.561Z