This is a heavily interactive web application, and JavaScript is required. Simple HTML interfaces are possible, but that is not what this is.
Post
mr. TIM
timkellogg.me
did:plc:ckaz32jwl6t2cno6fmuw2nhn
Distillation scaling laws
Distillation works, but you must balance the difference between loss (performance) of the teacher & student, they need to be similar
paper: arxiv.org/abs/2502.08606
o3-mini-high dive: https://chatgpt.com/share/67b1dced-3828-8004-b553-6e31a36d2d04
https://arxiv.org/abs/2502.08606
2025-02-16T12:47:56.072Z