This is a heavily interactive web application, and JavaScript is required. Simple HTML interfaces are possible, but that is not what this is.
Post
Dan Busbridge
dbusbridge.bsky.social
did:plc:gbjtrxs2syn7ym5gvdnj54p5
Reading "Distilling Knowledge in a Neural Network" left me fascinated and wondering:
"If I want a small, capable model, should I distill from a more powerful model, or train from scratch?"
Our distillation scaling law shows, well, it's complicated... 🧵
arxiv.org/abs/2502.08606
https://arxiv.org/abs/2502.08606
2025-02-13T21:50:31.452Z