This is a heavily interactive web application, and JavaScript is required. Simple HTML interfaces are possible, but that is not what this is.
Post
Dan Busbridge
dbusbridge.bsky.social
did:plc:gbjtrxs2syn7ym5gvdnj54p5
2. Maximum Update Parameterization (MuP, arxiv.org/abs/2011.14522) works out of the box for distillation. The distillation optimal learning rate is the same as the base learning rate for teacher training. This simplified our setup a lot.
2025-02-13T21:50:31.784Z