papersSEP 10 04:00 UTC
CompassOPD adapts on-policy distillation to cross-family model pairs
New research introduces CompassOPD, a method that extends on-policy distillation to settings where the teacher and student models come from different families. It derives within-family likelihood shifts to provide dense token-level supervision on student-generated outputs, tackling the effectiveness drop standard OPD exhibits in cross-family scenarios.