Researchers propose a certified, differentiable complexity estimator that extends algorithmic information dynamics beyond the piecewise-constant Block Decomposition Method. This allows gradient-based control of complexity during training, which they apply to the phenomenon of grokking. The work is published as an arXiv preprint in machine learning.
New research contrasts how over-parameterized ridge regression behaves with structured versus unstructured random feature maps. Prior work by Xu, Vardi and Safran proved that random Gaussian features grok, with the gap between memorization and generalization widening as the weight decay parameter shrinks. The authors report that structured features instead overfit in the regime where random features exhibit grokking.
A new arXiv preprint examines grokking, the phenomenon where neural networks keep memorizing training data before abruptly improving on held-out data. While prior work has focused on why this delay happens, the paper targets its quantitative structure, describing scaling laws and a phase structure that predict when the shift occurs. The authors present an arXiv preprint; the abstract excerpt provided does not detail the full experimental setup or results.