Category: Machine Learning

Summary: Locating the width regime where neural-network training crosses from active feature learning into a more kernel-like lazy regime.

A central theoretical question in deep learning is when wide networks stop substantially changing their internal features and begin behaving more like linearized kernel machines around initialization. This experiment asks where that crossover occurs in practice for a standard-parameterization network as width increases.

The setup uses a teacher-student task so learned features can be compared against an underlying structured target. Across a broad width sweep, the script tracks weight movement, changes in hidden activations, representation alignment, kernel-related quantities, and test accuracy at repeated checkpoints. Those measurements act like order parameters for a regime change between rich feature learning and lazy training.

That makes the project a phase-transition style study of training dynamics rather than a simple benchmark comparison. The goal is to identify not only whether the two regimes differ, but where the crossover sits and which observables detect it most clearly.

Method: Teacher-student MLP width sweep with checkpointed measurements of weight movement, activation change, feature alignment, and related lazy-training indicators.

What is measured: Relative weight movement, activation similarity, representation or kernel alignment change, teacher-feature alignment, test accuracy, and estimated crossover width.