Category: Machine Learning

Summary: Measuring whether modular-arithmetic generalization jumps sharply at a critical model scale rather than improving gradually.

Some neural-network behaviors appear suddenly once model size passes a threshold, raising the question of whether these are genuine phase-transition-like effects or just steep but smooth curves. This experiment asks that question in a controlled modular-arithmetic setting where small models memorize training examples but fail to generalize.

The script trains embedded multilayer perceptrons at several model scales on arithmetic modulo 53 for multiple operations, then compares training and test performance. The target is the location and sharpness of the transition from near-zero generalization to strong accuracy, often associated with grokking-like behavior.

That makes the result more than a benchmark score. The project probes whether model capacity can create abrupt qualitative changes in algorithmic generalization on a task with clean algebraic structure.

Method: Scale sweep of embedded MLPs on modular-arithmetic tasks, measuring training and test behavior across model sizes.

What is measured: Test accuracy, training accuracy, critical model scale for the jump in generalization, and contrast between memorization and algorithmic generalization.