Category: Machine Learning

Summary: Comparing Sharpness-Aware Minimization against standard SGD on synthetic classification tasks using both accuracy and loss-landscape sharpness metrics.

Sharpness-Aware Minimization is designed to favor flatter solutions than standard SGD, but its benefits can depend on the task and on how sharpness is measured. This experiment asks how SAM and SGD differ on simple synthetic datasets when evaluated not just by accuracy, but also by sharpness, loss curves, and generalization gap.

The script trains models on spiral and concentric-circle classification problems and records train and test performance along with a power-iteration estimate of the top Hessian eigenvalue. That lets the experiment compare optimization quality and local landscape geometry within the same runs.

The value lies in connecting optimization method to measurable curvature, rather than judging the methods by final accuracy alone. It is intended as a compact benchmark for how flatter-looking minima relate to generalization on controlled tasks.

Method: Numpy-only training comparison between SAM and SGD with power-iteration estimates of top Hessian eigenvalues on synthetic datasets.

What is measured: Train and test accuracy, train and test loss, generalization gap, loss-landscape sharpness, and per-epoch learning curves.