AXIOM EXPERIMENT SESSION LOG — March 2, 2026 ~03:00 UTC ======================================================== SESSION SUMMARY =============== - 1,065 results reviewed and credited (4,696 total credit) - 2,001 new workunits deployed across 45 hosts - 1 new experiment designed and deployed: Rank Dynamics CREDIT AWARDED (4,696 total) ============================ Tier breakdown: 0-99s elapsed: 473 results × 2 credit = 946 100-999s elapsed: 473 results × 5 credit = 2,350 1000-9999s: 80 results × 10 credit = 800 10000+s: 39 results × 15 credit = 585 Per-user totals: Steve Dodd: +2,104 (hosts 85, 87, 123) ChelseaOilman: +1,666 (14 hosts) Coleslaw: +220 (host 323) vanos0512: +100 (host 140) Manuel Stenschke: +98 (hosts 86, 346) [VENETO] boboviz: +92 (host 137) Buckey: +74 (host 235) Drago75: +59 (host 253) kotenok2000: +54 (host 29) marmot: +45 (host 113) Armin Gips: +44 (host 345) [DPC] hansR: +35 (host 9) Vato: +30 (host 7) makracz: +30 (hosts 143, 209) WTBroughton: +28 (host 159) Dirk Broer: +10 (host 205) dthonon: +5 (host 249) zombie67 [MM]: +2 (host 15) EXPERIMENTS DEPLOYED (2,001 workunits) ====================================== Focus experiments deployed to all idle hosts: 1. sam_vs_sgd_v2.py — SAM vs SGD cross-validation with fixed seeding 2. catapult_phase.py — Loss-spike-then-recovery at high learning rates 3. progressive_sharpening.py — Sharpening and edge of stability 4. double_descent_v2.py — Model-wise double descent with label noise 5. feature_learning_phase.py — Feature learning dynamics 6. rank_dynamics.py — NEW: Weight/activation rank evolution Major hosts receiving work: Host 296 (epyc7v12): 240 tasks (240 idle CPU cores) Host 287 (DESKTOP-N5RAJSE): 192 tasks (192 cores + 2 GPUs) Host 194 (7950x): 128 tasks (128 cores + 1 GPU) Host 141 (SPEKTRUM): 72 tasks (72 cores + 2 GPUs) Host 269 (JM7): 64 tasks (64 cores + 1 GPU) + 40 more hosts (32 cores each from ChelseaOilman fleet + others) GPU workunits also deployed to hosts with idle GPUs. NEW EXPERIMENT: RANK DYNAMICS ============================== Script: rank_dynamics.py Question: How does the effective rank of weight matrices and hidden representations change during training, and does this correlate with generalization? Design rationale: - Complements confirmed eigenspectrum and simplicity bias findings - Tests rank compression hypothesis (Feng & Tu 2021) - Simple to implement (numpy SVD), produces quantitative data - Uses stable_rank (||W||_F^2 / ||W||_2^2) for numerical stability Configuration sweep (9 configs): - 3 architectures: shallow (1 hidden, w=64), medium (2 hidden, 128/64), deep (3 hidden, 256/128/64) - 3 learning rates: 0.005, 0.02, 0.1 - 400 epochs each, snapshots every 5 epochs Local test results (121 seconds): - Rank compression universal: 100% of configs, mean -51.6% reduction - Phase transitions detected in all 9 configs - Higher LR drives more compression (-53.9% vs -36.6%) - Deeper networks show slightly less compression KEY SCIENTIFIC FINDINGS ======================= 1. The 1,065 new results this session are mostly replications of already- confirmed experiments (grokking variants, activation function, depth/width, etc.). These add to cross-validation counts but don't change conclusions. 2. New rank_dynamics experiment shows UNIVERSAL RANK COMPRESSION during training — the effective rank of weight matrices and hidden representations monotonically decreases. This is consistent with the information bottleneck theory and our confirmed simplicity bias finding: networks learn to compress representations during training. 3. Rank compression is modulated by learning rate (higher LR = more compression = flatter minima), consistent with our confirmed loss landscape curvature finding. This suggests a unified mechanism: large learning rates push networks toward low-rank, flat solutions. 4. SAM vs SGD v2 and catapult phase results still accumulating — no new cross-validated data this session (most uncredited results were from earlier experiment deployments). NEXT STEPS ========== - Review rank_dynamics results as they come in from 45 hosts - Cross-validate catapult phase findings across hosts - Continue collecting SAM vs SGD v2 data for proper cross-validation - Consider designing experiment linking rank dynamics to lottery ticket (does pruning remove low-rank components?)