Two-phase experiment: permute a trained model's 12 MLP modules and 144 attention heads across layers, reset the tied embedding, discard the optimizer, and retrain against a matched from-scratch baseline. Short answer: negative transfer from converged models, small positive transfer from very young ones.
Snapshots at 500 / 1,500 / 3,000 / 10,000 source steps, identically shuffled and retrained 3,000 steps. Final gap vs scratch rises monotonically with source age: −0.0055, +0.0131, +0.0404, +0.0858. All Holm-significant. Seven interactive charts including log–log views.
Read the ablation report →The original one-condition comparison (+0.0381 final). Kept for provenance — it had a schedule mismatch (3k vs 20k cosine horizon) that the ablation corrects; the corrected figure is +0.0334 and the conclusion is unchanged.
Read the pilot report →Code: shuffle/audit/plot/analysis scripts, pinned nanoGPT source and configs, tests (checkpoints, the .venv, and tokenized OpenWebText excluded). Logs: every train.log, metrics.jsonl, significance/ablation CSVs and JSONs, audits, and the generated plots.