Deep-Learning on David Wilde https://david.wilde-ventures.com/tags/deep-learning/ Recent content in Deep-Learning on David Wilde Hugo en-us <a href="https://creativecommons.org/licenses/by-nc/4.0/" target="_blank" rel="noopener">CC BY-NC 4.0</a> Thu, 26 Mar 2026 00:00:00 +0000 Adapting Karpathy's Autoresearch to Numerai: 358 Experiments Later https://david.wilde-ventures.com/posts/adapting-karpathy-autoresearch-to-numerai/ Thu, 26 Mar 2026 00:00:00 +0000 https://david.wilde-ventures.com/posts/adapting-karpathy-autoresearch-to-numerai/ <p><strong>TL;DR</strong>: I adapted Karpathy&rsquo;s autoresearch to optimize a neural network for the Numerai tournament. 358 autonomous experiments, 22 kept (6% hit rate), payout improved from -0.008 to 0.0280. Key adaptations: era-purged CV, multi-seed validation gates, synthesized learnings after each iteration, and a DO NOT RETRY table to prevent re-exploring exhausted search regions. Biggest surprise: every fancy tabular DL architecture (FT-Transformer, TabNet, GLU, MoE) lost to a plain 3-layer feedforward net with GELU.</p>