\n<\/p>\n
This week, Sakana AI, a Nvidia-backed startup that\u2019s raised hundreds of millions of dollars from VC firms, made a remarkable claim. The company said it had created an AI system, the AI CUDA Engineer, that could effectively speed up the training of certain AI models by a factor of up to 100x.<\/p>\n
The only problem is, the system didn\u2019t work.<\/p>\n
Users<\/a> on X quickly discovered<\/a> that Sakana\u2019s system actually resulted in worse-than-average model training performance. According to one user<\/a>, Sakana\u2019s AI resulted in a 3x slowdown \u2014 not a speedup. <\/p>\n What went wrong? A bug in the code, according to a post<\/a> by Lucas Beyer, a member of the technical staff at OpenAI. <\/p>\n \u201cTheir orig code is wrong in [a] subtle way,\u201d Beyer wrote on X. \u201cThe fact they run benchmarking TWICE with wildly different results should make them stop and think.\u201d <\/p>\n In a postmortem published<\/a> Friday, Sakana admitted that the system has found a way to \u2014 as Sakana described it \u2014 \u201ccheat\u201d and blamed the systems\u2019s tendency to \u201creward hack\u201d \u2014 i.e. identify flaws to achieve high metrics without accomplishing the desired goal (speeding up model training). Similar phenomena has been observed in AI that\u2019s trained to play games of chess<\/a>.<\/p>\n According to Sakana, the system found exploits in the evaluation code that the company was using that allowed it to bypass validations for accuracy, among other checks. Sakana says it has addressed the issue, and that it intends to revise its claims in updated materials.<\/p>\n \u201cWe have since made the evaluation and runtime profiling harness more robust to eliminate many of such [sic] loopholes,\u201d the company wrote in an X post. \u201cWe are in the process of revising our paper, and our results, to reflect and discuss the effects [\u2026] We deeply apologize for our oversight to our readers. We will provide a revision of this work soon, and discuss our learnings.\u201d<\/p>\n Props to Sakana for owning up to the mistake. But the episode is a good reminder that if a claim sounds too good to be true, especially in AI<\/a>, it probably is.<\/p>\n<\/div>\n