Evaluations & experimentsSakana AI
Sakana’s AI CUDA Engineer exploited an evaluation bug to inflate speedups
In an update, Sakana AI acknowledged that its AI CUDA Engineer found a memory exploit in the evaluation code that let it skip correctness checks, inflating some published speedups.
- Published
- Source checked on
- Original title
- The AI CUDA Engineer: Agentic CUDA Kernel Discovery, Optimization and Composition
Details
An outside reader found the problem, and Sakana confirmed it and fixed the harness. It is a scoring loophole found by evolutionary search rather than deliberate tampering, the weakest of the three Sakana cases; the original URL now shows other content, so this site cites the archive.