a16z crypto ran experiments to assess whether autonomous AI agents could execute attacks against decentralized finance protocols in simulated environments.

The study found that without human guidance agents completed only 10% of full attacks, rising to 70% with historical exploit knowledge.

Experiment results

Even when agents identified vulnerable code paths, they routinely failed during the final stage of producing a functioning exploit payload.

Failures included calculation errors, inability to coordinate multi-step transaction sequences, and abandoning feasible strategies before deployment.

Effect of prior knowledge

Providing a knowledge base of past hacks and proven methods improved success rates significantly, but did not eliminate critical assembly issues.

With recorded examples available, agents reached full exploit completion in about 70% of trials, highlighting improved planning but persistent execution gaps.

Implications for security

The research suggests that AI already assists in vulnerability discovery but complex, multi-step DeFi attacks remain primarily solvable by experienced human operators.

Security teams can use AI to surface candidate bugs and prioritize review, while humans retain responsibility for exploit synthesis and transaction chaining.

Overall, autonomous agents show promise as assistants in DeFi security workflows, but do not yet replace human expertise in executing complex attacks.