Can AI audit smart contracts? What the data shows
Yes. AI can now find a large share of the known vulnerabilities in real smart contract code. In our own run on 8 July 2026, the QuillShield alpha found 98 of 117 on EVMbench (83.8%, self-reported), against 45.6% for the best single model in the original EVMbench paper. AI does not replace a manual audit.
Two measurements sit behind that answer, and they measure different things. A public benchmark shows how many known bugs an AI system finds. A review by human auditors shows how many of its findings are real. Neither covers what a manual audit is for, which is where this post ends.
What is an AI smart contract audit?
An AI smart contract audit is a security review of contract code done by AI agents instead of a human team. The agents read the code, look for ways to break it, and report each finding with a severity, the affected lines and the impact. Some systems also write an exploit test to check that a finding works.
One example is QuillShield, made by the audit firm QuillAudits. QuillShield is an AI security auditor for smart contracts. It runs a full audit of a Solidity codebase, builds a security graph and tests its findings with proofs of concept, then checks every pull request against that graph.
How many known vulnerabilities does AI find today?
One public measure for Solidity is EVMbench, a benchmark from OpenAI and Paradigm. Its current version holds 117 high-severity vulnerabilities from 40 real audits. In detect mode an agent audits a repository, and its score is the share of the known vulnerabilities that its report contains.
| System | Known vulnerabilities found | Who measured it, and when |
|---|---|---|
| QuillShield alpha | 98 of 117 (83.8%) | Our own run, 8 July 2026, self-reported |
| Best single model in the original EVMbench paper | 45.6% | The paper's authors, February 2026 |
Our run covered all 40 audited projects. The alpha found every known vulnerability in 28 of them and missed 19 in total. The 45.6% baseline comes from the original EVMbench paper. The table compares a multi-agent system with one model working alone, and it is not a ranking of tools. Other teams publish their own runs, and scores from different setups are not directly comparable. Our EVMbench results covers the method and what is still unpublished.
One experiment in the arXiv version of the paper shows where single models struggle. When the authors gave one model a hint about which mechanism was broken, its patch score more than doubled. Their reading is that finding the bug is the bottleneck, and that fixing it is the easier part. Paradigm's launch post goes further and says "a growing portion of audits in the future will be done by agents". A follow-up paper, covered below, disputes the bottleneck reading.
Do the findings hold up when auditors check them?
A benchmark counts hits against a known list. It cannot say how many of a tool's other findings are wrong, because nobody grades those. The EVMbench paper states this as a limit of its own detect mode. The measure that answers the question is validity, the share of reported findings that turn out to be real.
During QuillShield's private beta, QuillAudits auditors reviewed findings it had produced on client Solidity code. 90% of the findings they reviewed were valid.
Read that number with its limits. It covers the findings auditors reviewed, which is not every finding the product has reported. We do not publish the count behind it. The reviewers work for the firm that makes the product. It says nothing about bugs that were missed. And some reviewed findings did not hold up, so plan to triage.
QuillShield does two things to back up a finding. A separate reviewer agent tries to disprove each finding before you see it. On a full audit, a finding can also carry a proof of concept status. In the demo audit on the home page, a full audit of a demo vault, QuillShield reported 5 findings, 3 of them Critical, and 5 of 5 proofs of concept passed. It took 2h 7m. A demo is the easy case. The docs list four statuses for a proof of concept (none, pending, passed and failed), and on real code not every finding gets a passing one.
An independent researcher also tested QuillShield on a live lending contract he had already audited. Read how that test went.
Where does a manual audit still add the most?
- It misses bugs. In our own EVMbench run the alpha missed 19 of the 117 known high-severity vulnerabilities, about one in six.
- The benchmark's bugs are public. They come from published audit reports, so a model may have seen them before the test. A follow-up paper tested agents on 22 incidents that happened after the models were released. The agents detected up to 65% of those vulnerabilities, and none completed an end-to-end exploit in 110 agent and incident pairs. Its authors conclude that agents are dependable on familiar bug patterns but "cannot replace human judgment".
- It does not judge the design. QuillShield can check your code against the whitepaper and past audits you attach. Whether the mechanism is sound, whether the incentives hold under stress and whether an admin role should exist at all are calls for people.
- It is narrow today. QuillShield supports Solidity for Foundry projects and nothing else.
- It takes hours. The demo audit took 2h 7m, and larger codebases take longer. Findings appear while the audit runs, so your team can start on them before it finishes.
How do teams use AI audits with manual audits?
QuillAudits has audited 1,500+ protocols since 2018 and also makes QuillShield, so it sells both. We suggest using them in this order.
- Before a manual audit, run an AI audit and fix what it finds. The auditors can then spend their time on design and economics.
- Between audits, review every change. Each merged pull request is code no auditor has seen. QuillShield checks each pull request against the last full audit, and the check fails on a High or Critical finding by default.
- When a finding matters, ask a person. From a completed audit you can ask a QuillAudits auditor to verify specific findings, or request a manual audit.
Handing unreleased code to any tool is a risk of its own. QuillShield's terms of service say findings are disclosed to you alone, source code is deleted after each audit, and your code and findings are never used to train a model.
AI audit vs manual audit sets the two side by side, and the quickstart walks through a first audit.
What to do next
Test this on code you already know. Start an audit on a repo that has been audited before, and compare the findings with the report you have. New accounts get $50 in credit, good for up to half of an order.
Common questions
Can AI replace a smart contract auditor?
No. AI finds many known bug types and can review every change, but it misses bugs and does not judge whether a design is sound. In our own EVMbench run on 8 July 2026, the QuillShield alpha missed 19 of the 117 known vulnerabilities. QuillShield does not replace a manual audit, and QuillAudits offers both.
How accurate are AI smart contract audits?
It depends on the system and on what you measure. In our own run on 8 July 2026, the QuillShield alpha found 98 of the 117 known vulnerabilities on EVMbench (83.8%, self-reported) across 40 audited projects. The best single model in the original EVMbench paper found 45.6%. Separately, 90% of the findings QuillAudits auditors reviewed on client Solidity code during the private beta were valid.
Which languages can AI audit today?
The benchmark evidence in this post is for Solidity. EVMbench covers Solidity contracts on EVM chains only. QuillShield supports Solidity today, for Foundry projects. More languages are coming, one at a time.
Is it safe to give an AI auditor private code?
Read the vendor's terms before you upload anything. QuillShield's terms of service say findings are disclosed to you alone, source code is deleted after each audit, and your code and findings are never used to train a model. The QuillShield GitHub App reads your code and writes only checks, statuses and pull request comments.
Sources
- Introducing EVMbench, OpenAI
- Introducing EVMbench, Paradigm
- EVMbench paper, original release of February 2026 (PDF)
- EVMbench paper, arXiv version of 5 March 2026
- Follow-up paper re-evaluating EVMbench, arXiv 2603.10795
- QuillShield docs, Review findings
- QuillShield docs, Quickstart
- QuillShield terms of service
- Manual smart contract audits by QuillAudits