AI Agents for Smart Contract Auditing: Finding Real Bugs Without Trusting Hallucinations

The Ethereum Foundation Protocol Security team's account of running coordinated AI agents against protocol code reaches the same practical conclusion: agents can find real bugs, but most of the work is distinguishing durable findings from confident noise. Its central operational rule is that a candidate needs a self-contained reproducer that another person can run against the real code.

Teams preparing for agent-assisted review should first understand the structure of a conventional smart contract audit and how to read smart contracts systematically. AI changes the speed and scale of hypothesis generation, but it does not remove the need to define trust boundaries, deployment assumptions, privileged roles, external dependencies, and intended economic behavior.

What an AI smart contract auditor actually does

A smart contract audit AI agent operates across code, tests, specifications, deployment data, and tool output. It can map external entry points, follow state changes, identify unchecked assumptions, search for familiar weakness patterns, draft Foundry tests, propose invariants, compare a new implementation with an older version, and summarize evidence for a reviewer.

Those capabilities are useful because smart contract security work contains a large amount of search. A reviewer must inspect authorization paths, value movement, upgrade logic, oracle assumptions, token behavior, rounding, callback boundaries, storage layout, initialization, signature handling, governance delays, emergency controls, and interactions across multiple contracts. An agent can inspect many of those surfaces in parallel and keep a structured record of what it examined.

The danger appears when search output is confused with truth. An agent can recognize a reentrancy-shaped pattern without proving reentrancy. It can find a privileged function without proving that the privilege is improperly assigned. It can identify a rounding direction without showing that the rounding is exploitable. It can claim an upgrade collision while reading the wrong implementation or storage layout. Each observation is a lead that deserves testing, not a final finding.

Use the agent as a hypothesis engine

A strong prompt does not ask the model to find all vulnerabilities. It gives the agent a bounded surface and asks for falsifiable claims. For example: identify ways an unprivileged caller could cause total assets to fall below total liabilities; identify sequences that let shares be minted for less value than the specification permits; or identify implementation changes that widen an administrative power.

Each claim should name the target, the invariant, the suspected mechanism, the attacker prerequisites, and the evidence that would settle the question. This structure forces the agent to move from vague language such as unsafe external call toward something testable.

target: Vault.withdraw(uint256)
property: user value cannot be withdrawn more than once
mechanism: external callback occurs before the balance is reduced
attacker: unprivileged contract with a funded vault position
success: attacker receives more assets than their recorded deposit
environment: release build, pinned commit, fork block recorded
reproducer: standalone test that fails against unchanged production logic
dedup_key: vault-withdraw-callback-before-accounting

Use deterministic tools as the judge

The model can write the test, but the test runner decides whether the observed behavior occurs. The compiler decides whether the proposed proof builds. A fork determines whether the external dependency behaves as assumed at a recorded block. A trace reveals the actual call order. Static analyzers, fuzzers, invariant runners, and formal tools add independent evidence, but none should inherit the model's conclusion without checking it.

This separation is the foundation of reliable agentic code security: probabilistic systems generate candidates, while reproducible systems evaluate claims.

Separate discovery, proof, severity, and remediation

A weak workflow lets one agent discover an issue, write the report, assign critical severity, propose a patch, and declare the patch complete in the same uninterrupted context. That encourages confirmation bias. Once the model commits to a story, later steps often defend it instead of testing it.

Discovery should maximize useful search

During discovery, false positives are acceptable if each candidate is specific and inexpensive to reject. The scout can explore suspicious authorization, asset-accounting inconsistencies, state transitions, external-call ordering, oracle freshness, signature replay, initialization, rounding, or upgrade assumptions. It should not spend time polishing severity language.

Proof should try to disprove the candidate

The proving stage should be hostile to the original hypothesis. Confirm that the suspected entry point is externally reachable. Confirm that every guard, modifier, upstream validation step, and protocol constraint is represented. Compile with the project settings. Use real interfaces. Avoid magical state writes unless the threat model genuinely gives an attacker that ability. If the claim collapses, close it with a rejection reason rather than rewriting the scenario until it appears valid.

Severity should begin only after reproduction

Before proof, severity is speculation. After proof, the triager can measure who can trigger the behavior, what they need, how much value is exposed, whether the attack can be repeated, whether a pause or upgrade can stop it, and whether existing monitoring would detect it. A real bug with extreme prerequisites may be less urgent than a smaller loss reachable by any user.

Remediation must target the mechanism

A patch that only makes the proof fail can be misleading. The test may have depended on one selector, one ordering, or one token. The underlying issue may remain accessible through another route. Review the root cause, search sibling functions, test alternate asset types and callers, and check whether the change creates a new denial-of-service, rounding, storage, or compatibility problem.

The AI security evidence funnel

Agent-assisted review works best as a funnel. It starts wide because search is cheap, then becomes increasingly strict as a candidate approaches disclosure. The objective is not to preserve every idea. The objective is to reject weak ideas early and concentrate human attention on findings that carry durable evidence.

A five-role agent team for smart contract bug triage

Multiple agents are not automatically better. Five agents using the same prompt and sharing the same assumptions can produce five versions of the same mistake. The value comes from role separation, independent context, and explicit disagreement.

The scout maps behavior before scanning patterns

The scout begins with architecture. It identifies assets, actors, permissions, external dependencies, upgrade boundaries, pause paths, signature domains, accounting units, and critical state transitions. It reads the specification beside the code. It should ask what must always remain true and which party benefits when that property fails.

Pattern matching comes later. A direct external call is not inherently a vulnerability. It becomes relevant when combined with mutable state, callback reachability, insufficient guards, or an accounting decision that can be repeated. Context-first scouting reduces superficial alerts.

The test writer converts claims into executable evidence

The writer chooses the smallest test that can settle the claim. A local unit test may be sufficient for a pure arithmetic error. A stateful invariant is better for multi-step share accounting. A fork test may be required when the claim depends on a deployed oracle, token, bridge, or proxy. Differential testing may be best when the risk came from a code change.

The writer must state why the test represents a valid attacker path. Direct storage mutation, impersonation, arbitrary bytecode replacement, and mocked return values are useful test tools, but they can also manufacture capabilities the attacker does not possess. Every cheat code needs a threat-model explanation.

The critic tries to break the proof

The critic checks whether the test fails for the claimed reason. It removes the suspected vulnerable action and confirms the failure disappears. It validates assertions before and after the attack. It checks event logs, balance deltas, storage, revert data, and call traces. It asks whether setup code already violated the invariant or whether a permissive mock produced the result.

The critic also checks reachability through the public system, not merely through an internal function. If a router normalizes an input before the library receives it, a proof that calls the library with an impossible value does not demonstrate an exploitable router.

The reproducer starts from clean state

The reproducer should not receive the original agent's narrative as its main input. It receives the candidate record, repository revision, build instructions, and test artifact. It checks out the target, installs locked dependencies, runs the real build mode, executes the test, captures the trace, and records whether the failure is stable.

This independent run catches hidden environment state, uncommitted changes, local dependency substitutions, cached artifacts, fork drift, and tests that only work on the author's machine.

The triager owns rejection as well as acceptance

A good triage system records why candidates failed. Useful statuses include verified, needs context, duplicate, unreachable, intended behavior, invalid oracle, environment mismatch, out of scope, and not reproducible. Rejection data improves later scouting because agents can avoid rediscovering closed paths.

The triager also groups symptoms under one root cause. If one missing authorization check affects five wrapper functions, five generated reports should not be treated as five independent vulnerabilities. Conversely, two similar traces may reflect different trust failures and require separate fixes.

Give the agents a security context package

Many hallucinations are not random. They are caused by missing information. An agent sees a function but not the deployment script, sees an interface but not the implementation, sees an implementation but not the proxy, or sees arithmetic without the economic definition of a share.

Pin the contract identity

Source code is not the deployed system until its identity has been established. Confirm the repository revision, compiler settings, linked libraries, constructor arguments, proxy implementation, and deployed bytecode. The smart contract verification guide explains why a verified source label alone does not resolve every deployment or proxy question.

Separate intended powers from accidental powers

An owner function that can pause withdrawals may be intentional. An owner function that can transfer user collateral to an arbitrary address may be intentional, unacceptable, or incorrectly scoped. The agent cannot decide from syntax alone. The threat model must state which administrative actions are part of the design, what governance delay applies, and which properties remain inviolable even for privileged actors.

Define economic terms precisely

Words such as value, profit, loss, solvency, price, share, collateral, and debt require exact definitions. If the oracle reports six decimals while internal accounting expects eighteen, the numeric mismatch matters. If total assets include pending yield but withdrawals cannot realize it, an apparently correct invariant may be economically false. Give agents the formulas and permitted tolerances.

Testing methods that turn AI guesses into evidence

Unit tests isolate one mechanism

A unit test is ideal when the claim depends on a narrow function and controlled state. It should include a positive control, a negative control, and assertions tied directly to the security property. If the concern is unauthorized parameter change, test the unauthorized caller, the authorized caller, and any intermediate role expected to be denied.

Agents are effective at drafting unit tests because the task is bounded. Reviewers still need to inspect setup. A test that grants the attacker an owner role and then demonstrates owner behavior is not an access-control finding.

Property-based tests explore input space

Example-based tests ask whether the system works for selected values. Property-based tests ask whether a rule survives many generated values. Useful properties include conservation of assets, monotonic exchange rates under defined conditions, bounded fees, authorization consistency, nonce uniqueness, and equivalence between preview and execution functions within an explicit rounding tolerance.

The agent can propose properties from NatSpec, specifications, existing tests, and economic formulas. The human reviewer must confirm that the property expresses the intended rule. A formally proven or heavily fuzzed weak property can create false confidence. Foundry's official invariant testing guide describes how Forge runs randomized sequences of calls and checks properties after each call, which makes the harness and selected invariant central to the result.

Stateful invariants explore sequences

Many severe smart contract bugs appear only after a sequence: deposit, donate, borrow, repay partially, liquidate, transfer shares, then withdraw. Every individual call may be valid. The order creates the failure. A stateful harness generates sequences across selected handlers and checks invariants after each step.

For a vault, the handler might deposit, mint, withdraw, redeem, transfer, donate, and advance time. Ghost variables track the model's expected deposits and withdrawals. Invariants compare on-chain state with the model. The agent's best role is suggesting handlers, sequence hypotheses, boundary values, and missing invariants. The runner remains the search engine for execution order.

// Simplified educational example
contract VaultInvariant is Test {
    Vault internal vault;
    VaultHandler internal handler;

    function setUp() public {
        vault = new Vault();
        handler = new VaultHandler(vault);
        targetContract(address(handler));
    }

    function invariant_accountedAssetsDoNotExceedHeldAssets() public view {
        assertLe(vault.totalAccountedAssets(), vault.assetBalance());
    }

    function invariant_withdrawalsNeverExceedDeposits() public view {
        assertLe(handler.ghostWithdrawn(), handler.ghostDeposited());
    }
}

This example is intentionally small. A production harness must model fees, yield, losses, token quirks, rounding, debt, donated assets, privileged recovery, and any asynchronous flow. An invariant is useful only when its model matches the protocol's actual accounting.

Differential testing compares behavior

Differential testing sends the same inputs or action sequences to two systems and compares relevant outputs. The comparison can be old implementation versus new implementation, optimized code versus reference code, Solidity versus another language, or the protocol versus a simple mathematical model.

Not every difference is a bug. A migration may intentionally change fee rounding or access policy. The test needs an allowlist of intended differences and explicit equivalence properties for everything else. The TokenToolHub Smart Contract Diff helps reviewers identify changed control paths and focus agent testing on the code that actually moved.

Exploit traces explain the real path

A passing exploit test is stronger when paired with a trace that shows the sequence of calls, callers, values, storage effects, token movements, emitted events, and reverts. The trace lets a reviewer confirm that the observed loss followed the claimed mechanism.

Minimize the trace. Remove actions that are not required. Shrink fuzzed values to the smallest counterexample. Name the attacker, victim, protocol, and privileged actors. Record balance deltas in the asset's native decimals and in economically meaningful units. If profitability depends on gas, liquidity, or price movement, include those costs.

Fork tests validate integration assumptions

A local mock often behaves more cleanly than a deployed dependency. Fork tests can validate proxy addresses, token behavior, oracle decimals, pool state, callback behavior, and permission configuration at a specific block. Always pin the chain and block number. A latest-block test that passes today and fails tomorrow is not a stable reproducer.

Forking does not eliminate assumptions. Historical state providers can differ, external services may be unavailable, and the proof may rely on liquidity that changes after the pinned block. Document every environmental dependency.

Static analysis and formal tools provide independent lenses

Static analyzers can identify patterns, data dependencies, upgrade risks, or suspicious control flow without executing every state. Formal tools can prove or disprove specified properties under their models. These are valuable complements to agents because they use different reasoning mechanisms.

Yet the specification remains decisive. Solidity's official SMTChecker documentation explicitly notes that formal verification compares the formal specification with the implementation. It cannot tell a team whether the specification itself captured the behavior they truly wanted. Unsupported features, abstractions, timeouts, and environmental assumptions must also be understood before interpreting results.

The evidence bar for a real finding

A finding should be able to stand without the model's prose. If a reviewer deletes the explanation and keeps the repository revision, test, configuration, and trace, the failure should remain understandable and reproducible.

Reproduction must use the real build

Compiler versions, optimizer runs, EVM targets, feature flags, and release settings can change behavior. The proof should run under the configuration that produces the shipped contract. If the contract is deployed, compare the relevant runtime bytecode or verified build artifacts. A debug-only assertion, test harness helper, or development implementation does not establish production impact.

The attacker must be realistic

List every prerequisite: role, signature, capital, allowance, token balance, timing, oracle movement, governance action, cross-chain message, or cooperating counterparty. If one prerequisite is unavailable to an unprivileged actor, the threat model changes. A malicious administrator finding can still matter, but it should not be described as a permissionless exploit.

The success condition must measure security impact

A revert may be expected input validation. A balance change may be a fee. A rounding difference of one unit may be required by the standard. A paused function may intentionally deny service. The proof must connect behavior to a violated property such as unauthorized asset transfer, insolvency, permanent lock, privilege escalation, replay, incorrect settlement, or bypassed governance.

The result must survive a clean rerun

Record the exact command, dependency versions, environment assumptions, chain ID, fork block, seed, and relevant configuration. Where fuzzing is involved, save the seed and minimized counterexample. Where time is involved, record timestamp assumptions. Where external data is involved, capture the state required for the proof without exposing secrets.

Why false positives dominate AI bug triage

AI can generate candidates much faster than a security team can validate them. That asymmetry makes triage the controlling cost. The right response is not to suppress all speculative search. It is to build a cheap rejection pipeline and reserve human attention for candidates that pass increasingly strict gates.

Unreachable call chains

The model follows a dangerous internal path but misses the external wrapper that validates inputs, checks a role, or normalizes state. Validate reachability from a public or external entry point available to the stated attacker. Include proxy fallback behavior, routers, hooks, delegatecalls, and upstream contracts.

Impossible state construction

The reproducer writes storage, mints assets, impersonates governance, or creates an object no real transaction can produce. Build the state through public actions whenever possible. If direct setup is necessary to isolate a mechanism, add a second reachability test demonstrating how the same state arises legitimately.

Wrong proxy or implementation context

The agent reviews an implementation as a standalone contract, misses initializer state, reads a stale implementation, or interprets storage without the proxy layout. Establish the live implementation, admin, beacon if present, initialization status, and storage layout before treating any code path as deployed reality.

Mocked-away security behavior

A mock oracle returns an invalid price, a token mock violates the expected interface, or a callback mock cooperates in a way the real dependency does not. Mocks are useful for unit isolation, but integration claims need fork tests or accurate dependency models.

Compiler and arithmetic misunderstandings

The report assumes unchecked wraparound where Solidity inserts a revert, assumes a revert where assembly wraps, or ignores casting and rounding semantics. Compile the actual code and test boundary values. Check whether custom math libraries, Yul blocks, and compiler versions change the behavior.

False oracles inside tests

A test can assert the wrong thing and still pass. It may compare two values derived from the same flawed calculation, check only that a call reverted rather than why, or define a trivially true property. Review the oracle separately from the target. Introduce a known faulty implementation and confirm the test detects it.

Duplicate symptoms

Agents often report the same root cause through different functions, callers, assets, or wording. Use a deduplication key based on the affected component, violated property, and causal mechanism. Merge shared root causes while preserving materially different impacts.

Missing business and protocol context

A fee recipient may be configurable by design. A liquidation may intentionally allow third parties to seize discounted collateral. A bridge may deliberately rely on a guardian set. The security question is whether behavior violates documented trust assumptions, not whether it looks centralized or surprising in isolation.

Severity inflation

Generated reports tend to narrate the largest imaginable outcome. Require a concrete upper bound based on deployed balances, caps, permissions, timing, liquidity, and recovery controls. Separate theoretical impact from currently exposed value.

Assign severity from conditions and consequences

Severity is a decision about risk, not a reward for cleverness. A sophisticated exploit affecting no reachable deployment may be informational. A simple missing check that lets any caller drain active collateral may be critical.

Document the severity argument in facts. State the minimum attacker capability, maximum demonstrated impact, current exposure, limiting conditions, and recovery assumptions. If an estimate depends on market liquidity or governance response time, label it as an estimate.

Fix validation is a second audit

Once a finding is accepted, the team must correct the mechanism without creating a regression. This stage deserves independent attention because rushed security patches often widen privileges, break integrations, alter storage, introduce denial-of-service conditions, or fix only the demonstrated input.

Preserve the original proof

Keep the reproducer unchanged as a historical artifact. Run it against the vulnerable revision to confirm it still demonstrates the issue, then run it against the patched revision to confirm the behavior is blocked. Add a clear regression test to the normal suite so future changes cannot silently restore the bug.

Compare the complete change

Review every modified line, dependency update, generated file, interface change, storage layout change, and deployment script. A narrow security patch should not carry unrelated refactoring. Use Smart Contract Diff to focus on changed control logic, then validate the final deployed contract rather than stopping at the repository diff.

Search for sibling paths

If one function failed because accounting occurred after an external call, inspect every function with the same ordering. If a signature omitted chain separation, inspect every signed message. If a role was misconfigured, inspect role grants, revocations, admin roles, initializers, and upgrade paths. Root-cause search prevents a partial patch.

Rerun broad properties

The fix may satisfy the specific exploit and still violate a system invariant. Rerun unit, fuzz, invariant, differential, and fork suites appropriate to the change. Inspect coverage, rejected calls, and handler distributions so a green run is not caused by the fuzzer failing to reach useful states.

Validate operational controls

If the remediation requires an upgrade, migration, pause, role change, or oracle update, review the operational transaction. Confirm the signer, target, calldata, implementation address, initialization call, timing, and rollback plan. A correct patch sent through an incorrect upgrade transaction is not a completed fix.

Prevent unsafe agent behavior and disclosure

Security findings can create direct risk. An agent should not publish a report, open a public issue, submit a public pull request containing an exploit, contact a protocol, or broadcast a transaction without explicit human authorization. The same applies when the agent is uncertain whether the target is a controlled project or an external system.

Use coordinated disclosure

Verified findings should enter the project's established security process. Record affected versions, reproduction steps, impact, suggested containment, and contact history. Give maintainers a reasonable opportunity to investigate and remediate. Public disclosure timing should reflect user risk, active exploitation, patch availability, and the relevant program rules.

Do not inflate a public narrative to make the research appear more important. Precise, restrained reporting helps maintainers act and protects users.

A production workflow before a professional audit

Agent-assisted review is most valuable before independent auditors begin. It can remove obvious defects, deepen tests, clarify specifications, and produce a cleaner evidence package. That allows professional reviewers to spend more time on complex economic, compositional, governance, and adversarial questions.

Phase 1: freeze and identify the review target

Create a review branch or release candidate. Pin dependencies and compiler settings. Record contract addresses when reviewing a deployment. Remove dead code and unresolved experimental features that are not part of the target. Generate a contract inventory and architecture map.

Phase 2: define the properties that matter

Write invariants in plain language and convert the highest-value ones into executable tests. Cover authorization, asset conservation, solvency, share accounting, pricing, settlement, signature uniqueness, upgrade restrictions, pause behavior, and recovery. State permitted rounding and trusted roles.

Phase 3: run bounded scouts

Assign narrow surfaces such as withdrawal accounting, liquidation, permit signatures, upgrade initialization, oracle handling, or fee calculation. Ask for candidate records, not finished reports. Enforce deduplication and coverage tracking.

Phase 4: reproduce independently

Send every candidate through the critic and clean reproducer. Reject impossible paths quickly. For survivors, minimize the test, pin the environment, capture the trace, and identify the root cause. Do not assign final severity yet.

Phase 5: triage with human ownership

A security engineer reviews the code and proof, confirms threat assumptions, checks known issues, evaluates impact, and assigns status. Findings that depend on business intent go to protocol designers. Findings that depend on external systems go to integration owners.

Phase 6: remediate and retest

Patch root causes, preserve proofs, add regression tests, review diffs, inspect storage and proxy consequences, and rerun relevant campaigns. Prepare a remediation summary that maps each accepted finding to a fix and test.

Phase 7: hand off a clean audit package

Give professional auditors the pinned code, architecture, trust model, deployment information, invariants, tests, known issues, accepted risks, agent findings with status, and remediation history. Do not bury them in untriaged AI output. A thousand speculative reports waste audit time; ten verified issues and a clear rejection log improve it.

Use contract intelligence around the agent workflow

AI review becomes more reliable when the agent's source-level reasoning is checked against deployment facts and contract controls. Before testing, establish whether the source corresponds to the target and whether a proxy changes the code path. After remediation, compare the exact versions and verify that the intended control changes appear without unrelated privilege expansion.

For token contracts, the TokenToolHub Token Safety Checker can help validate authority, upgradeability, source-verification, and trading-control signals around the contract being reviewed. These signals do not prove safety, but they can reveal deployment context an isolated source review may miss.

Production checklist for AI-assisted smart contract review

What AI agents still cannot certify

An agent cannot prove that it examined every meaningful behavior simply because it read every file. Coverage of text is not coverage of state space. Complex systems combine contracts, governance, user behavior, market liquidity, cross-chain delays, oracle failure, off-chain operators, and upgrade processes. Some failures emerge only under conditions absent from the repository.

An AI audit also cannot certify that a deployment is safe indefinitely. Administrators can change, implementations can upgrade, dependencies can be compromised, markets can move, and previously safe assumptions can expire. Security requires monitoring and operational discipline after launch.

Most importantly, the agent cannot own accountability. A human team must decide what the system promises, what risks it accepts, whether evidence is sufficient, how severity is communicated, and when disclosure protects users. Professional independent auditors remain valuable because they bring adversarial experience, domain judgment, accountability, and a fresh threat model that the building team and its agents may share too closely.

Conclusion

An AI agent for smart contract audit work can materially improve security when it is positioned correctly. It can search more code, propose more invariants, draft more tests, inspect more diffs, and explore more attack hypotheses than a small team could pursue manually in the same period. That increased generation capacity is real.

Trust, however, comes from the funnel after generation. The candidate must describe a falsifiable property. The path must be reachable. The state must be valid. The test must fail against the correct code for the claimed reason. A clean environment must reproduce it. The root cause must be isolated, duplicates removed, impact measured, severity justified, remediation retested, and disclosure controlled by people.

Use AI to arrive at a professional audit with clearer architecture, stronger invariants, deeper test coverage, fewer obvious defects, and a well-triaged issue history. Use the smart contract audit guide to prepare the broader review, confirm deployment identity with the verification workflow, and use contract comparison and control analysis to validate what actually changed. The model can accelerate the search. Evidence and human judgment must still decide what is real.

Frequently asked questions

References and further learning

• Ethereum Foundation Protocol Security: The triage is the product
• Foundry documentation: Invariant testing
• Building Secure Contracts: Testing a property with Echidna
• Solidity documentation: SMTChecker and formal verification
• Solidity documentation: Security considerations
• Solidity documentation: List of known compiler bugs

This guide is educational and does not certify any contract, deployment, protocol, or AI auditing system as secure. Security decisions should be based on the specific code, deployment, threat model, evidence, and qualified independent review.