If the underlying model is exactly the same, why can simply adding a methodology layer push the detection rate from 34% to 92% — that big a difference?
A frontier model on its own is a highly capable general-purpose system, but without being specifically guided to think about a particular domain's problems in a particular way, it handles any task in a generic mode. Shown a piece of Smart Contract code, a bare model might offer generic code review comments, but won't automatically know which categories of DeFi-specific vulnerability patterns to prioritize checking — for instance, what verification process to run for reentrancy attacks, price manipulation, or access control flaws respectively. What the report calls "structured review phases" and "DeFi-focused security heuristics" are, in essence, a conversion of security experts' actual vulnerability-hunting workflows, checklists, and common vulnerability pattern libraries into a framework that guides the model step by step. The model's underlying reasoning capability hasn't changed — it's been guided to use the right method, in the right order, checking the right places — which is why the gap in outcomes gets amplified so much.
Cecuro chose not to release the full tool, citing fear it could be repurposed for attacks — but the dataset itself is already public. Does this kind of restraint actually matter?
It matters, but only to a limited degree, and this is precisely the dilemma this category of research generally faces. Publishing the benchmark dataset (90 already-public historical exploit contracts) lets other researchers compare different systems' detection capabilities against the same standard, and carries relatively low risk on its own, since these vulnerability cases were already public information. Knowing "what problems these contracts had in the past" and actually possessing a complete system capable of actively scanning for and finding new vulnerabilities are two entirely different tiers of capability. If the latter were released, it would effectively hand a ready-made "automatically find vulnerabilities" tool to anyone, regardless of whether their intent is defensive or offensive. Withholding the full tool narrows who can conveniently access that capability, but it can't stop another team from independently building something similar — this buys time, it doesn't categorically Block the outcome.
As a DeFAI user, if I see a protocol claiming it "uses AI for security audits," what specific questions should I actually press on to judge whether that claim has any substance?
Concrete questions worth asking include: which model is being used, and has it been given a dedicated methodology specific to DeFi or the particular subdomain this protocol belongs to (lending, derivatives, cross-chain bridges, say), or is it just a generic model applied as-is; what detection rate has this audit system actually measured on a public historical exploit dataset, with a verifiable concrete number rather than a vague claim like "high accuracy"; and is the audit a one-time run or ongoing continuous monitoring — since new attack techniques and vulnerability patterns keep emerging, a one-time audit's guarantee degrades over time. A protocol that can't give concrete answers to these questions is usually signaling that "we use AI for audits" is, at this point, just marketing language not yet backed by an actual methodology.
Is this research overall optimistic or pessimistic evidence for the direction of "using AI for defense"?
Both, depending on which half you're looking at. The optimistic part: a 92% detection rate proves that a specifically designed AI system can genuinely approach a practically useful level of defensive capability — this isn't speculation, it's a quantified, verified result. The pessimistic part: that 34% baseline figure is precisely what suggests most products currently on the market claiming to "use AI for security" without publishing a concrete methodology or detection rate figure are more likely performing closer to that lower baseline than to the carefully engineered 92%. The real value of this research isn't telling you whether AI defense works — it's telling you that under the same label ("AI security"), products can differ in effectiveness by a factor of three, and as a user, marketing language alone won't let you tell which one you're actually getting.
A study published by security firm Cecuro on February 20, 2026 produced a comparison that's particularly useful for evaluating "AI security tools" as a category: the exact same frontier model, with a targeted methodology layered on top, saw its detection rate jump from 34% to 92%. That gap isn't a story about which model is smarter — it's a story about how large the gap can be between a bare model and an application layer actually designed for a specific domain.
The research team built a test set of 90 real-world exploited DeFi smart contracts, with cumulative verified losses totaling $228 million. A specifically designed AI system successfully detected vulnerabilities representing $96.8 million in exploit value across those 90 contracts — a 92% detection rate. A baseline comparison using GPT-5.1 with no additional methodology layered on top detected only $7.5 million in exploit value — a 34% detection rate. Both ran on the exact same underlying frontier model; the entire difference came from the application layer, which the report describes as "domain-specific methodology, structured review phases, and DeFi-focused security heuristics layered on top of the model."
The most direct lesson from this comparison is that the phrase "we use AI to detect vulnerabilities" conveys almost no useful information on its own — because the exact same underlying model's detection rate can differ by nearly threefold depending entirely on whether it's wrapped in a dedicated methodology. If a security product only advertises which frontier model it uses without describing what domain-specific design sits on top of that model, the claim by itself is no guarantee of quality whatsoever.
Notably, Cecuro stated it did not release the full version of this detection system, citing concern that the same tooling could be repurposed for offensive use — the exact same capability for finding where vulnerabilities are becomes, in different hands, a capability for finding where to attack. What was published is the benchmark dataset and methodology-level findings, letting other researchers compare different systems against the same benchmark, without handing anyone a ready-made offensive or defensive tool.
A separate figure cited in this study echoes observations from other institutions around the same period: research from Anthropic and OpenAI shows that AI agents executing end-to-end exploitation average roughly $1.22 per contract, with this category of attack capability roughly doubling every 1.3 months. Taken together, these two bodies of research paint two sides of the same trend: the cost of acquiring this capability on the offensive side is falling fast, and a defensive side that merely "installs an AI tool" without confronting the gap at the methodology level may end up fielding a 34%-detection-rate system against an adversary whose capability keeps accelerating.