The AI Era Is Creating a Bug Hunting Arms Race

A decade ago, the advent of programs that reward researchers for reporting software vulnerabilities marked a significant shift toward recognizing the importance of vulnerability disclosure. Initially met with skepticism, these "bug bounty" initiatives have evolved from a climate of defensiveness to one where organizations willingly seek external input to facilitate timely fixes. When Apple introduced its bug bounty program in 2016, the highest reward offered was $200,000. This figure escalated to $1 million in 2019 and reached $2 million by last year. However, the landscape is on the verge of another transformation.

With the advancement of intelligent AI models capable of autonomously detecting software vulnerabilities and creating exploits, vulnerability disclosure programs are experiencing an overwhelming influx of submissions. Simultaneously, organizations are also identifying a greater number of bugs themselves. This surge fundamentally alters the economics of bug bounties, impacting both the institutions soliciting submissions and the researchers involved, many of whom rely on bug hunting for their livelihood or as supplemental income. It is crucial to note that attackers are also adapting to these changes.

Independent security researcher Joseph Thacker notes, "I've probably submitted three times more bugs than I did last year at this time—I would suspect that a company like Google is going to spend two to 10 times as much on bug payouts as they did last year." He observes that larger tech firms may accommodate such financial pressures, but many smaller companies struggle to keep pace. Presently, many submissions consist of low- to medium-severity issues, while intelligent agents are uncovering significant vulnerabilities. Thacker predicts that in the upcoming year, fewer bugs will be submitted as a substantial number will have already been identified, with certain companies likely increasing their rewards again.

Thacker, along with other researchers, acknowledges the uncertainty surrounding the long-term implications of these changing supply and demand dynamics. Depending on the effectiveness of AI-based exploit discovery and automated system scanning methodologies employed by malicious actors, developers may face increased urgency to release patches swiftly. This need could potentially accelerate the adoption of established practices such as the 90-day disclosure timelines, which traditionally serve as windows between the identification of vulnerabilities and their public acknowledgment to encourage timely patch releases.

Security researcher Himanshu Anand articulated this evolution by stating that "the 90-day responsible disclosure window was built for a world where bug finders were rare and exploit development was slow. That world is gone. LLMs have compressed both timelines."

Moreover, enhanced accountability for attackers might drive organizations to improve the speed at which they deploy vulnerability fixes. The proliferation of patches has long presented a formidable challenge within cybersecurity, as installing new software at scale without adequate testing can result in unintended outcomes, including service outages.

As real-world attacks facilitated by AI appear to intensify, both skilled and lesser-skilled actors are seeking to broaden their capabilities while minimizing costs. Recent findings from Google researchers indicate that "prominent cyber crime threat actors" have been leveraging AI tools to create exploits for a zero-day vulnerability capable of circumventing two-factor authentication on an open-source system administration platform. Following this discovery, Google promptly informed the affected developer, who implemented a fix. This incident serves as a pivotal example of the evolving dynamics within the bug-hunting arena.

"We all assumed it was already happening, and this is our first evidence that it is happening," commented John Hultquist, chief analyst for Google Threat Intelligence Group, reflecting on the utilization of AI by attackers to unearth new vulnerabilities and craft exploits.

Hultquist emphasized that while the issues posed by nation-state actors are critical, the majority of incidents faced by organizations stem from criminal actors, many of which carry significant implications. He remarked, "Zero-day use by criminal actors has been fairly limited, and the ones that do use them tend to be really successful, so I think we shouldn't underestimate the impact of more criminals with a zero day in their hands."

However, the landscape of bug hunting for researchers is changing. The command-line tool Curl suspended its bug bounty program (managed via HackerOne) in January, overwhelmed by a surge of low-quality submissions generated by AI.

"We have concluded the hard way that a bug bounty gives people too strong incentives to find and make up 'problems' in bad faith that cause overload and abuse," the organization stated at the time, while affirming their appreciation for valid vulnerability reports.

On a related note, Linus Torvalds, the creator of Linux, commented that the renowned Linux security mailing list has become "almost entirely unmanageable" due to the influx and redundancy of AI-generated bug reports.

More recently, in April, Daniel Stenberg, the founder and lead developer of Curl, announced via LinkedIn that the quality of submissions had improved. "Over the last few months, we have stopped getting AI slop security reports in the curl project," he stated. "Instead, we get an ever-increasing number of exceptionally good security reports, almost all utilizing AI. They're submitted at an unprecedented frequency, placing significant demands on our resources."

Towards the end of April, Google revealed it would be revamping its Vulnerability Reward Programs for both Chrome and Android, reducing payouts for certain categories of bugs while increasing them for others.

"As the security research landscape evolves with AI, we're making changes in our programs to ensure we're rewarding the most challenging and impactful vulnerabilities in our products," the company explained.

Jonathan Dunn, a cardiologist and bug bounty hunter, remarked, "I think 90th percentile bug hunters with special skills will always be able to have findings and get payouts from big companies. But even with AI, we also need to heavily incentivize ethical researchers to find vulnerabilities in public infrastructure and other critical systems that otherwise may not receive adequate attention from defenders."

Presently, many organizations are prepared to implement every conceivable solution to tackle the challenges—and reap the advantages—of accelerated bug discovery. "This is changing the dynamics of the bug-hunting industry, but it absolutely still requires human time," stated Alex Zenla, chief technology officer of cloud security firm Edera.

In a progressive move, Anthropic recently launched a HackerOne bug bounty program to encourage researchers to submit findings related to the company's systems and Claude AI models. Increasingly, some researchers contend that there is a pressing need for structural defenses to address the swift rise in vulnerability discoveries. In other words, they are devising digital solutions that mitigate or greatly reduce the exploitability of different classes of vulnerabilities.

"You can't patch your way out of this," asserts Niels Provos, a veteran security engineer and researcher. "You need to build infrastructure that makes as many bugs as possible irrelevant."