Bug Bounty Submissions Should Require Deposits

Serious, not trolling

AI slop has been doing real damage to bug bounty programs. My company has expended significant engineering resources wading through the garbage. The bug bounty platforms do a decent job of filtering but some things are simply outside their expertise and need to be forwarded along.

This is an unfortunate turn of events. A few years ago false security reports were not terribly onerous and even when they happened they were usually someone earnestly thinking they’d found something. Even when the person was horribly confused they were usually serious enough that it felt right to try to encourage them.

To be clear, there’s nothing wrong with using AI as a tool for searching for bugs. If someone finds a completely legitimate security problem using AI as part or all of their toolchain and submits a properly formatted report they are free to claim it. I would give the benefit of the doubt and think that even the mostly bogus reports we’re getting are from people who are doing nontrivial amounts of work to train models specifically for bug finding with their own filters and processing to maximize chances of success. They must be submitting because they have some real hit rate.

The problem is that the burdens of evaluating false positives are borne entirely by the entity handing out the rewards. This wasn’t a problem back when submissions were done manually because back then having an instance of a report which was probably wrong but having a 1% chance of success was rare, and the costs of validating such things properly were small compared to the costs of coming up with the possible attack in the first place, and if you did submit and got a follow-up question answering it was a real burden on the submitter. Now none of those things apply so there’s a flood of low probability but worth a shot reports.

The solution to this I’d like to propose is something which would have been completely verboten a few years ago but now unfortunately may be necessary: Anyone submitting for a bug bounty should have to put down a deposit. Even a relatively low amount like $100 would probably make a huge difference. Ideally there’s a policy in place that there’s a generous refund program that submissions which are at all earnest get their deposit back even if they’re mistaken. If that causes too much arguing about what’s ‘earnest’ it may be necessary to make it a fee rather than a deposit, but I think it’s always legally okay to have a policy of returning such fees as long as it’s made clear up front that it’s completely discretionary on the part of the evaluator.

No doubt this suggestion will make some people very upset because it completely violates the traditional ethos of how bug bounties work. It would also create an opportunity for scammers to set up bug bounties for fake projects with lots of security holes which they then pocket the fees for submissions on and refuse to pay out any owed bug bounties. These are real problems and there are mitigations but rather than diving into the weeds I’d just like to say I know and I’m sorry but the situation is sufficiently out of control that this is probably necessary. I’m suggesting this publicly so I can be the bad guy who other people point to when they suggest it as well.