A well-designed and purposefully run bug bounty program with stakeholder buy-in can have a tremendous impact on an organization’s attack resistance. However, just like any tool within your security stack, bug bounty programs require a level of investment and mutual buy-in upfront to ensure that your business retains optimal value. With the right investment, bug bounty programs pay dividends by ensuring continuous visibility for your public-facing digital assets, complimenting your existing security team, and providing contextual feedback for developers.
With this two-part blog series, we will delve into strategies and tactics exhibited by dozens of enterprise customers over several years that will optimize the effectiveness of your bug bounty program.
This article will examine the three most important steps to consider when designing an effective strategy for a public or private bug bounty program.
Define Program Success
At HackerOne I often hear new customers often say, “we want to receive lots of critical bugs.” The goal of a bounty program should center around reducing risk—but this doesn’t always mean focusing on only finding high Common Vulnerability Scoring System (CVSS) base score bugs. Ways that bug bounty can provide value beyond high-severity bugs include:
- Lower severity bugs chained together that enable a higher report impact
- The “discoverability” or how obvious a vulnerability is from the outside
- Asset discovery
- Bug fix validation to confirm remediation and look for bypasses
Every organization should define the goals for their program’s success. Consider the assets to be placed in scope, your organization’s industry and business model, controls or security testing already in place, and resource (time, money, and personnel) constraints.
Of course, report severity and volume also play a role in program success. In most cases, what programs should be aiming to do is encourage ethical hackers to submit vulnerabilities with business impact (often evaluated as the CVSS base score + environmental score) and reward hackers accordingly.
Aligning a bounty program’s reward table with true business impact is key to demonstrating a high return on investment (ROI) to stakeholders.
Set KPIs and Stick to Them
Bounty programs should use key performance indicators (KPIs) to define and measure success. Commonly used metrics for HackerOne customers include:
- Volume of duplicate reports
- Distribution of weakness types on individual assets or systems
- Mean Time to Resolve (MTTR)
Each of these metrics can help confirm success or hint at underlying issues in an organization’s security posture. For example, a given weakness type appearing disproportionately on a single product line across an attack surface may be indicative of a design flaw or dependency issue. Depending on your organization’s needs, there are many more possible KPIs you could use to maintain an efficient, effective program that supports cybersecurity objectives. However, if you’re unsure, the three KPIs listed above are a good starting point.
The best run bug bounty programs at peak maturity focus on soliciting novel and elusive (“NoEl”) vulnerabilities. These are the types of bugs that no automated tool or pentest methodology would likely uncover, such as advanced business logic issues, bug chains, and rare weakness types. Quantifying and tracking these types of bugs are an example of a more advanced program health metric.
No matter what level your program operates at, when comprehensive security controls are in place (e.g., DevSecOps, complete buy-in from stakeholders, hardened attack surface, etc.) program managers can be confident in raising bounty levels and adding additional attack surface to their program scope. These actions will attract increasingly skilled hackers and security researchers capable of finding NoEl bugs or zero day vulnerabilities with complex approaches, custom tools, and little-known techniques.
Understand Where Bug Bounty Fits
Top programs use bug bounty as a compass—a tool to help navigate an attack surface to find gaps in security testing coverage. Top-tier security teams leverage bounty programs as a performance monitoring system for their internal security strategy and a safety net that automatically deploys when other security testing systems and processes fail.
Bugs that slip through to production can be used to identify and resolve underlying challenges in security programs. Through the output of a bug bounty program, improvement opportunities can be identified in the software development life cycle (SDLC) to implement or bolster controls such as:
- Secure development training
- Static application testing
- Human code review
- Small pentest/security hygiene checks
Some programs involve pre-production environments as well. This isn’t ideal but might be done for many reasons such as hacker engagement, regulatory requirements, or even as an added layer of scrutiny and testing. If you can’t test production, consider a dedicated, mirrored testing environment instead or try to include other areas of crowdsourced security strength like new product releases, scattered secrets, or asset discovery on ever-changing attack surface.
No matter where it’s implemented, a well-integrated bug bounty program pinpoints challenges across the full vulnerability management landscape, including remediation practices, SLAs, stakeholder relationships, and pentesting habits. By learning from the outputs of KPIs, an organization can systematically tighten its security controls to the point where NoEl bugs become a more frequently reported class of vulnerability.
Level Up Your Bug Bounty Strategy
Designing, managing, and refining a program can be challenging, particularly if your organization is new to bug bounty.
HackerOne’s expert security advisors can help you uncover opportunities for improvement. Advisors offer bespoke engagement plans based on a program’s objectives and KPIs. Each plan built by the advisors is dynamic, harnessing the feedback loop they provide in conjunction with reported vulnerabilities. Based on these inputs, HackerOne advisors can recommend new or augmented security practices that will help to prevent less sophisticated vulnerabilities from slipping into live environments.
HackerOne’s Security Advisory Services already work with security teams from Amazon, the U.S. Department of Defense, Hyatt, Goldman Sachs, and the U.K. Ministry of Defence to build a set of ready-to-go initiatives and help implement them in line with business priorities.
To find out more about how a fully managed bug bounty or vulnerability disclosure program can help you address the security talent shortage, address visibility gaps within your security program, and cut down on remediation times, check out The Executive Guide to Human Security Testing.
The above post was originally published on the HackerOne Blog. Special thanks to Michiel Prins and Dane Sherrets for their review.
Continue reading part two.