False negatives

Definition

A false negative is an erroneous report indicating that a vulnerability does not exist or is absent.

Specifically, Fluid Attacks considers false negatives as instances when it fails to report a vulnerability that was within the scope of its tests, and all the necessary inputs to reproduce this vulnerability were available.

False negatives are a serious problem in software development projects, as overlooked issues may be involved in leaks or successful cyberattacks.

Report false negatives

Fluid Attacks’ Continuous Hacking offers very low rates of false negatives, as it tests systems comprehensively with multiple techniques.

Accordingly, it includes high accuracy levels in its service-level agreement  (SLA). However, in the course of Continuous Hacking Advanced, your organization might find a vulnerability that Fluid Attacks did not flag. For the accuracy SLA  to apply, your false negative request must satisfy several criteria (see criteria in item 5 below).

To ensure a thorough investigation and swift resolution of any false negative, both parties shall adhere to the following protocol:

1. Report the potential false negative over any of the available communication channels.

2. Fluid Attacks shall assess the level of urgency in your report, that is, whether or not active exploitation of the vulnerability is identified. If so, Fluid Attacks shall prioritize helping you contain the incident.

3. Fluid Attacks shall give you the potential false negative form .

4. Fill out the form and send it to Fluid Attacks.

5. Fluid Attacks shall investigate the potential false negative as follows:

   • Fluid Attacks assigns a security analyst to thoroughly investigate the report.
   • The analyst attempts to reproduce the vulnerability in a controlled environment. If reproducible, the analyst attempts to map the vulnerability to the codebase and determines if the vulnerability constitutes a false negative based on the following criteria:

     • It is not a false negative if

       • Fluid Attacks was not in possession of both the source code and the corresponding environment;

       • the environment was not paired with the provided branch;

       • the environment was not stable for at least 80% of business days due to unsolved events ;

       • the data required for continuous testing flows (e.g., credentials, input fields) was incomplete or unusable;

       • remote access without human intervention (e.g., CAPTCHA, OTP) was not enabled;

       • the vulnerability predates the addition of the repository to Fluid Attacks’ testing, and no Health Check  was performed, or the post-Health Check review period had not yet been completed;

       • the vulnerability’s status changed from “Open” to “Closed” due to exclusions, deactivations, or removals;

       • the average monthly insertions per author exceed 8,000, calculated over a rolling two-year window counting backward from the date of the potential false negative report;

         The average monthly insertions are calculated as the total number of insertions made by all the client’s authors, divided by the total number of active months accumulated across those authors.

       • the potential vulnerability was reported within the tolerable window of 90 calendar days after its date of injection via a commit to the branch registered for tests in the platform (for vulnerabilities associated with SCA, the vulnerability will be considered a false negative only if at least 48 hours have elapsed since the CVE was published in any of the sources consulted by Fluid Attacks, as defined in “Vulnerability signature update ”);

       • the vulnerability was found in an unsupported  technology;

       • the vulnerability was found in an application that the client does not own  or is not authorized to submit for testing.

     • It is a false negative if Fluid Attacks had access to the code and the vulnerability was present during the testing period, but Fluid Attacks did not identify it.

6. Fluid Attacks communicates the investigation results.