Accuracy SLA

Description

F2 and F0.5 scores of at least 90% are achieved in reports of a client’s software risk exposure  and vulnerabilities, respectively.

Since January 2025, this service-level agreement (SLA) indicator seeks to address management’s pain (e.g., critical and overlooked issues) and developers’ pain (e.g., waste of time and effort). The Accuracy SLA is breached when both the F2-score-measured accuracy reporting risk exposure (henceforth, “severity accuracy”) and the F0.5-score-measured accuracy reporting vulnerabilities (henceforth, “quantity accuracy”) are below 90%, given the criteria presented below.

Criteria

For this SLA indicator to apply, your group must be subscribed to the Advanced plan . Additionally, false negative and false positive reports must meet the defined conditions to be considered as true. (See the articles False negatives  and False positives  for details.)

Details

In addition to the general measurement aspects , the following is taken into account to measure this SLA indicator:

• The risk exposure caused by vulnerabilities is calculated using the formula CVSSF = 4^(CVSS-4).
• Accuracy is calculated based on false positives, false negatives, and the F-Score model .
• In groups subscribed to black-box testing , vulnerabilities detectable only via source code review  are not considered false negatives.

Indicator calculation

Fluid Attacks does not measure the Accuracy SLA through only false positive or false negative rates, as these two are complementary to the assessment of security testing accuracy.

For instance, a good false positive rate may be achieved while no actual vulnerability is found (i.e., even if no alarms were raised, even legitimate ones). And a good false negative rate may be achieved when everything is flagged as a vulnerability (i.e., even if all possible false alarms were raised).

This SLA indicator involves two measures: Severity accuracy and quantity accuracy.

Accordingly, the former is equal to the value of the F2 score, which places significant weight on false negatives. Using this model and involving the CVSSF metric allows Fluid Attacks to measure accuracy in terms that matter most to your organization’s management, i.e., that critical issues have not been overlooked.

Quantity accuracy, in contrast, is equal to the value of the F0.5 score, in which false positives have more weight. This model, and using the number of vulnerabilities, allows Fluid Attacks to measure accuracy in a way that interests your development team the most, i.e., that reported issues to fix are indeed correct.

Severity accuracy calculation

1. Compute the CVSSF for each individual vulnerability using the formula CVSSF = 4^(CVSS-4).
2. Calculate the total CVSSF for true positives, false positives and false negatives.
3. Compute the following intermediate indicators:
   • Precision: true positives / (true positives + false positives)
   • Recall: true positives / (true positives + false negatives)
4. Compute the F2 score with the formula 5 * {[Precision * Recall] / [(4 * Precision) + Recall]}

Quantity accuracy calculation

1. Calculate the total number of vulnerabilities, differentiating by true positives, false positives and false negatives.
2. Compute the following intermediate indicators:
   • Precision: true positives / (true positives + false positives)
   • Recall: true positives / (true positives + false negatives)
3. Compute the F0.5 score with the formula 1.25 * {[Precision * Recall] / [(0.25 * Precision) + Recall]}