OWASP Benchmark results

Last updated: Oct 27, 2026

---

Fluid Attacks uses the OWASP Benchmark

Fluid Attacks wants to give very accurate security testing results. To do this, Fluid Attacks keeps the errors of the automated tools and security analysts of its solution to a minimum. To measure accuracy, Fluid Attacks uses more than one benchmark, including the OWASP Benchmark Project.

The Open Worldwide Application Security Project (OWASP) is a non-profit foundation that helps to improve software security. It is an open, online community where each person can give resources and knowledge about application security (AppSec). In this community, developers and security professionals learn from each other and get information about the latest threats and best practices.

The OWASP Benchmark Project is a free Java test suite from 2015 that measures the accuracy, speed and coverage of automated software vulnerability detection tools. It helps developers and security professionals identify the strengths and weaknesses of different AppSec testing solutions. Thus, they can compare these solutions objectively.

With the OWASP Benchmark, you can examine different types of security testing tools that apply one of these techniques:

• Static application security testing (SAST)
• Dynamic application security testing (DAST)
• Interactive application security testing (IAST)

The benchmark measures how well a tool correctly identifies vulnerabilities (true positives) and secure code (true negatives), with a minimum of incorrect results (false positives and false negatives). This analysis helps you select the correct security testing tool for your software development lifecycle (SDLC).

OWASP Benchmark measuring system

The OWASP Benchmark uses two primary metrics to evaluate tool performance:

• True positive rate (TPR) or Sensitivity: This metric shows the percentage of real vulnerabilities that the tool correctly identifies. A higher TPR means the tool is more effective at finding real security risks.
• True negative rate (TNR) or Specificity: This metric shows the percentage of safe code that the tool correctly identifies as not vulnerable. A higher TNR means that the tool gives fewer false alarms.

As the previous section shows, two other result categories are possible. This page does not give more information about them:

• False positives: Incorrect reports that show code, inputs or ports as vulnerable (these are a problem, because your team examines issues that are not real)
• False negatives: Incorrect reports that show code, inputs or ports as secure (these are a serious problem, because they can cause a false sense of security and the deployment of vulnerable versions of your system into production)

This illustration shows the result categories. The larger circle contains what the tool reports as vulnerable:

Fluid Attacks' results

To make sure that its accuracy is high, Fluid Attacks tested its SAST automated vulnerability detection tool (a primary component of the plans) with the OWASP Benchmark test suite.

This scorecard compares Fluid Attacks' scanner with other vulnerability detection tools that the OWASP Benchmark test suite measured:

As the image above shows, Fluid Attacks' scanner gets better results than other vulnerability detection tools:

• 100% True positive rate: The scanner accurately identifies all the real vulnerabilities in the test suite.
• 0% False positive rate: The scanner does not flag secure code as vulnerable.

This performance gives a perfect OWASP Benchmark Score of 100%. This score is much higher than the industry averages:

• Almost 3x higher than the average score of commercial (paid) vulnerability detection tools
• More than 1.5x higher than the highest-scoring non-commercial (free) tool

Above all, Fluid Attacks cares about the same things as you:

• Finding all vulnerabilities before they impact your business
• Maintaining your team's efficiency with zero false positives

Reproduce Fluid Attacks' OWASP Benchmark results

To reproduce the OWASP Benchmark results, follow these steps:

1. Install Docker to run the primary CLI scanner of Fluid Attacks.

2. Download the Docker container and pull the image.

3. Clone the OWASP Benchmark v1.2 repository:

   git clone https://github.com/OWASP-Benchmark/BenchmarkJava.git benchmark
   cd benchmark

4. Create a configuration file using the following content:

   checks:
     - F004
     - F008
     - F021
     - F034
     - F042
     - F052
     - F063
     - F089
     - F107
     - F112
     - F130
   namespace: OWASP
   multifile: true
   output:
     file_path: /working-dir/results/Benchmark_1.2-Fluid-Attacks-v2024.csv
     format: CSV
   sast:
     include:
       - src/main/java/org/owasp/benchmark/testcode/
       - src/main/java/org/owasp/benchmark/helpers/DatabaseHelper.java
       - src/main/java/org/owasp/benchmark/helpers/SeparateClassRequest.java
       - src/main/java/org/owasp/benchmark/helpers/Thing1.java
       - src/main/java/org/owasp/benchmark/helpers/Thing2.java
       - src/main/java/org/owasp/benchmark/helpers/ThingFactory.java
       - src/main/resources/benchmark.properties

5. Execute the scan:

   docker run --rm -v .:/working-dir fluidattacks/sast:latest sast scan /working-dir/config.yaml

   When the scan completes, the scanner saves the results in a CSV file named Benchmark_1.2-Fluid-Attacks-v2024 located in the results/ folder of the cloned repository.

6. Install the OWASP plugin. You must have it to create a scorecard from the results. At this time, Fluid Attacks uses a modified version of the plugin to make sure that it is compatible with the latest scanner version. (A pull request to add native support is open in the official OWASP Benchmark repository.)

   To install the plugin locally (make sure you have Maven installed):

   cd ..
   git clone https://github.com/alejolagosm/BenchmarkUtils.git benchmark_utils
   cd benchmark_utils
   mvn install
   cd ../benchmark

7. Inside the benchmark repository, add a YAML file named benchmark_config.yaml with the following content to use the plugin:

   expectedresults: expectedresults-1.2.csv

8. Run the plugin to generate the scorecards:

   mvn org.owasp:benchmarkutils-maven-plugin:create-scorecard -DconfigFile=benchmark_config.yaml

9. Open the results in your browser. Examples:

   On Firefox:

   firefox scorecard/Scorecard_Home.html

   On Google Chrome:

   google-chrome-stable scorecard/Scorecard_Home.html