Reproducibility

Everything at Fluid Attacks is Open Source. This means that you can download, inspect, modify and enhance the source code that powers it all.

Going Open Source gives our customers the confidence that what we do is transparent and secure.

In order to verify the OWASP benchmark results we'll need to:

Meet the requirements for installing Fluid Attacks' scanner.
Install the scanner as explained in the installation guide.

Clone the OWASP Benchmark 1.2.

git clone https://github.com/OWASP-Benchmark/BenchmarkJava.git benchmark
cd benchmark

Create a config file as follows:

config.yaml
checks:
  - F004
  - F008
  - F021
  - F034
  - F042
  - F052
  - F063
  - F089
  - F107
  - F112
  - F130
namespace: OWASP
multifile: true
output:
  file_path: results/Benchmark_1.2-Fluid-Attacks-v2023.csv
  format: CSV
sast:
  include:
    - src/main/java/org/owasp/benchmark/testcode/
    - src/main/java/org/owasp/benchmark/helpers/DatabaseHelper.java
    - src/main/java/org/owasp/benchmark/helpers/SeparateClassRequest.java
    - src/main/java/org/owasp/benchmark/helpers/Thing1.java
    - src/main/java/org/owasp/benchmark/helpers/Thing2.java
    - src/main/java/org/owasp/benchmark/helpers/ThingFactory.java
    - src/main/resources/benchmark.properties

Execute:
```
m gitlab:fluidattacks/universe@trunk /skims scan config.yaml
```
This will take some time, enough for reading our blog post of the week

When this ends, the results file will be stored in a CSV file with the name of Benchmark_1.2-Fluid-Attacks-v2023 located in the results/ folder of the benchmark repository you just downloaded

This CSV has a line at the end with a summary of the number of vulnerabilities found by the scanner. We do not need this line to generate the scorecards and it needs to be removed. You can do so manually, or, executing the following commands:
```
cd results
sed -i '$d' Benchmark_1.2-Fluid-Attacks-v2023.csv
cd ..
```
To generate a scorecard for these results, you need to use an owasp plugin.

Right now, we are using a fork of the plugin, with some adjustments that allow it to run with the latest version of the scanner.

There is an open pull request at the OWASP Benchmark official repository to add support natively.

First, you need to install the plugin locally (Be sure to have maven installed in your system beforehand)
```
cd ..
git clone https://github.com/alejolagosm/BenchmarkUtils.git benchmark_utils
cd benchmark_utils
mvn install
cd ../benchmark
```
You need to add this yaml file to the benchmark repo in order to execute the plugin
benchmark_config.yaml
```
expectedresults: expectedresults-1.2.csv
```
And then run the plugin to generate the scorecards
```
mvn org.owasp:benchmarkutils-maven-plugin:create-scorecard -DconfigFile=benchmark_config.yaml
```

Open the results in your browser.

Example:

firefox scorecard/Scorecard_Home.html

Or:

google-chrome-stable scorecard/Scorecard_Home.html

If you want a simplified and fast version of all these steps, you can run a script that we have on the repo.

Execute:

m gitlab:fluidattacks/universe@trunk /skims/benchmark/owasp

In addition to generating the scorecards (You can open them as was previously explained), this script also generates a json file called "benchmark.json" with a summary of the results for the scanner.