Skip to main content

Observability

Atatus was implemented as our observability solution across all our products, recently it was integrated to Skims. This integration allows us to track key metrics about the behavior and performance of Skims on the different environments it runs, that includes manual CLI executions by a Developer or a member of the Open Source community, by ourselves using AWS Batch, or by the CI/CD pipelines.

What can be seen in Atatus?

In Atatus there's a dashboard for each of our products, in this case the Skims dashboard shows the results based on a time range filter of registered executions:

Dashboard for Skims

The metrics shown in the dashboard are based on time and performance of the product on each execution.

Each transaction is shown by clicking the "Transactions" tab, where you can see the details of each execution like start/end time, duration, status, errors if any, etc.

Types of transactions

Theres different types of transactions that can be tracked in Atatus for Skims, those types can be differentiated by the name:

  • $namespace: This $namespace is used in the configuration file for a Skims execution, we use this in Atatus to make relations across different transactions running on the same file/environment/repository.

  • skims_scan/$namespace: The main Skims execution, it is initialized at the start of a CLI session until it finishes. Here is registered information about the environment where it runs and metrics about the execution performance and time, E.g:

    CLI transaction

  • analyze_*/$namespace: Each CLI session creates different transactions individually for each module that is executed during de Skims session, each of this analyze_* will be named after the module being executed, the list of names could be sca, sast, cspm, sbom, apk, and dast, some of this are also separated in submodules depending on the kind of analysis being done. Inside of each analyze_* transaction you can see relevant information related with the parent transactions (skims_scan/*), E.g:

    Transaction tree Inside each sub-transaction you can also see the recorded timeline and a dashboard filtering a search by the $namespace, E.g:

    Filtered dashboard

Rationale

To be able to record the app execution correctly, and to reach every component on the execution, we need to have in mind how Skims works, this can explain why the transactions are being splitted when the app reaches different modules and submodules individually. Due to the fact that the app internally creates spaces on the Thread pool and Process pool, the traceability gets lost at some point, making it unable to track the main transaction to that point in the execution, that's why when it reaches the Tread pool/Process pool generation a new transaction is being created, which corresponds to the modules and submodules being executed, this way we achieve a complete visualization on what is going on in a complete execution of skims.

Side notes

There are some concrete cases, like the SBOM or CSPM modules that cannot generate detailed information about the environments on the child transactions (besides the $namespace that is always required), but this records correctly to the dashboard where you can see time and performance metrics. Here's an example of this grouped transaction, where you cannot get any Session traces, but you can still see relevant information about the executions related to that $namespace:

Filtered dashboard

extra note

when executions are manually triggered and not controlled by us, the traceability can lose expected data, but it can still record the behavior, time and performance of the session.