Platform audit logs (in-house)

Rationale

Audit logging for Fluid Attacks’ platform  is performed by an in-house implementation . Such implementation uses AWS DynamoDB Streams and Lambda triggers  to update a separate database  that is completely focused on providing real-time traceability on every single entry within the platform.

This approach gives us full granularity on every change an item has been through since it was created until it was deleted. History for deleted items is retained for six months before deletion.

The main reasons why we chose it over other alternatives are the following:

• It is custom-made for our specific needs.
• It has scaled well in terms of performance throughout the years.
• It gives us the ability to run analytics over a highly granular history.
• It allows us to generate highly detailed reports for our clients.
• It provides full traceability when investigating incidents is required.

Alternatives

As the application evolved, so did our interest in using a third-party audit logging solution. Below are the ones we have reviewed, ordered from most to least interesting based on our specific needs.

AWS SQS + Lambda + Aurora Serverless

• This is another in-house approach that aims to replace the current one.
• This approach would focus on allowing us to log relevant changes performed by users in a consistent and unified way. The backend would send a notification to SQS  for every change within the platform. Once there, a Lambda  would process such a change and write it to Aurora . An Audit Logs view would be provided to organization administrators within the platform.
• Being custom-made, it would allow us to adapt audit logging to any needs we may find in future.
• It would most likely cost more than WorkOS due to the fact that it would be implemented by us.
• SLAs would not apply here, as it would be part of our backend.
• Infrastructure handling would be completely performed by us.
• aioboto3  would be used for communication with SQS, Lambda and Aurora.
• A Terraform provider would not be needed.
• Log streaming could be possible if we decide to support it.
• Exporting logs could be possible if we decide to support it.

WorkOS

• WorkOS  provides SaaS solutions to requirements usually needed by enterprise applications.
• Due to its broad focus, it provides services for administrative portals , authentication , authorization , etc. Here we will be discussing its Audit Logs  capability.
• Being a third-party provider, we would have to adapt ourselves to the features it supports and its limitations.
• It has a per-organization pricing model  that scales properly for our use case.
• It has a >99% availability SLA .
• It is a SaaS that automatically handles user databases, scaling, security, redundancy, and disaster recovery, among others.
• Its Python library supports asynchronous environments .
• It does not have a Terraform provider.
• It supports log streaming  to AWS S3 and other SIEM providers.
• It supports exporting logs  to CSV format.

Usage

We use an in-house implementation for audit logging flows at Fluid Attacks’ platform .

Other in-house dependencies

• Platform authentication 
• Platform authorization