Incident management

Last updated: Nov 4, 2026

---

Definition

Confirmed incidents include malfunctions, performance degradation, service disruptions, or errors in Fluid Attacks products or components. These incidents must have affected a minimum of two users, or they must have the potential to affect a minimum of two users, if someone reported them or not. This criterion applies to all incidents, including internal incidents that affect pentesters or other team members.

Incident Manager

The Incident Manager is responsible for the creation and management of incidents from the time of detection until the publication of the incident postmortem.

Assigned developer

The assigned developer is responsible for the resolution of the issue. The assigned developer must address the case quickly and with the highest priority. The assigned developer must also keep the Incident Desk updated about all progress, obstacles, and other events during the process. Also, the assigned developer gives all the necessary information to the Incident Manager. With this information, the Incident Manager can identify and correctly document the root cause and write the postmortem.

Detection

Detection is when an external or internal user identifies an incident and then reports it for publication and resolution.

Users usually report incidents through [email protected]. Team members can also report them through an issue, the internal chat, a direct email, or other means of contact.

Issue

When an incident is detected, the help area creates a detailed issue that describes the problem. The Incident Manager is responsible for the assignment of a developer to resolve it. Most issues come from the help area, but other Fluid Attacks employees can also create them.

Publication

After the Incident Manager assigns a developer, the Incident Manager must quickly open the incident panel and create the publication with these fields:

• Incident name: It shows the nature of the problem clearly and concisely.
• Incident status: At publication, the status should always be "Identified."
• Message: A description of what the incident affects and how.
• Workaround: A description of how to avoid the issue with alternative methods, with a link to these instructions in the documentation.
• Affected components: Checkboxes to select the affected components. These are the components:
  • Platform: The platform and API
  • Web: The website with all the commercial information about Fluid Attacks and its products
  • Docs: The user documentation and internal documentation
  • CI Gate: The tool that verifies if there are unaccepted vulnerabilities in software versions and can break the build to prevent the publication of these versions
  • Cloning: The tool that downloads the code repositories of the end users and some other utilities that the pentesters of Fluid Attacks require
  • Scanning: The security vulnerability detection tools that scan source code, infrastructure, and applications and report vulnerabilities
  • Extensions: IDE extensions to show reported vulnerabilities and fix them with AI suggestions
  • Mailing: The tool that sends notifications to the users of Fluid Attacks products
• Severity: The options are major outage, partial outage, degraded performance, and operational.
• Notifications: A checkbox that stays selected to keep subscribers informed.
• Long-running incident reminders: It keeps admins updated while the incident continues. This checkbox stays selected, and the reminders occur every hour.

After the Incident Manager creates the incident, it is immediately published on status.fluidattacks.com, and subscribers get a notification.

Incident Desk

The Incident Desk is an internal chat group with a minimum of four members with extensive knowledge of the business. Its purpose is to support the developer who must remediate the incident.

When a developer is assigned to resolve an incident, the Incident Manager adds them to the group. Then, the Incident Manager posts a description of the problem, the name of the accountable developer, the issue URL, and the incident URL. This starts a discussion thread about the incident.

The assigned developer is expected to use 100% of their working time to resolve the incident. Also, they must quickly tell the group about all mitigations, solutions, obstacles, or other relevant updates about the case.

After the incident is successfully resolved, the Incident Manager removes the developer from the Incident Desk group.

Follow-up

The Incident Manager is responsible for contact with the developer who resolves the incident. The Incident Manager tracks its progress, keeps the Incident Desk updated about the status of the incident, and gives support if obstacles occur.

For long-running incidents, the Incident Manager must continuously report the incident status. The Incident Manager can do this through the crisis desk, through responses to the interval notifications of the status page, or directly through incident updates.

Also, the Incident Manager gives a short update every hour on the status page. This update gives clear and relevant information about the current status of the incident.

Closure

When the incident is resolved, the Incident Manager completes the process. The Incident Manager changes the incident status to "Resolved" and writes a concise two-line message. This message says that the incident is resolved and that the affected component operates normally again.

The status page shows the updated incident state, and subscribers get a notification. Then, the Incident Manager can write a postmortem.

Postmortem

The postmortem is a public report about the incident. It has four specific sections: Impact, Cause, Solution, and Conclusion. The Incident Manager must write the postmortem when the incident is closed ("Resolved").

The Incident Manager is responsible for writing and publishing the postmortem with the option in the incident management panel. To write it, the Incident Manager investigates the incident with the support of the developer who resolved it and of other stakeholders who can give the necessary information.

The postmortem should be easy for all readers to understand. It should avoid technical details, and it should be clear and concise. The sections below describe each of its sections.

Impact

This section should include these items:

• How many users were affected and how
• The timestamp of the timeline of the incident: "UTC-5 yy-mm-dd hh:mm" (the date of the merge request that caused the incident)
• How the incident was discovered: proactively (by a person of the Fluid Attacks team) or reactively (by a customer)
• The time to detect the incident: "Time to detect was elapsed_time". The time to detect (TTD) is the time from the date when the failure gets to production to the date when the failure is reported through a support channel of Fluid Attacks. The TTD always has a precision of a maximum of one decimal place. The unit of measurement for elapsed_time must be as follows:
  • If elapsed_time < 1 hour, then "X minutes"
  • If 1 day > elapsed_time > 1 hour, then "X hours"
  • If 1 month > elapsed_time > 1 day, then "X days"
• The time to fix the incident: "Time to fix was elapsed_time". The time to fix (TTF) is the time from the date when the failure is reported through a support channel of Fluid Attacks to the date when the incident is resolved. The TTF always has a precision of a maximum of one decimal place. The unit of measurement for elapsed_time must be as follows:
  • If elapsed_time < 1 hour, then "X minutes"
  • If 1 day > elapsed_time > 1 hour, then "X hours"
  • If 1 month > elapsed_time > 1 day, then "X days"
• Who reported the incident, and through which medium
• The total exposure time of the incident: "Window of exposure was elapsed_time". The window of exposure (WOE) is the time from the date when the failure gets to production to the date when the incident is resolved. The WOE always has a precision of a maximum of one decimal place. The unit of measurement for elapsed_time must be as follows:
  • If elapsed_time < 1 hour, then "X minutes"
  • If 1 day > elapsed_time > 1 hour, then "X hours"
  • If 1 month > elapsed_time > 1 day, then "X days"
• The reference to the issue that was created to resolve the incident. This reference is essential.

Cause

This section requires a careful investigation to identify the root cause or causes of the incident. At this point, it is essential to work with the developer who is responsible for the remediation, and with any other stakeholder who can give the necessary information to identify and describe the cause.

At the end of this section, the root cause of the incident should be clear. For this section, the 5 Whys Technique can be helpful.

If a merge request introduced the cause, add the URL of that merge request as a reference (always the merge request URL, not the commit URL).

Solution

This section requires work with the developer who remediated the incident. Its purpose is to clearly explain the remediation process.

If the resolution of the incident required code changes, add the URL of the merge request (not the commit) that fixed the problem as a reference.

Conclusion

In this section, the Incident Manager should document the lessons learned during the incident process. This includes the architectural aspects that let the problem get to the user (production environment), the actions of the responsible developer to remediate the root cause, and, at the end, a taxonomy term or a taxonomy tree.

Taxonomy term

These terms are tags to categorize the different incidents that can occur. Their purpose is to summarize and classify the nature of a problem or error in a concise and systematic way. This gives a quick and structured understanding of why an issue got to the production environment.

Taxonomy tree

A taxonomy tree is a hierarchical structure that describes the root cause of a complex problem. It is used when the incident comes from a series of interconnected errors or failures, and not from one clear cause.

The hierarchical structure helps to divide the problem into its parts and connections. Thus, it is easier to identify the main issue. The structure is as follows:

Principal Failure < Primary Cause < Secondary Cause < N Cause

Read the structure as follows:

Failure occurred due to < Which was caused by < Which was in turn caused by

This is an example:

DATA_QUALITY < FAILED_MIGRATION

Incidents page

For transparency, and to keep stakeholders informed, an incidents page is available.

Other transparency measures

• Complaint management
• Data leakage policy
• Ethics and compliance hotline
• Help channel
• Information security responsibility
• Quality policy
• Status page
• Testing our technology
• Vulnerability releasing