End to end tests

End to end tests (e2e) are focused in testing entire flows of our application to ensure that all components and interactions work together correctly. These tests guarantee that our application as a whole fulfills its requirements rather than simply testing individual elements in isolation.

Stack

We use Cypress for our e2e tests for its testing facilities, developer experience, automatic assertions, smart asynchronous DOM elements management, low flakiness and easy maintainability.

Running end to end tests

To execute all e2e tests in one go, use the following command from the universe project root:

m . /integrates/web/e2e run

You can also execute any of the single specs available with:

m . /integrates/web/e2e run <some_spec>

For example m . /integrates/web/e2e run test_trial

For an interactive experience, particularly useful during setup and result analysis, you can open the Cypress Electron application. To do this, run:

m . /integrates/web/e2e open

This will launch the Cypress Test Runner, allowing you to select and run tests from a user-friendly interface.

Adding custom commands

The Commands Cypress API is useful for adding new custom commands or overriding existing ones. For further detais, please check the Commands API documentation.

For adding new commands and being compliant with typing code quality, first extend the Chainable interface with the definition of your commands in the integrates/web/e2e/cypress/support/commands_def.ts file:

declare namespace Cypress {
  interface Chainable<Subject> {
    yourCustomCommand(arg: string): Chainable<Subject>;
    anotherCustomCommand(arg1: string, arg2: number): Chainable<Subject>;
    ...
  }
}

Then, add the implementation with the Commands.add method in the integrates/web/e2e/support/commands.ts file:

Cypress.Commands.add("yourCustomCommand", (arg: string) => {
  cy.log("Running your custom command");
  // your command code goes here
  ...
});

Cypress.Commands.add("anotherCustomCommand", (arg1: string, arg2: number) => {
  cy.log("Running another custom command");
  // more custom command execution
  ...
});

Write tests

Write your tests in a spec file in the integrates/web/e2e/specs directory with file extension test_{your test name}.cy.ts.

Once in your spec file, group your tests by the target URL to be visited for the test inside a description block as follows:

// test_{your-test-name}.cy.ts

describe("Test findings", () => {
  // Type here all tests which will start visisting /this/url
});

describe("Test findings", () => {
  // Type here all tests which will visiting /this/one/other/url
});

CI, users and roles

Cypress runs in CI with a different user (or users) per node instance. A single CI job can run tests with multiple users, but no tests should be executed with a same user in multiple CI nodes, such level of redundancy is not necessary. This is a parallelization strategy which does not depend on cypress.io proprietary tools but from a different job split strategy based on the user(s) mapped to execute them. A mapping between users an CI nodes can be found in integrates/web/e2e/support/user.ts and contains a mapping logic like the following:

User1, User2 and User3    (Should run in) => CI Node 1
User 4                    (Should run in) => CI Node 2
User 5 and User6          (Should run in) => CI Node 3
...

Which as raw code looks like:

CI_USER_MAP.set(IntegratesUsers.integratesmanager, 1);
CI_USER_MAP.set(IntegratesUsers.continuoushack2, 2);
...
CI_USER_MAP.set(IntegratesUsers.integratesuser2, 6);
CI_USER_MAP.set(IntegratesUsers.integratesadmin, 6);

When writing a test, it is required to choose which users will log in under the hood during cypress execution, and will concurrently run in the pipeline. For selecting the users which will run your test, user the runForUsers() function:

describe("Test an awesome aspect", () => {
  runForUsers(
    [
      // users for running the following tests
      IntegratesUsers.integratesmanager,
      IntegratesUsers.continuoushack2,
      IntegratesUsers.integratesadmin,
    ],
    (user) => {
      // Now you can place your tests and will only run for the previous users!
      it("Test feature 1", ()=>{...})
      it("Test feature 2", ()=>{...})
      it("Test feature 3", ()=>{...})
    }
  );
});

In this approach, we distribute tests across multiple CI instances, introducing redundancy in testing for various roles. This helps prevent unanticipated feature breakage due to changes impacting specific roles.

Code quality

Currently, cypress linting process is done through eslint-plugin-cypress and the typescript compiler targeted for type check only.

Once your e2e is ready, run its linting with:

m . /integrates/web/e2e/lint