Skip to main content

GitLab CI

Serenity/JS integrates with GitLab CI to run your acceptance tests and publish interactive HTML reports with trend history, flaky test detection, and error clustering.

In this guide, you'll learn how to configure GitLab CI pipelines that:

  • Run Serenity/JS test scenarios using the official Serenity/JS Docker image
  • Publish the HTML Reporter output to GitLab Pages
  • Preserve execution history across builds for trend analysis

Before you start​

Set up a working Serenity/JS test suite using one of the official Serenity/JS Project Templates, or add Serenity/JS to your existing project by following the Serenity/JS installation guide.

If you're new to Serenity/JS, follow the tutorial to learn more about the framework.

Running Serenity/JS on GitLab CI​

GitLab CI executes pipelines inside Docker containers.

To ensure a stable, reproducible testing environment, use the official Serenity/JS Docker image as your job image. It comes pre-configured with:

  • The latest Long-Term Support (LTS) version of Node.js
  • All Playwright browser engines, plus stable Chrome and Edge
  • OpenJDK Java Runtime Environment (for teams also using Serenity BDD Reporter)

The easiest way to get started is to:

Why use a container image?

Running tests inside a pre-configured container eliminates environment drift between local development and CI. Browser versions, system dependencies, and font rendering all stay consistent — reducing flaky failures caused by infrastructure differences rather than product bugs.

Single-module project​

A single-module project has one test suite that runs in a single CI job. This is the most common setup for small to medium projects.

The HtmlReporter crew member handles both data collection and report generation automatically — no separate aggregation step is needed.

Basic pipeline​

A typical Serenity/JS GitLab CI pipeline consists of at least two stages:

  • test — runs your test scenarios and preserves the report output
  • pages — publishes the report to GitLab Pages
.gitlab-ci.yml

Always preserve artifacts

Use artifacts: when: always so the report is available even when tests fail — that's exactly when you need it most.

Pipeline naming conventions

Neither GitLab CI nor Serenity/JS force you to use any particular naming convention for your pipelines. In this guide, we use test and pages as that's a great starting point and a convention often followed by teams using Serenity/JS.

Preserving trend history​

To enable trend analysis across builds, use GitLab CI's cache to persist the test-runs/ directory. The cache writes only on the default branch, so feature branches can read historical data but don't pollute it:

.gitlab-ci.yml

On the default branch, the cache restores previous test-runs/ data, the HTML Reporter aggregates all runs, and the pages job publishes the result. Old runs are pruned based on maxHistory.

On feature branches, tests still run and the report is available as a downloadable artifact (with read-only access to main's history) — but the cache isn't updated and Pages aren't overwritten.

Multi-module project​

A multi-module project runs tests across multiple parallel jobs — for example, testing separate applications, browser variants, or sharded test suites. Each job produces its own test data, and a final job aggregates everything into one report.

Example: admin-ui and customer-ui​

Consider a monorepo with two UI modules, each with its own test suite:

my-project/
├── modules/
│   ├── admin-ui/
│   │   ├── spec/
│   │   └── playwright.config.ts
│   └── customer-ui/
│       ├── spec/
│       └── playwright.config.ts
└── package.json

Step 1: Configure each module to archive data only​

In each module's playwright.config.ts, use TestRunArchiver instead of the full HtmlReporter:

modules/admin-ui/playwright.config.ts
import { defineConfig } from '@playwright/test';
import type { SerenityFixtures, SerenityWorkerFixtures } from '@serenity-js/playwright-test';

export default defineConfig<SerenityFixtures, SerenityWorkerFixtures>({
    testDir: './spec',

    reporter: [
        ['line'],
        ['@serenity-js/playwright-test', {
            crew: [
                ['@serenity-js/html-reporter:TestRunArchiver', {
                    outputDirectory: './reports/serenity-js',
                    specDirectory: './spec',
                }],
            ]
        }]
    ],
});

Step 2: Run tests in parallel and aggregate​

Use GitLab CI's parallel:matrix to run modules in parallel, then aggregate in a final job:

.gitlab-ci.yml

What happens:

  1. The test job runs both modules in parallel via parallel:matrix. Each produces a test-runs/ directory as an artifact.
  2. The aggregate job downloads all test artifacts, restores the previous GitLab Pages report for trend history, then combines everything into a single report.
  3. The pages job deploys the combined report to GitLab Pages (on the default branch only).
Parallel deployments

If your team uses merge requests, you can use GitLab Parallel Deployments to publish a separate report instance per merge request.

Controlling report history​

The maxHistory option limits how many test runs are retained. Without it, the report grows indefinitely.

['@serenity-js/html-reporter', {
    outputDirectory: './reports/serenity-js',
    maxHistory: 20,  // keep the last 20 runs
}]

For the CLI aggregate command:

npx @serenity-js/html-reporter aggregate \
  --input "..." \
  --output ./reports/serenity-js \
  --max-history 20
Choosing a maxHistory value

A good starting value is the number of builds you'd realistically compare when investigating a regression. For most teams, 10–20 runs provides useful trend data without bloating the cache or deployment.

Generating JUnit reports​

GitLab CI natively recognises test reports that conform to the JUnit XML standard and surfaces test results in Merge Requests.

Serenity/JS integrates with the native reporters offered by all supported test runners, including those producing JUnit reports:

To make GitLab surface test results, configure artifacts:reports:junit to point to the JUnit XML output:

.gitlab-ci.yml
serenity:
  stage: test
  # ...
  artifacts:
    when: always
    paths:
      - reports/serenity-js
    reports:
      junit: reports/junit-results.xml
Pro Tip

Use artifacts: when: always so GitLab uploads and analyses the JUnit report even when tests fail — which is exactly when you need it most 😊

Using Serenity BDD Reporter​

If your team already uses Serenity BDD or prefers its multi-page report format, your pipeline needs a separate serenity-bdd run step after tests complete to generate the HTML report.

The official Serenity/JS Docker image includes OpenJDK, so no additional Java setup is required.

.gitlab-ci.yml

→ Learn more: Serenity BDD Reporter guide | Migrating to the HTML Reporter

Learn more​