Skip to main content

GitHub Actions

Serenity/JS integrates with GitHub Actions to run your acceptance tests in CI and publish interactive HTML reports with trend history, flaky test detection, and error clustering.

In this guide, you'll learn how to configure GitHub Actions workflows that:

  • Run Serenity/JS test scenarios using the official Serenity/JS Docker image
  • Publish the HTML Reporter output to GitHub Pages
  • Preserve execution history across builds for trend analysis

Before you start​

Set up a working Serenity/JS test suite using one of the official Serenity/JS Project Templates, or add Serenity/JS to your existing project by following the Serenity/JS installation guide.

If you're new to Serenity/JS, follow the tutorial to learn more about the framework.

Running Serenity/JS on GitHub Actions​

GitHub Actions workflows run on virtual machines or inside containers.

To ensure a stable, reproducible testing environment, use the official Serenity/JS Docker image as a job container. It comes pre-configured with:

  • The latest Long-Term Support (LTS) version of Node.js
  • All Playwright browser engines, plus stable Chrome and Edge
  • OpenJDK Java Runtime Environment (for teams also using Serenity BDD Reporter)
Why use a container?

Running tests inside a container eliminates environment drift between local development and CI. Browser versions, system dependencies, and font rendering all stay consistent — reducing flaky failures caused by infrastructure differences rather than product bugs.

Single-module project​

A single-module project has one test suite that runs in a single CI job. This is the most common setup for small to medium projects.

The HtmlReporter crew member handles both data collection and report generation automatically — no separate aggregation step is needed.

Basic workflow​

This workflow runs your tests and uploads the report as a build artifact:

.github/workflows/test.yml

Always upload the report

Use if: always() on the upload step so the report is available even when tests fail — that's exactly when you need it most.

Publishing to GitHub Pages​

To publish the report to GitHub Pages with trend history preserved across builds, add a deployment job that restores the previous report before generating the new one:

.github/workflows/test.yml

How trend history works:

  1. Before running tests, the workflow checks out the gh-pages branch into reports/serenity-js/. This restores the test-runs/ directory containing data from previous builds.
  2. When tests run, the HtmlReporter writes new data to reports/serenity-js/test-runs/<runId>/ and aggregates all available runs into the final report.
  3. After tests complete, the entire reports/serenity-js/ directory — including the new run — is deployed back to gh-pages.

Each build accumulates history. Use the maxHistory option to control how many runs are retained.

GitHub Pages setup

Ensure GitHub Pages is enabled for your repository (Settings → Pages → Source: "Deploy from a branch", branch: gh-pages). The first run will create the gh-pages branch automatically.

Multi-module project​

A multi-module project runs tests across multiple parallel jobs — for example, testing separate applications, browser variants, or sharded test suites. Each job produces its own test data, and a final job aggregates everything into one report.

Example: admin-ui and customer-ui​

Consider a monorepo with two UI modules, each with its own test suite:

my-project/
├── modules/
│   ├── admin-ui/
│   │   ├── spec/
│   │   └── playwright.config.ts
│   └── customer-ui/
│       ├── spec/
│       └── playwright.config.ts
└── package.json

Step 1: Configure each module to archive data only​

In each module's playwright.config.ts, use TestRunArchiver instead of the full HtmlReporter. This collects test data without generating the final report:

modules/admin-ui/playwright.config.ts
import { defineConfig } from '@playwright/test';
import type { SerenityFixtures, SerenityWorkerFixtures } from '@serenity-js/playwright-test';

export default defineConfig<SerenityFixtures, SerenityWorkerFixtures>({
    testDir: './spec',

    reporter: [
        ['line'],
        ['@serenity-js/playwright-test', {
            crew: [
                ['@serenity-js/html-reporter:TestRunArchiver', {
                    outputDirectory: './reports/serenity-js',
                    specDirectory: './spec',
                }],
            ]
        }]
    ],
});

Step 2: Run tests in parallel jobs​

Each job uploads its test-runs/ data as a build artifact:

.github/workflows/test.yml

What happens:

  1. The test job matrix runs both modules in parallel. Each uploads its test-runs/ directory as a separate artifact.
  2. The report job downloads all artifacts, restores the previous gh-pages report for trend history, then runs aggregate to combine everything into a single report.
  3. The combined report is deployed to GitHub Pages (on main only).
fail-fast: false

Set fail-fast: false on the test matrix so all modules run to completion even if one fails. This ensures the aggregated report contains results from all modules, not just the ones that happened to finish first.

Adapting for other multi-job patterns​

The same approach works for:

  • Sharded Playwright tests — replace the module matrix with Playwright's --shard flag
  • Multi-browser testing — matrix on browser names, each producing its own test run data
  • Monorepo CI — each package in a workspace uploads independently

The key pattern is always the same: archive data in parallel → aggregate in a final step.

Controlling report history​

The maxHistory option limits how many test runs are retained. Without it, the report grows indefinitely.

['@serenity-js/html-reporter', {
    outputDirectory: './reports/serenity-js',
    maxHistory: 20,  // keep the last 20 runs
}]

For the CLI aggregate command:

npx @serenity-js/html-reporter aggregate \
  --input "..." \
  --output ./reports/serenity-js \
  --max-history 20
Choosing a maxHistory value

A good starting value is the number of builds you'd realistically compare when investigating a regression. For most teams, 10–20 runs provides useful trend data without bloating the GitHub Pages deployment.

Using Serenity BDD Reporter​

If your team already uses Serenity BDD or prefers its multi-page report format, your workflow needs Java and a separate report generation step (serenity-bdd run) after tests complete.

The official Serenity/JS Docker image includes OpenJDK, so no additional Java setup is required when running inside the container.

.github/workflows/test.yml

The key differences from the HTML Reporter workflow:

  • npm test invokes npm-failsafe to chain test execution and report generation
  • serenity-bdd run requires Java (provided by the Docker image)
  • The report output goes to target/site/serenity/ rather than reports/serenity-js/

→ Learn more: Serenity BDD Reporter guide | Migrating to the HTML Reporter

Learn more​