GitLab CI

Serenity/JS offers excellent support for GitLab CI: you can run your acceptance tests in CI, generate rich Serenity BDD reports, and publish both living documentation and JUnit-style test results.

In this guide, you'll learn how to configure your GitLab CI pipeline to use the official Serenity/JS Docker image and ensure a stable environment for running your Serenity/JS test scenarios.

Before you start

Set up a working Serenity/JS test suite using one of the official Serenity/JS Project Templates, or add Serenity/JS to your existing project by following the Serenity/JS installation guide.

If you're new to Serenity/JS, follow the tutorial to learn more about the framework.

Running Serenity/JS on GitLab CI

GitLab CI executes pipelines inside Docker containers.

To ensure stability of your execution environment, use a container with:

• The latest Long-Term Support (LTS) version of Node.js
• A recent Java Runtime Environment (JRE)
• Your desired web browsers and other dependencies, as per the Serenity/JS installation guide

The easiest way to do that is to:

• Use the official Serenity/JS Docker image directly, as it comes pre-configured with all the required dependencies
• Create a custom Docker container based on the official Serenity/JS Docker image to suit your needs

Configuring GitLab CI pipelines

GitLab CI pipelines are configured using a file called .gitlab-ci.yml, located at the root of your repository.

A typical Serenity/JS GitLab CI pipeline consists of at least two stages:

• test - responsible for running your Serenity/JS test scenarios and collecting their artifacts
• publish - responsible for publishing artifacts, such as the Serenity reports, to GitLab Pages or other destinations

Pipeline naming conventions

Neither GitLab CI nor Serenity/JS force you to use any particular naming convention or structure for your pipelines. In this guide, we use test and publish as that's a great starting point and a convention often followed by teams using Serenity/JS.

In the example below, you see the test stage with a job called serenity that:

• Uses the official Serenity/JS Playwright Docker image
• Installs the project dependencies using npm ci, which is the recommended way to install Node.js modules in CI environments
• Runs the test suite using npm test - all Serenity/JS Project Templates are pre-configured to run tests and generate reports via this standard script, and you can adjust it to your needs if necessary
• Preserves any artifacts under target, including the Serenity BDD reports, so that they can be reused in subsequent stages of the pipeline

The publish stage has a job called create-pages that:

• Declares a dependency on the serenity job to access its artifacts
• Uses the pages keyword to specify the location of the report
• Uses the rules keyword to publish the reports only when the pipeline runs on the default branch. This helps to avoid overwriting the main branch report when working on feature branches

stages: - test - publish serenity: stage: test # Use the official Serenity/JS Docker image image: ghcr.io/serenity-js/playwright:v1.57.0-noble script: - npm ci - npm test # Preserve test artifacts, even when the job fails artifacts: when: always paths: - target create-pages: stage: publish pages: publish: target/site/serenity dependencies: - serenity script: - echo "Publishing Serenity reports" # Publish reports to GitLab Pages only on the default branch rules: - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH # other configuration

.gitlab-ci.yml

If your team uses merge requests in your workflow, you might want to use GitLab Parallel Deployments to publish an instance of Serenity reports per merge request.

Generating JUnit reports

GitLab CI recognises test execution reports that conform to the JUnit XML standard. Serenity/JS integrates with all the native reporters offered by the supported test runners, including the ones producing JUnit reports, such as:

• Cucumber.js JUnit formatter
• Jasmine JUnit reporter
• Mocha JUnit reporter
• Playwright Test JUnit reporter
• WebdriverIO

To make GitLab CI recognise a JUnit-standard report produced by your Serenity/JS test suite and surface test results in your Merge Requests, you should:

• Use a native JUnit reporter appropriate for your test runner
• Configure artifacts:reports:junit to tell GitLab where to find the report.

For example, if you have configured your JUnit reporter to produce a report at target/junit-results.xml, you can configure your GitLab CI job as follows:

.gitlab-ci.yml

stages:
  - test

serenity:
  stage: test
  # ...
  artifacts:
    when: always
    paths:
      - target
    reports:
      junit: target/junit-results.xml

# other configuration

Pro Tip

Note that in the above example I use artifacts:when: always. This instructs GitLab to upload and analyse the test report even when the test run fails, which is exactly when you need your test reports the most 😊