CosmoCode

Running in CI/CD

Automate your Playwright tests in GitHub Actions and other CI systems

9 min readNode.js

Running in CI/CD

In the previous tutorial, we learned to run tests in parallel across browsers. Now for the final boss: making it all automatic.

Tests that don't run automatically might as well not exist. CI/CD integration ensures your tests run on every push, every pull request, and catch bugs before they hit production. Playwright was designed with CI in mind ā€” let's set it up.

GitHub Actions

GitHub Actions is the most popular CI for open source and many companies. Here's how to get Playwright running.

Basic Workflow

Create .github/workflows/playwright.yml:

name: Playwright Tests

on:
  push:
    branches: [main, master]
  pull_request:
    branches: [main, master]

jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          
      - name: Install dependencies
        run: npm ci
        
      - name: Install Playwright Browsers
        run: npx playwright install --with-deps
        
      - name: Run Playwright tests
        run: npx playwright test
        
      - uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 30

That's it. Push this file and your tests run automatically. Boom!

Understanding the Workflow

Let's break it down:

on:
  push:
    branches: [main, master]
  pull_request:
    branches: [main, master]

Runs on pushes to main and on all PRs targeting main.

runs-on: ubuntu-latest

Uses GitHub's Ubuntu runners. Playwright supports Linux, macOS, and Windows runners.

- name: Install Playwright Browsers
  run: npx playwright install --with-deps

Downloads browser binaries and system dependencies. The --with-deps flag installs OS-level libraries needed for browsers.

- uses: actions/upload-artifact@v4
  if: ${{ !cancelled() }}

Uploads the HTML report. The !cancelled() condition ensures artifacts upload even if tests fail.

Caching Browsers

Browser downloads are slow. Cache them:

- name: Cache Playwright browsers
  uses: actions/cache@v4
  id: playwright-cache
  with:
    path: ~/.cache/ms-playwright
    key: playwright-${{ runner.os }}-${{ hashFiles('**/package-lock.json') }}

- name: Install Playwright Browsers
  if: steps.playwright-cache.outputs.cache-hit != 'true'
  run: npx playwright install --with-deps

- name: Install Playwright system deps
  if: steps.playwright-cache.outputs.cache-hit == 'true'
  run: npx playwright install-deps

First run caches browsers. Subsequent runs reuse the cache and only install system dependencies.

Handling Artifacts

Upload Report and Traces

- uses: actions/upload-artifact@v4
  if: ${{ !cancelled() }}
  with:
    name: playwright-report
    path: |
      playwright-report/
      test-results/
    retention-days: 30

Viewing Artifacts

  1. Go to your GitHub repo
  2. Click "Actions" tab
  3. Click the workflow run
  4. Scroll to "Artifacts" section
  5. Download playwright-report
  6. Unzip and open index.html

Or use the Playwright CLI:

# After downloading the report
npx playwright show-report ./playwright-report

Artifact Size Tips

Artifacts can get large. Control the size:

// playwright.config.ts
export default defineConfig({
  use: {
    trace: 'retain-on-failure',      // Only keep traces for failures
    screenshot: 'only-on-failure',   // Only screenshot failures
    video: 'retain-on-failure',      // Only keep video for failures
  },
});

Sharding Across Jobs

"My tests take 20 minutes. Can I speed that up?"

Absolutely. Speed up CI by splitting tests across multiple machines:

name: Playwright Tests

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shard: [1/4, 2/4, 3/4, 4/4]
    
    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          
      - name: Install dependencies
        run: npm ci
        
      - name: Install Playwright Browsers
        run: npx playwright install --with-deps
        
      - name: Run Playwright tests
        run: npx playwright test --shard=${{ matrix.shard }}
        
      - uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: playwright-report-${{ matrix.shard }}
          path: playwright-report/
          retention-days: 30

  merge-reports:
    if: ${{ !cancelled() }}
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          
      - name: Install dependencies
        run: npm ci

      - name: Download reports
        uses: actions/download-artifact@v4
        with:
          path: all-reports
          pattern: playwright-report-*
          
      - name: Merge reports
        run: npx playwright merge-reports --reporter html ./all-reports
        
      - uses: actions/upload-artifact@v4
        with:
          name: playwright-report-merged
          path: playwright-report/

Four jobs run in parallel, then merge their reports. A 20-minute suite becomes 5 minutes. How cool is that?

fail-fast: false ensures all shards complete even if one fails ā€” you want to see all failures, not just the first.

Environment Variables

Using Secrets

Store credentials in GitHub Secrets, not in code:

- name: Run Playwright tests
  run: npx playwright test
  env:
    BASE_URL: ${{ secrets.STAGING_URL }}
    TEST_USER_EMAIL: ${{ secrets.TEST_USER_EMAIL }}
    TEST_USER_PASSWORD: ${{ secrets.TEST_USER_PASSWORD }}

Access in tests:

// playwright.config.ts
export default defineConfig({
  use: {
    baseURL: process.env.BASE_URL || 'http://localhost:3000',
  },
});

// In tests
const email = process.env.TEST_USER_EMAIL!;
const password = process.env.TEST_USER_PASSWORD!;

Different Environments

Run against staging on PRs, production on main:

- name: Run Playwright tests
  run: npx playwright test
  env:
    BASE_URL: ${{ github.event_name == 'push' && secrets.PROD_URL || secrets.STAGING_URL }}

Starting Your App

If tests need your app running:

- name: Build app
  run: npm run build

- name: Run Playwright tests
  run: npx playwright test

With webServer in your config:

// playwright.config.ts
export default defineConfig({
  webServer: {
    command: 'npm run start',
    url: 'http://localhost:3000',
    reuseExistingServer: !process.env.CI,
    timeout: 120_000,
  },
});

Playwright starts the server, waits for it, runs tests, then stops it.

Other CI Systems

GitLab CI

Create .gitlab-ci.yml:

stages:
  - test

playwright:
  stage: test
  image: mcr.microsoft.com/playwright:v1.40.0-jammy
  script:
    - npm ci
    - npx playwright test
  artifacts:
    when: always
    paths:
      - playwright-report/
      - test-results/
    expire_in: 1 week

CircleCI

Create .circleci/config.yml:

version: 2.1

jobs:
  playwright:
    docker:
      - image: mcr.microsoft.com/playwright:v1.40.0-jammy
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: npm ci
      - run:
          name: Run tests
          command: npx playwright test
      - store_artifacts:
          path: playwright-report
      - store_artifacts:
          path: test-results

workflows:
  test:
    jobs:
      - playwright

Azure Pipelines

Create azure-pipelines.yml:

trigger:
  - main

pool:
  vmImage: ubuntu-latest

steps:
  - task: NodeTool@0
    inputs:
      versionSpec: '20'
    displayName: 'Install Node.js'

  - script: npm ci
    displayName: 'Install dependencies'

  - script: npx playwright install --with-deps
    displayName: 'Install Playwright browsers'

  - script: npx playwright test
    displayName: 'Run Playwright tests'

  - task: PublishPipelineArtifact@1
    condition: always()
    inputs:
      targetPath: playwright-report
      artifact: playwright-report

Jenkins

In your Jenkinsfile:

pipeline {
    agent {
        docker {
            image 'mcr.microsoft.com/playwright:v1.40.0-jammy'
        }
    }
    stages {
        stage('Install') {
            steps {
                sh 'npm ci'
            }
        }
        stage('Test') {
            steps {
                sh 'npx playwright test'
            }
        }
    }
    post {
        always {
            archiveArtifacts artifacts: 'playwright-report/**', fingerprint: true
        }
    }
}

Docker

Using Playwright's Official Image

Playwright provides Docker images with browsers pre-installed:

# GitHub Actions
jobs:
  test:
    runs-on: ubuntu-latest
    container:
      image: mcr.microsoft.com/playwright:v1.40.0-jammy
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npx playwright test

Available images:

  • mcr.microsoft.com/playwright:v1.40.0-jammy ā€” Ubuntu 22.04
  • mcr.microsoft.com/playwright:v1.40.0-focal ā€” Ubuntu 20.04

Check the Playwright releases for the latest version.

Custom Dockerfile

If you need a custom image:

FROM mcr.microsoft.com/playwright:v1.40.0-jammy

WORKDIR /app

# Copy package files
COPY package*.json ./

# Install dependencies
RUN npm ci

# Copy test files
COPY . .

# Run tests
CMD ["npx", "playwright", "test"]

Build and run:

docker build -t my-playwright-tests .
docker run --rm my-playwright-tests

Pull Request Comments

Add test results as PR comments:

- name: Run Playwright tests
  id: playwright
  run: npx playwright test --reporter=json --reporter=html > results.json
  continue-on-error: true

- name: Comment on PR
  if: github.event_name == 'pull_request'
  uses: actions/github-script@v7
  with:
    script: |
      const fs = require('fs');
      const results = JSON.parse(fs.readFileSync('results.json', 'utf8'));
      const passed = results.suites.flatMap(s => s.specs).filter(s => s.ok).length;
      const failed = results.suites.flatMap(s => s.specs).filter(s => !s.ok).length;
      
      await github.rest.issues.createComment({
        owner: context.repo.owner,
        repo: context.repo.repo,
        issue_number: context.issue.number,
        body: `## Playwright Results\n\nāœ… Passed: ${passed}\nāŒ Failed: ${failed}`
      });

Best Practices

1. Run on PRs, Not Just Main

Catch issues before merge:

on:
  push:
    branches: [main]
  pull_request:

2. Use Retries in CI

Networks are flaky. Retry failed tests:

// playwright.config.ts
export default defineConfig({
  retries: process.env.CI ? 2 : 0,
});

3. Parallelize

Use sharding for large suites:

npx playwright test --shard=1/4

4. Keep Artifacts

Always upload reports and traces:

- uses: actions/upload-artifact@v4
  if: ${{ !cancelled() }}

5. Set Timeouts

Don't let hung tests block CI:

jobs:
  test:
    timeout-minutes: 30

// playwright.config.ts
export default defineConfig({
  timeout: 30_000,
});

6. Use the GitHub Reporter

Get inline annotations on PRs:

reporter: process.env.CI ? 'github' : 'list',

7. Separate Browser Installation

Cache browsers separately from dependencies:

- name: Cache Playwright
  uses: actions/cache@v4
  with:
    path: ~/.cache/ms-playwright
    key: playwright-${{ hashFiles('**/package-lock.json') }}

Complete Production Workflow

Here's a battle-tested GitHub Actions workflow:

name: E2E Tests

on:
  push:
    branches: [main]
  pull_request:
  workflow_dispatch:

env:
  CI: true

jobs:
  test:
    timeout-minutes: 30
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shard: [1/3, 2/3, 3/3]
    
    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: 'npm'
          
      - name: Install dependencies
        run: npm ci
        
      - name: Cache Playwright browsers
        uses: actions/cache@v4
        id: playwright-cache
        with:
          path: ~/.cache/ms-playwright
          key: playwright-${{ runner.os }}-${{ hashFiles('**/package-lock.json') }}
          
      - name: Install Playwright Browsers
        if: steps.playwright-cache.outputs.cache-hit != 'true'
        run: npx playwright install --with-deps
        
      - name: Install Playwright deps
        if: steps.playwright-cache.outputs.cache-hit == 'true'
        run: npx playwright install-deps
        
      - name: Run Playwright tests
        run: npx playwright test --shard=${{ matrix.shard }}
        env:
          BASE_URL: ${{ secrets.STAGING_URL }}
        
      - uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: playwright-report-${{ strategy.job-index }}
          path: |
            playwright-report/
            test-results/
          retention-days: 7

  report:
    if: ${{ !cancelled() }}
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          
      - run: npm ci
      
      - uses: actions/download-artifact@v4
        with:
          path: all-reports
          pattern: playwright-report-*
          
      - name: Merge reports
        run: npx playwright merge-reports --reporter html ./all-reports
        
      - uses: actions/upload-artifact@v4
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 30

Congratulations! šŸŽ‰

You did it! You've completed the entire Playwright for Node.js tutorial series.

Here's everything you've learned along the way:

  • Writing reliable tests with locators and assertions
  • Navigating and waiting for dynamic content
  • Organizing tests with describe blocks and hooks
  • Creating reusable fixtures and page objects
  • Debugging failures with Inspector, UI Mode, and traces
  • Handling authentication efficiently
  • Mocking APIs and testing them directly
  • Running tests in parallel across browsers
  • Automating everything in CI/CD

You went from zero to a production-ready test suite. That's seriously impressive.

Now go write some tests and break some bugs! šŸš€

Previousā† Parallel & Cross-Browser Testing