Lesson 22 / 25
Running in CI
Headless runs, retries and artifacts.
What a CI job needs
A CI job installs dependencies, installs browsers (npx playwright install --with-deps for Playwright; Cypress downloads its binary on install, and caching that binary speeds builds), starts or builds the app, runs tests headless, and uploads artifacts such as the Playwright HTML report and traces or Cypress screenshots and videos, even when tests fail. Configure retries only in CI: Playwright retries: process.env.CI ? 2 : 0; Cypress retries: { runMode: 2, openMode: 0 }. Use secrets for credentials, pin tool versions, and consider the official Docker images so browsers and fonts match between runs.
Fast feedback on every pull request
E2E suites earn their keep when they run headless in CI, in parallel, with artifacts that explain failures.
A GitHub Actions workflow for Playwright
Action versions change; check the docs for current ones.
name: e2e
on: [push, pull_request]
jobs:
playwright:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
- run: npx playwright install --with-deps
- run: npx playwright test
env:
E2E_USER: ${{ secrets.E2E_USER }}
E2E_PASSWORD: ${{ secrets.E2E_PASSWORD }}
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: playwright-report
path: playwright-report/
retention-days: 14A GitHub Actions job for Cypress
The official cypress-io/github-action installs, caches and runs (check the docs for the current major version).
jobs:
cypress:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: cypress-io/github-action@v6
with:
build: npm run build
start: npm start
wait-on: 'http://localhost:3000'
browser: chrome
- uses: actions/upload-artifact@v4
if: failure()
with:
name: cypress-screenshots
path: cypress/screenshotsAlways upload the report
Use a condition such as if: ${{ !cancelled() }} or if: failure() so the report and traces are uploaded exactly when you need them most: after a failing run.
Quick check: Why enable retries only in CI?
- CI cannot run a test only once
- Retries make tests run faster locally
- Local runs do not support assertions
- Locally you want to see failures immediately; in CI retries reduce noise while flaky tests are reported
Answer
Locally you want to see failures immediately; in CI retries reduce noise while flaky tests are reported — Retries are a safety net, not a fix.