Continuous Integration
Playwright tests can be executed in CI environments. We have created sample configurations for common CI providers.
#
Introduction3 steps to get your tests running on CI:
Ensure CI agent can run browsers: Use our Docker image in Linux agents. Windows and macOS agents do not require any additional dependencies.
Install Playwright:
Run your tests:
#
CI configurations#
GitHub ActionsThe Playwright GitHub Action can be used to run Playwright tests on GitHub Actions.
We run our tests on GitHub Actions, across a matrix of 3 platforms (Windows, Linux, macOS) and 3 browsers (Chromium, Firefox, WebKit).
#
DockerWe have a pre-built Docker image which can either be used directly, or as a reference to update your existing Docker definitions.
Suggested configuration
By default, Docker runs a container with a
/dev/shm
shared memory space 64MB. This is typically too small for Chromium and will cause Chromium to crash when rendering large pages. To fix, run the container withdocker run --shm-size=1gb
to increase the size of/dev/shm
. Since Chromium 65, this is no longer necessary. Instead, launch the browser with the--disable-dev-shm-usage
flag:- Sync
- Async
This will write shared memory files into
/tmp
instead of/dev/shm
. See crbug.com/736452 for more details.Using
--ipc=host
is also recommended when using Chromium—without it Chromium can run out of memory and crash. Learn more about this option in Docker docs.Seeing other weird errors when launching Chromium? Try running your container with
docker run --cap-add=SYS_ADMIN
when developing locally.dumb-init is worth checking out if you're experiencing a lot of zombies Chromium processes sticking around. There's special treatment for processes with PID=1, which makes it hard to terminate Chromium properly in some cases (e.g. in Docker).
#
Azure PipelinesFor Windows or macOS agents, no additional configuration required, just install Playwright and run your tests.
For Linux agents, you can use our Docker container with Azure Pipelines support for running containerized jobs. Alternatively, you can refer to the Dockerfile to see additional dependencies that need to be installed on a Ubuntu agent.
#
Travis CISuggested configuration
- User namespace cloning should be enabled to support proper sandboxing
- xvfb should be launched in order to run Chromium in non-headless mode (e.g. to test Chrome Extensions)
- If your project does not have
package-lock.json
, Travis would be auto-cachingnode_modules
directory. If you runnpm install
(instead ofnpm ci
), it is possible that the browser binaries are not downloaded. Fix this with these steps outlined below.
To sum up, your .travis.yml
might look like this:
#
CircleCIRunning Playwright on CircleCI requires the following steps:
Use the pre-built Docker image in your config like so:
If you’re using Playwright through Jest, then you may encounter an error spawning child processes:
This is likely caused by Jest autodetecting the number of processes on the entire machine (
36
) rather than the number allowed to your container (2
). To fix this, setjest --maxWorkers=2
in your test command.
#
JenkinsJenkins supports Docker agents for pipelines. Use the Playwright Docker image to run tests on Jenkins.
#
Bitbucket PipelinesBitbucket Pipelines can use public Docker images as build environments. To run Playwright tests on Bitbucket, use our public Docker image (see Dockerfile).
While the Docker image supports sandboxing for Chromium, it does not work in the Bitbucket Pipelines environment. To launch Chromium on Bitbucket Pipelines, use the chromiumSandbox: false
launch argument.
- Sync
- Async
#
GitLab CITo run Playwright tests on GitLab, use our public Docker image (see Dockerfile).
#
Caching browsersBy default, Playwright downloads browser binaries when the Playwright NPM package is installed. The NPM packages have a postinstall
hook that downloads the browser binaries. This behavior can be customized with environment variables.
Caching browsers on CI is strictly optional: The postinstall
hooks should execute and download the browser binaries on every run.
node_modules
are cached (Node-specific)#
Exception: Most CI providers cache the npm-cache directory (located at $HOME/.npm
). If your CI pipelines caches the node_modules
directory and you run npm install
(instead of npm ci
), the default configuration
will not work. This is because the npm install
step will find the Playwright NPM package on disk and not execute the postinstall
step.
Travis CI automatically caches
node_modules
if your repo does not have apackage-lock.json
file.
This behavior can be fixed with one of the following approaches:
- Move to caching
$HOME/.npm
or the npm-cache directory. (This is the default behavior in most CI providers.) - Set
PLAYWRIGHT_BROWSERS_PATH=0
as the environment variable before runningnpm install
. This will download the browser binaries in thenode_modules
directory and cache them with the package code. See installation docs. - Use
npm ci
(instead ofnpm install
) which forces a clean install: by removing the existingnode_modules
directory. See npm docs. - Cache the browser binaries, with the steps below.
#
Directories to cacheWith the default behavior, Playwright downloads the browser binaries in the following directories:
%USERPROFILE%\AppData\Local\ms-playwright
on Windows~/Library/Caches/ms-playwright
on MacOS~/.cache/ms-playwright
on Linux
To cache the browser downloads between CI runs, cache this location in your CI configuration, against a hash of the Playwright version.
#
Debugging browser launchesPlaywright supports the DEBUG
environment variable to output debug logs during execution. Setting it to pw:browser*
is helpful while debugging Error: Failed to launch browser
errors.
#
Running headfulBy default, Playwright launches browsers in headless mode. This can be changed by passing a flag when the browser is launched.
- Sync
- Async
On Linux agents, headful execution requires Xvfb to be installed. Our Docker image and GitHub Action have Xvfb pre-installed. To run browsers in headful mode with Xvfb, add xvfb-run
before the Node.js command.