Overview
BackstopJS automates visual regression testing of your responsive web UI by comparing DOM screenshots over time.
It is integrated with Kalamuna Default Drupal project https://github.com/kalamuna/drupal-project currently the code is on regression-tests branch.
Setup
Install Drupal project as per https://github.com/drupal-composer/drupal-project/blob/8.x/README.md instructions do not deploy to git yet. Stop after
composer install
You should notice that after the installation you have backstop.js in the root of your project.
Open this file and in the scenarios section update URL and Reference URL
"onReadyScript": "puppet/onReady.js",
"scenarios": [
{
"label": "BackstopJS Homepage",
"cookiePath": "backstop_data/engine_scripts/cookies.json",
"url": "[VISUAL SITE MULTIDEV URL]",
"referenceUrl": "REFERENCE SITE URL",
"readyEvent": "",
"readySelector": "",
"delay": 0,
"hideSelectors": [],
"removeSelectors": [],
"hoverSelector": "",
"clickSelector": "",
"postInteractionWait": 0,
"selectors": [],
"selectorExpansion": true,
"expect": 0,
"misMatchThreshold" : 0.1,
"requireSameDimensions": true
}
],
Replace [VISUAL SITE MULTIDEV URL] with the environment that is attached to the visual branch of your project. If you don’t have visual branch create one and deploy to github and pantheon. Spin off multidev for it.
Replace REFERENCE SITE URL with the reference site URL in the beginning of your project this will be dev environment.
Run tests on your local
If you don’t have backstop installed on your local run
npm install -g backstopjs
Make sure that your version of the library matches the one used in CI. Currently CI uses 5.0.1
It can be updated in the docker image section of the .circleci/config.yml.
backstop:
docker:
- image: backstopjs/backstopjs:5.0.1
working_directory: ~/drupal
If you do then you can skip this step. Add backstop_data folder in the root of your project. Don’t commit it to Git.
commit backstop.json file to Git.
To run the test run these commands form the root of your project:
backstop init git checkout backstop.json backstop reference --config=backstop.json backstop test --config=backstop.json || touch backstop_failed.txt
You can see the report in the backstop_data folder.
Run tests in CI
Make sure your CircleCI integration is working when you are committing to master and you can see reports in CircleCI dashboard for your project.
Commit or merge PR into visual branch of your project this will trigger the additional workflow - test that will run backstop regression test against dev and visual multidev.
Once the workflow is done you can see the results in the Artifacts tab of Circle CI. Click in report.html and you will be redirected to the report page with the results of the test scenarios.
Additional configuration
backstop.json provides default configuration with 2 breakpoints: tablet and phone. You can update viewports section of the configuration and extend it with additional breakpoints.
"viewports": [
{
"label": "phone",
"width": 320,
"height": 480
},
{
"label": "tablet",
"width": 1024,
"height": 768
}
],
To test different pages not just homepage extend the configuration for scenarios:
"scenarios": [
{
"label": "BackstopJS Homepage",
"cookiePath": "backstop_data/engine_scripts/cookies.json",
"url": "[VISUAL SITE MULTIDEV URL]",
"referenceUrl": "REFERENCE SITE URL",
"readyEvent": "",
"readySelector": "",
"delay": 0,
"hideSelectors": [],
"removeSelectors": [],
"hoverSelector": "",
"clickSelector": "",
"postInteractionWait": 0,
"selectors": [],
"selectorExpansion": true,
"expect": 0,
"misMatchThreshold" : 0.1,
"requireSameDimensions": true
}
],
More details can be found in backstop documentation on GitHub.