Intro
With this document, I am going to show how a JavaScript web application developer can craft and maintain acceptance tests. I call the effort acceptance testing, but this type of testing is referred to by weaker terms:
- End-to-End Testing
- Integration Testing
- Automation Testing
Acceptance underscores the primary purpose: demonstrate to a customer that the entire app is behaving as specified. Acceptance pertains to both accepting a story and a regression test of past accepted stories.
Not a replacement for Unit & Integration Testing
Acceptance testing has the benefit of being a fast, visual proof that can test all supported browsers. But, yet, for the standard developer loop while implementing a story: design, edit, debug, it is too slow. Running an acceptance test by a developer is necessary before publishing a story as 'done'.
The testing pyramid above illustrates a project's testing effort. Unit Testing (UT) and the Integration (service) testing overshadows acceptance testing effort. The focus on UT and Integration testing is high coverage that runs fast. By contrast, the effort spent on acceptance testing should be low, limited to an all-success path and runs much slower.
ReactJS design naturally lends itself to crafting UT's than other design patterns. Ease in UT'ing will tend to encourage thorough coverage catching and avoiding bugs. Bugs that would have otherwise relied on acceptance or manual testing to find.
Acceptance test accumulation over the course of a project will become significant. This is an inevitability even if they are all-success focused. Acceptance testing must be as easy as possible to both craft and maintain over the course of a project. UT and integration testing must remain a higher priority.
Can Acceptance Tests be Low Maintenance?
Historically, no. The situation has spawned a cottage industry of non-programmer, test engineers. Instead of developers, they implement and maintain acceptance tests. This is unfortunate and introduces the following detractors that foil these efforts:
- Test engineers leverage brittle user recording tools to capture test scripts
- Significant sprint effort spent to fix tests broken due to development changes
- Sharing developer changes with test engineers is a low priority
- Acceptance test delivery lag the pace of developer story delivery
- Erosion of the end of sprint point of acceptance testing.
Any level of acceptance test has merit as an automated, end-to-end regression test. But, yet, it serves the customer better if acceptance tests are current to a sprint delivery story set. Keeping acceptance tests current also serves the developer during story implementation. It may help with quality of implementation, but it also helps ensure there is no regression break.
This document argues that acceptance testing can be low maintenance. The current developer has access to a health competition of JavaScript test drivers. Leveraging the PageObject abstraction when crafting tests also lowers maintenance.
Past acceptance testing relied heavily on programming languages not familiar to web application developers. E.g.: Java, C#, Python. Or written in a Domain Specific Language (DSL) focused on acceptance testing. Acceptance test DSL's aren't a bad idea. But, yet, they have some learning friction and tend to be so opinionated that they can frustrate efforts to cover an edge case.
JS Selenium Client
Now, a rich set of JavaScript packages1 are readily available from package installers like npm. They provide a Behavior-Driven Development (BDD) rigor that enhances test readability. Yet are not so opinionated that edge conditions can't be accommodated. Acceptance tests can now be written in the a programming language familiar to a web developer. They can react to edge cases, change test design, or refactor test with ease.
This document focus on using NightwatchJS, a worthy player in the space.