Inpsyder David Remer belongs to one of our three development teams for customer projects. That’s why he often deals with testing and approving website projects. In his contribution to our Advent Calendar he describes the optimal way of such testing processes and lists 4 reasons why automated acceptance tests significantly simplify everyday work.
Acceptance Criteria: Alpha and Omega of Website Projects
Companies run websites with different goals. But no matter whether it is simply an advertising campaign, the direct sale of products, customer loyalty via community management or offering support via forums, in the end the website should work.
What people understand as “it works” depends on the respective goals and both the client and an agency develop them jointly on the basis of the catalog of requirements during the offer phase. This is a tedious process in which the agency and the potential client have to eliminate many inaccuracies and blank spaces. In the end there is usually a document which is the basis of the agency’s offer. This document also serves as the basis for acceptance by the client. That’s because it contains the acceptance criteria that decide whether the agency executed the order as agreed or not.
At Inpsyde, we offer our customers the opportunity to formalize these acceptance criteria in order to make them testable computer-assisted. In addition to the manual acceptance by the customer, there is an automated acceptance of all acceptance criteria, which have been worked out so far, takes place before each go-live. Especially for complex systems such as online shops or communities, we recommend such automation, as it guarantees the functioning of a website not only at the time of the go-live of a feature but also beyond that.
Four reasons for automated Acceptance Tests
Websites evolve
After the first go-live, further adaptations and extensions usually take place. Findings about the actual user behavior, the further development of the company, new product features and much more must be implemented on the website. So, you write acceptance tests for these innovations, too, and you test the new features. However, before the go-live, you don’t only perform automated tests of the new feature, but also of the existing ones. In this way, you can identify existing requirement conflicts and eliminate them at an early stage.
Let’s say, for example, that on a page visitors can log in and have to leave their email address. There are two different forms on this page that manage this registration. These are aimed at different target groups, but use the same registration routine in the end. After a lot of unrelated changes and some time passing by, the application requirement changes: In addition to the e-mail address, target group A should also enter the telephone number. The form for this target group is changed, as is the registration routine so that registrations without a telephone number are no longer permitted. The form for target group B was forgotten during the development and manual acceptance of the new feature. But not from the automated test for target group B, which only gives an email address and gets an error message back instead of a successful registration.
Open Source Software evolves
At Inpsyde we are proud of our contributions to the WordPress community. Again and again our developers support the development of different WordPress versions. We also make numerous plugins available under the GPL so that every WordPress user can quickly and easily adapt his own website to his needs. With WordPress, you can set up and adapt your own page flexibly.
Besides WordPress, we also regularly make use of established plugins that pass our internal code review in order to guarantee our customers a fast and flexible implementation of their requirements. Software, however, continues to evolve. The operation of a WordPress website is accompanied by the need to update WordPress. And you also need to update the plugins you use when new versions are released. This means that the codebase on which a page runs is subject to constant change.
Even if WordPress and numerous freely available plugins handle so-called “Breaking Changes” very carefully, such changes are never completely excluded. However, automated acceptance tests ensure that changes that affect the defined functionality of a website can be detected before they go live. This means that visitors are no longer beta testers.
Acceptance Tests are condensed Knowledge
Websites are complex applications. Let’s take a company blog as a supposedly simple example. Publishing an article means that it will be visible on the website, but where? There is a unique view of the article, i.e. a URL where this article can be found. In addition, the article will probably also be visible in different archives, for example under different categories and keywords. Maybe also on the start page. There it could be presented very prominently as long as it is a new article and then move further down the page.
At first this article may be presented with a very large picture, as time passes by the post might be presented with a smaller picture, just as one of many posts. So the blog post has a picture! Maybe the post also appears in the sidebar under a section “Recent posts”, but only if it is listed under the newest five posts. We could iterate this example endlessly. This is usually exactly the knowledge that is gradually being developed between our customers and our project managers and developers. This knowledge generation is already done by simply shouting “The picture is still too small” or “The post is not in the sidebar”!
If this knowledge is now scattered across many documents, this results in long-term costs. Acceptance tests reduce these costs. Project participants may change departments, get promotions or reorient themselves professionally. New employees are then assigned to the project and have to familiarize themselves with the project. Acceptance tests, on the other hand, withstand personnel changes in the project and ensure that knowledge of critical features is not forgotten.
Acceptance Tests are unambiguous
Two characteristics characterize acceptance tests: They are readable and machine-readable. The clarity of machine-readable acceptance criteria leads to a more precise formulation of these criteria and thus increases mutual understanding of the requirements. In this exemplary test, it becomes clear that only a summary of the latest article, but not the complete text of the article, would be found next to the title on the start page. This would only become visible if one would follow the link with the article title as the link text. On this post page you would now see the complete text, but not the summary of the post.
How do Acceptance Tests work?
At Inpsyde we use Behat to reliably roll out websites. Behat is a framework for behavior-driven development (BDD) written in PHP. As such, it allows us to formulate clear and testable examples of the expected behavior of the website. These expectations are formulated in Gherkin, a language that is easily readable by both machines and humans. As such, it can be used to clearly define acceptance criteria and include them in our test suite.
The test suite is integrated into our deployment process. This means that the code must first pass all tests of the suite before it can be rolled out. This is done automatically. As soon as we start the deployment process, the tests run. If these fail, the deployment will be aborted.
However, we do not only test immediately before the live event, but also while we are working on new features or an update. This allows us to identify problems and conflicts early and resolve them before we run the final test during deployment.
Automated Acceptance Tests – A Video Example
In the following video I show a short demonstration of automated acceptance tests:
From the console on the right side we start Behat. Behat now searches for all the tests and goes through the different scenarios. In this example we test a WooCommerce shop. We check whether a product can be added to the shopping cart from the start, shop and category pages.
In a second feature, we test whether the checkout works for users who are not logged in. First, we add a product to our shopping cart. Then we automatically go to the checkout and fill in all fields. After we have clicked the “Place Order” button, we expect to receive the success message “Order received”. We then log into the admin area to check whether we can also see the order there.
In the last feature we check if guests can register on the site. To do this we go to the corresponding page and create a user. Should this have worked, we will end up in the dashboard, where we should be welcomed friendly with “Hello max.mustermann”.
Thanks to the integration into our deployment, such test suites can now run in isolated test instances. This way errors are detected at an early stage.
Failed to submit:
one or more fields are invalid.