Automation Tests - Best Practices

It’s a danger to consider automation just as a routine task to be completed. Automation should be considered as an integral part of delivering a quality end product and the focus should be consistently on designing and developing relevant test suites. Below set of practices would come in handy.

Design Tests Before Automating Them

We need to keep in mind that test design has to be of prime importance. A well designed test is what finds the bug. The tests and scenarios needs to be clearly defined before start of test automation.

Foccus on the big picture

The objective of automation is not passing the test or making it work. The focus should be on producing a quality deliverable and hence the scope of testing should not be compromised for making the automation tests work.

Consistent and stable tests

We should be able to rely on the automation test results as the automation regression suite gives confidence to the team. Unstable tests that produce inconsistent results should not be part of the automation test suite.

Use testing techniques

Using testing techniques - Boundary Value Analysis, Equivalence Partitioning, and State Transition, while designing tests can greatly increase the value of the tests.

Manual testing vs Automation

When transitioning from a manual tester to automation test developer avoid trying to automate every single test scenario. Automation tests should be written based on the business critical scenarios and considering the objectives of test automation.

Review tests

To avoid redundant tests in the test suite it is a good practice to review the tests constantly (maybe in every sprint) so that the relevancy of the test suite can be maintained.

Create tests that take data sets as inputs

Hard coding data into the tests is never a good idea. When tests can accept different data combinations from data sheets , maintaining the tests becomes a much easier job.

Use data sets in tests according to business rules

Rather than using random data, tests should take data based on conditions. For example selection of date, can be tested by grouping input data into holidays, weekdays, weekends.

When I started with automation tests I did some research and by asking below questions I was able to set the expectations from automation and gave me some direction on the way forward.

Question 1 : Why to automate ?

The tests should generate value to the project. And the value automated tests can give are:

Quick feedback:

In the times of continuos integration running automated tests can let the developers know if the new changes have not caused any regression defects.

Tests that needs to be repeated in every cycle :

The business critical scenarios, end to end scenarios, the flows that user will follow (for example in an ecommerce site (select product, add to card,checkout, payment) should be automated.

Confidence for the team:

When a new feature is rolled out automation ensures that regression tests pass and gives confidence on the existing functionalities.

Automate monotonous tasks:

Manual testers can be freed from doing monotonous repeated tasks and can focus more on exploratory testing.

Question 2: When to automate ?

Enhancements to existing functionality

When existing functionalities are modified on account of defects or enhancements. In such cases the existing automated tests for the functionalities may just need to modified.

Stable system

Automation works best when the application is comparatively stable and well defined. The data set is known and the outputs are defined.

When at least one round of manual tests have been already run

Automation test results can be considered meaningful if the manual test has been passed before and the results are available for comparison.

Question 3: What to automate ?

Tests that need to be executed repeatedly in each test cycle.

Regression tests are the best candidates. Helps in time saving and QA can do more exploratory testing and be freed from doing boring tasks. Gives confidence that existing functionalities continue to function correctly after the new features are introduced.

The business critical cases and scenarios must be automated.

This can be figured out by using a Risk based approach. A matrix can be created

Challenges during automation test development:

Let us consider some challenges that we may run into while automating tests:

Automating a feature that’s getting developed: The automation test developer may end up spending more time on writing and updating tests as the feature under development keeps changing during the release cycle.

Maintaining and reviewing tests: A set of regression tests , if not maintained properly can have redundant tests which may not add any value in the current scenario.

Long test suites: If the tests suite takes a long time to finish running , it may cause bottleneck while considering continuous integration as the developer may become reluctant to commit frequently.

Considering the Why, When and What and keeping in mind the challenges we encounter we will be able to create automation test suites that will add value to the tests and project delivery as a whole.