When I started with automation tests I did some research and by asking below questions I was able to set the expectations from automation and gave me some direction on the way forward.

Question 1 : Why to automate ?

The tests should generate value to the project. And the value automated tests can give are:

Quick feedback:

In the times of continuos integration running automated tests can let the developers know if the new changes have not caused any regression defects.

Tests that needs to be repeated in every cycle :

The business critical scenarios, end to end scenarios, the flows that user will follow (for example in an ecommerce site (select product, add to card,checkout, payment) should be automated.

Confidence for the team:

When a new feature is rolled out automation ensures that regression tests pass and gives confidence on the existing functionalities.

Automate monotonous tasks:

Manual testers can be freed from doing monotonous repeated tasks and can focus more on exploratory testing.

Question 2: When to automate ?

Enhancements to existing functionality

When existing functionalities are modified on account of defects or enhancements. In such cases the existing automated tests for the functionalities may just need to modified.

Stable system

Automation works best when the application is comparatively stable and well defined. The data set is known and the outputs are defined.

When at least one round of manual tests have been already run

Automation test results can be considered meaningful if the manual test has been passed before and the results are available for comparison.

Question 3: What to automate ?

Tests that need to be executed repeatedly in each test cycle.

Regression tests are the best candidates. Helps in time saving and QA can do more exploratory testing and be freed from doing boring tasks. Gives confidence that existing functionalities continue to function correctly after the new features are introduced.

The business critical cases and scenarios must be automated.

This can be figured out by using a Risk based approach. A matrix can be created

Challenges during automation test development:

Let us consider some challenges that we may run into while automating tests:

Automating a feature that’s getting developed: The automation test developer may end up spending more time on writing and updating tests as the feature under development keeps changing during the release cycle.

Maintaining and reviewing tests: A set of regression tests , if not maintained properly can have redundant tests which may not add any value in the current scenario.

Long test suites: If the tests suite takes a long time to finish running , it may cause bottleneck while considering continuous integration as the developer may become reluctant to commit frequently.

Considering the Why, When and What and keeping in mind the challenges we encounter we will be able to create automation test suites that will add value to the tests and project delivery as a whole.