An Introduction to Software Test Automation
Today,‘Test automation’ is a much talked about topic in the world of software testing and quality. In this article Saket Godase discusses the basics of test automation. What is test automation? He looks at the types oftest automation and also discusses the various myths about test automation.
An Introduction to Software Test Automation
Abstract
Though there are many formal definitions of software testing it basically boils down to “Software testing is a process used to identify the correctness, completeness and quality of developed computer software.”In reality, testing can never establish the correctness of the software. It can only find defects, not prove that there are none. The testing process is carried out either manually or using automation tools or an effective combination of both.
Manual Testing –is defined as developing and executing tests that rely primarily ondirect human interaction throughout the entire test case, especially interms of evaluating correctness and ascertaining test status.Automation Testing – isdefined as developing and executing tests that can run unattended,comparing the actual to expected results and logging status. The restof the paper will concentrate primarily on automation testing.
- Detailed Test Cases
- Predictable “Expected Results”
- Dedicated Test Environment
- And most importantly dedicated and skilled resources.
However, test automation can be cost-effective by following some of thetime-tested techniques given below.
- Choosea test tool that best fits the testing requirements of yourorganisation. A good place to start is “Automated TestingHandbook” available from the Software TestingInstitute, which covers all of the major considerations involved inchoosing the right test tool for your purposes.
- One-time tests are not worth automating. Only automate thosetests that are going to be repeated.
- Before starting off with Test Automation, identify the test casesthat can be automated.
- Concentrate on automating the majority of your tests, which arefairly straightforward.
- Leave the complex tests for manual testing.
- Nevermake the mistake of using Record and Playback as a method of automationtesting. This method is fraught with problems, and such scripts arevery costly to maintain. At best it can be used to perform a Proof ofConcept. In the next section, I will elaborate on why I feel sostrongly against using this approach.
- As far as possible use adata-driven automated testing methodology. This allows you to developautomated test scripts that are more “generic”, requiring only that theinput and expected results be updated. This methodology is ideal fortesting systems that are huge, complex and have varied testenvironments. The different data-driven methodologies will be discussedin detail later in this article.
- Most importantly, have a setof resources (hardware, software and people) dedicated to testautomation. Resources that share their time across manual andautomation testing rarely achieve effective test automation.
Record and Playback (A Test Automation Nightmare)
- Suchscripts contain hard coded values, which must change if anything at allchanges in the application. This creates an additional overhead, as thetesting team needs to update the scripts each time the data changes.
- Thesescripts contain very little error handling and recovery scenarios.Consequently, such scripts are not reliable (even in cases where theapplication doesn’t change) and often fail during playback for avariety of reasons such as pop-up windows, messages etc.
- The cost of maintaining these scripts is huge and quiteunacceptable.
- If the application under test changes, the tests must bere-recorded.
- Record/Playbackscripts work well only if the application under test is already workingas per the expectations. In essence, this defeats the purpose of testautomation.
- Record/Playback might work in some cases; if thetools used are intelligent enough to analyze the application under testand auto generate meaningful test data.
- Functional Decomposition Method:A system broken down into its components in such a manner that thecomponents correspond directly to system functions and sub functions.
- Key Word Driven Method:A system broken down into certain predefined keywords in such a mannerthat the keywords correspond directly to system functions and subfunctions.
Lets have a look at different approaches to test automation in detail.Functional Decomposition Method
Themain and most important concept behind functional decomposition is toseparate the data from the functions. This is achieved by employing ahierarchical architecture with a modular design.
The highestscript in the hierarchy is the controller / controlling script. This isthe engine of the test automation suite. Typically the Driver scriptinternally calls one or more scripts. These verification scripts arethe one’s, which perform the application testing. Apart from this,usually there are a few utility scripts that are used by the driver andverification scripts to perform a number of tasks.
- Controller/ Driver Scripts: perform initialization (if required), and then callthe business logic verification scripts in the desired order.
- Verification/ Test Case Scripts: perform the Business Functions. Usually, theentire business logic is embedded within these scripts.
- Subroutine/ Utility Scripts: perform application specific tasks required by twoor more Business scripts. The generic nature of these scripts cancontribute heavily to rapid script development.
- User-Defined Functions: general and/or application-specificfunctions. Such scripts can be shared across test automation suites.
Listing 1 will explain the above steps in much better way.
1. Open a web browser.
3. Verify that the login page is displayed correctly.
4. Write any errors to an Error Log.
5. Retrieve the login details from a data file / database.
6. Enter the user name and password and click on the login button.
7. Verify that the login is correct.
8. Write any errors to an Error Log.
9. Verify the main menu displayed.
10. Write any errors to an Error Log.
11. Logout from the application and close the web browser.
Advantages of using the Functional Decomposition Method:
- Amodular design and using files or records to both input and verifydata, reduces redundancy and amount of effort in creating automatedtest scripts.
- Scripts can be developed even when theapplication development is in progress. If the functionality changes,only the specific verification scripts containing the business logicneed to be updated.
- Script reusability is very high. As thescripts are written to test various business functions independently,they can easily be combined in a controlling script in order toaccommodate complex test scenarios.
- Maintaining the expected results for such scripts is very easy.
- Error handling is much more robust in these scripts. This allowsunattended execution of the test scripts.
- Since such scripts have very little interdependency they can beused in a plug and play manner.
- Technical and skilled personnel are required to create andmaintain the scripts.If the test automation suite is very large,then maintaining the expected results can be quite time consuming.
- The testing team needs to maintain both the scripts as well asthe test data.
- Propercare should be taken that the test data is maintained in appropriatefile formats. If the test data is updated using some non-standard tool,it can cause problems during test execution.
Consider the above test case of logging.
| Key_Word | Field/Screen Name | Input/Verification Data | Comment | Pass/Fail |
| Login_Page | Login | Login | Verify the active screen | |
| Input | Username and Password |
Test/Test | Verify that the current username and password is authentic. |
|
| Action | Press Login button | Login successful | The user is logged into the application successfully. |
- Easier to construct and maintain than the functiona ldecomposition scripts.
- Due to the diminished learning curve, even a non-technical resource can be much more productive in a short period of time.
- The test automation team can reuse the existing (manual) test cases.
- Maintaining the expected results for such scripts is very easy.
- Once the generic utility scripts are in place, building theverification scripts take a much shorter period of time.
- Building the core utility scripts requires a very degree of proficiency in the testing tool being used.
- Constructing scripts for complex scenarios could be time consuming.
- Without a good architecture in place these scripts have a much higher risk of failure.
A Gentle Reminder
- A good test environment should exist for the test automation team.Preferably it should be a replica of the production environment.
- A good hardware setup is equally important in any test automation project. It is essential to have dedicated for script execution.
- The expected data against which the checks are to be performed should be a known baseline.
- Skilled and experienced resources should be hired.
- Test cases to be automated should be comprehensive and up to date.
Myths and Realities
Myth: The tool is going to replace the testers.
Reality:This is hardly the case. One has to remember that it is indeed verydifficult to eliminate manual testing. In reality the test tool helpsthe testers to do their job in a much better manner. They relieve thetesters from performing mundane and boring tasks and free up their timeso that they can concentrate on the complex tasks.
Myth: The time span required to train all of the testers to usethe tool is too long.
Reality:If the correct methodology is followed then the learning curve can bereduced to a large extent and the testers can become productive withina short span of time.
Myth: The tool will be too difficult for testers to use.
Reality: With adequate training the testers can be easilymigrated to any new testing tool.
Myth: A single tester can easily perform a dual role ofmanual and automation.
Reality:This rarely works and usually is one of the biggest blunders anorganization makes. Ultimately, the tester gets burned out and both,the manual as well as automation testing suffers. For any testautomation project to succeed, it is imperative to have a set ofresources dedicated only to test automation.
Myth: Test Automation is very easy. Just record the scriptsat any given point of time and replay them whenever you want to.
Reality:This kind of attitude is probably most responsible for companiesdiscarding test automation projects and switching back to manualtesting. Automation testing requires a lot of planning, preparation andskilled resources.
Conclusion
Inthis article we saw a brief introduction to test automation. In thenext few articles, we will explore some of the widely used TestAutomation Tools and have a look at their advantages, disadvantages andusage. The bottom line is, with skilled resources, careful planning andsufficient preparation there is a very high probability that a testautomation project can turn out to be a huge success. Compromising onany of these can easily transform the Test Automation software into a”shelf-ware”.
Very informative!
Thank You!
Are you familiar with a tool name Automation Anywhere?
If so do you have any opinions on it?
Thanks,
Jason
pls answer me this quest sir
the range of values used in input data is ?
0-100
100-500
0-500
none of the above
The writer has explained the core concept of the subject matter comprehensively. It will surely help me go further deep into software automation. Thanks and keep it up.
Haroon, NY, USA
I wonder, how can you read this article seriously, when author does not know elementar english?
We can read this article seriously because we concentrate on the content in it and not on the grammer. We are here to learn about automation testing not to learn grammer. Please post comments if anything technically wrong here…
sorry ur statement i am feeling it is absolutely wrong ,if you dont feel the concept is not informative its fine compare if the information is correct followit because i been searching from some time and this article is quite good and don’t bother much about english.Moreover content is quite important.
That’s really a wonderful explanation about Test automation and its process as well as manual. Thanks a lot Saket…Keep posting…Cheers 🙂 Prayaga
That’s really a wonderful explanation about Test automation and its process as well as manual. Thanks a lot Saket…Keep posting…Cheers 🙂 Prayaga
I like the way the test tools were classified- it gives a clear overview and this is difficult to find; I would be very much interested to see a follow-up of this article where some more examples are given…
Best regards- and congrats for a very well written article !
I like the way the test tools were classified- it gives a clear overview and this is difficult to find; I would be very much interested to see a follow-up of this article where some more examples are given…
Best regards- and congrats for a very well written article !
I do agree with the Myth and Realities. During these recession time companies are following ‘Automation’ as saving method to replace the cost of having manual testers, which may not work for all the companies and projects.
I do agree with the Myth and Realities. During these recession time companies are following ‘Automation’ as saving method to replace the cost of having manual testers, which may not work for all the companies and projects.
can you provide some insight on making test plans without having features specs?
can you provide some insight on making test plans without having features specs?