Parent page: Test Automation Problems

Requirements

 Trying to do something without understanding what you are supposed to get often produces unexpected results. With regards to computer software the expectations must be even more precise and specific, as all the functionalities are built from atomic instructions executed by a machine. Test Automation, as a software development project, also needs requirements to be identified prior to implementation but often suffers from the opposite. 

Problems and impact 

1)     Ignoring programming practices 

• With all the “record/playback” features and wizards creation of a script does not require programming and this is always stated as a great advantage. But is it really that good? Evolution of programming as a discipline produced a lot of internal rules intended to keep good quality of the code: structure, readability, naming conventions and namespace, – just to name a few. Scripts, created by ignorant non-programmers are doomed to have poor quality. And manager has to wait until the painful evolution process is gone through again, while there is no actual need in reinventing all the wheels 
• Anti-patterns of programming practices, including “copy/paste programming” and hard-coding, unfortunately became very popular in software development, heavily impacting both regular application development and automation development 

2)     Ignoring non-business requirements 

• As test case document could be re-used by many testers multiple times, testing script could cover multiple test cases going through the same GUI. Ignoring external customization possibilities (implemented with parameters and flags) results in vast number of scripts to create and maintain 
• Testing script is not supposed to work “just anyhow” but too many of them were so vulnerable in execution, that a special term appeared – “baby-sitting script execution”. Why those scripts never grow up? Read in the previous posts the consequences of ignoring Robustness requirements 
• If anytime you need to drive somewhere you would have to spend a few hours checking/fixing/setting your car, would you really need it? Testing script that requires significant maintenance just while it is needed to be used is the same waste of time and resources as that car. Read in the previous posts the consequences of ignoring Maintenance requirements 
• As any software product, test automation has its end-users. Is it really user-friendly? Convenient Test and Defect Reports, produced by the scripts, quick and simple setup and customization are the ways to satisfy the end-user who is still manual tester. Making scripts “black box” way with complicated setup procedure results in low trust and rejection 

3)     Misunderstanding business requirements 

• Script execution is not exactly the same as manual test execution. Human tester reads behind the lines and uses common sense. Testing script does only what it is coded to do. Exactly following the manual test case steps will not produce testing script of the same quality in test coverage. So manual test cases must be extended with additional validation and verification steps
• Not always, but often, the same functionality could be covered in a various ways. The way it is easy to do manually is not always the way it is easy to perform automatically. Coverage-oriented automation design is much more efficient, even if it sometimes requires re-design of test cases