Parent page: Test Automation Problems

“Heavyweight” planning

 Traditional “Waterfall” type approach: planning of automation development activities months and months ahead, creating project phases, extensive documentation, trying to predict future coverage.

Problems and impact 

• Long-term budgeting and resource allocation planning is barely possible for automation projects as it is a secondary deliverable vs. the main software product
• Initial automation requirements (as well as expectations) are often not identified in detail – so there is a high risk of leading the project based on the wrong assumptions
• As the software product is constantly evolving requirements change very often. Moreover already implemented scripts may require update/rewrite before they were deployed and integrated into test cycle

“Ad-hock” Automation

 Absence of any planning: one day automation is just started, not really qualified (maybe not even trained) resources are assigned to do the job, test cases picked up without any consideration, etc.

Problems and impact 

• Initial automation requirements (and, definitely, expectations) are not identified – the outcome will be unpredictable and most likely will bring no business value
• Implementation most likely won’t follow any of proven automation methodologies, producing scripts with questionable usability, coverage, and robustness as the result
• Efforts spent maintaining and executing (probably, baby-sitting too) scripts, and manually validating execution results, may easily become equal to or exceed efforts spent just manually testing the application
• Due to informal procedures and lack (or total absence) of documentation cross-training and knowledge transfer on using the implemented automation is hard and has a long learning curve

“Outsourced Drive”

 Giving up all the initiatives on the project planning and management to the external contractor, like consulting or off-shore company.

While the most of consultants are very experienced professionals the business goals of outsourcing company and company-client are opposite in terms of business expenses for automation. Client-side end-users of Test Automation are Quality Assurance and Project Management that require as quick as possible implementation and integration of automated tests into the software product development cycle, as well as minimal as possible adjustment for any changes. Outsourcing company strives to obtain a long-term relationship and budget flow.

Problems and impact 

• Exaggeration of requirements, volume, and, in certain cases, time and efforts required to implement. Since it’s paid why not to try automating 100% of test cases and even going beyond that number?
• Communication, integration, and collaboration issues, especially with off-shore outsourcing team
• Implementation may require continuous and high maintenance efforts and support by contractor’s resources