An engineering approach to Software Testing and Test Automation

Reference Page: Comparison Rules ...

Test Automation Problems series in my blog has received its own page. I also added Recommended Reading section to provide a few links to external resources. Don’t miss the latest article by Linda Wilkinson. ...

Do you know enough about your test environment when designing a Load Test plan? What can you do and what would you do to provide stakeholders with the most valuable information in the context and at the moment? How do you use their feedback in your testing work? ...

Parent page: Test Automation Problems Support and Feedback  Business logic investigation, test case design, and test script programming are distinct activities. A professional automation engineer is capable of carrying out the all three, however the overall progress will slowdown. Since test automation project often starts when automated testing results are already needed there is no [...] ...

Parent page: Test Automation Problems Requirements  Trying to do something without understanding what you are supposed to get often produces unexpected results. With regards to computer software the expectations must be even more precise and specific, as all the functionalities are built from atomic instructions executed by a machine. Test Automation, as a software development [...] ...

Parent page: Test Automation Problems Robustness  Automated Testing is about delivering test results – on time, within the Test Phase and reliable enough, with no manual retesting. There is no or minimal time reserved for scripts fixing, debugging and rerunning during the Test Cycle. If Automated Testing didn’t work then QA resources have to perform [...] ...

Parent page: Test Automation Problems Verification  Manual verification is complicated and informal. For test automation purposes, making it logical and specific requires time and effort of a seasoned professional. Workarounds like “picture-based” verification do not require formal logic but do not save man hours at the end due to high maintenance cost and manual validation [...] ...

Parent page: Test Automation Problems Coverage  How thoroughly script goes through the test case flow and how many test cases are automated per test plan?   Points of consideration  1)     Scattered vs. in-depth functionality coverage  Each functionality module of the application requires initial effort spent on GUI mapping and common functions development. However, this effort is [...] ...

Parent page: Test Automation Problems Framework  There are 3 most popular approaches in implementation of Test Automation, and none of them isn’t ideally suitable for any environment. Approach and problems  1)     Linear scripts  A  single test case could be automated fairly quickly (especially using recording) but robustness without error-handling is very questionable  GUI mapping is [...] ...

Parent page: Test Automation Problems Resource Conversion When starting up in-house Test Automation, companies often look for internal resources. While the benefits are obvious, there are some snags on this path. Problems and impact  1)     Converting manual testing QA resources to test automation  Strong programming skills and technical mindset are mandatory requirements, and must be considered [...] ...

General Recommendation Object-Oriented GUI Recognition method should be used whenever possible. If an object is completely supported by testing tool then wrapping functions (provided by the testing tool) should be used to interact with the object. If an object is partially supported by testing tool then object’s native methods and properties should be used when [...] ...

Parent page: Test Automation Problems Software  Technical problems related to Test Automation Tools, and platform-product-tool compatibilities. Problems Test Automation Tool does not support Product’s platform (operating system) completely or partially Test Automation Tool does not support Product’s GUI (for Functional/Regression Test Automation) completely or partially (this may include total absence of support for the particular [...] ...

Parent page: Test Automation Problems “Heavyweight” planning  Traditional “Waterfall” type approach: planning of automation development activities months and months ahead, creating project phases, extensive documentation, trying to predict future coverage. Problems and impact  Long-term budgeting and resource allocation planning is barely possible for automation projects as it is a secondary deliverable vs. the main software [...] ...

Original date: 12 May 2009, 5:00pm Quality Assurance Functional/Regression Automated Testing and Test Automation Requirements: Usability, Maintainability, Scalability, and Robustness. Maintainability Requirements Matrix Setup - What level of convenience you want to have setting-up the scripts? If Test Data file and path are hard-coded in the script then pointing to another data file requires code change. [...] ...

Original date: 6 May 2009, 12:30pm Quality Assurance Functional/Regression Automated Testing and Test Automation Requirements: Usability, Maintainability, Scalability, and Robustness. Scalability Requirements Matrix Platform - What applications should be able to handle the Framework? The business need for automation of QA/testing activities appears when testing volume, turnover and coverage requirements make it beneficial enough. Volume means ...

Original date: 28 Apr 2009, 1:00pm Quality Assurance Functional/Regression Automated Testing and Test Automation Requirements: Usability, Maintainability, Scalability, and Robustness. Robustness Requirements Matrix Manned / Unmanned Test Execution - What level of manual efforts in automated testing would be acceptable for you? Baby-sitting Test script may stop or break execution at any moment. Manual action [...] ...

Original date: 21 Apr 2009, 12:30pm Quality Assurance Functional/Regression Automated Testing and Test Automation Requirements: Usability, Maintainability, Scalability, and Robustness. Usability Requirements Matrix 2 Transparency - Do you want Automated Test execution to be another “black box”, or you want to have a fully reproduced picture with documented test results? Execution Log Built-in execution log [...] ...

Original date: 14 Apr 2009, 1:30pm Quality Assurance Functional/Regression Automated Testing and Test Automation Requirements: Usability, Maintainability, Scalability, and Robustness  Usability Requirements Matrix 1 Applicability - How would you want to use Automated Testing solution? Basic Regression Testing Implemented as a sequence of data entry steps going through the screens. No or simplified verification based on hard-coded [...] ...

QA Automation – GUI Function Wrapping Some of the statements below will be covered in detail in my subsequent posts. Now I use them to get straight to Test Step requirements. Please read details below the picture. Quality Assurance Functional/Regression Automated Testing and Test Automation Requirements: Usability, Maintainability, Scalability, and Robustness. Let’s break them down [...] ...

Original date: 30 Mar 2009, 1:30pm Front-End Test Automation Practices – Model-based Hybrid Keyword/Data Driven Framework 1. Description • Framework creation – Pure programmatically • Test creation – scriptless, visual design • Internal Data Model, capable to import/export data from various sources • GUI/Database checkpoints, parameterized/transitioned • Business Verification Rules • Framework-based structure; ...