preload

“How do you choose which test cases to ...

Posted by Albert Gareev on Oct 10, 2016
2
Despite of all critique and challenges, automation is a valuable aspect of testing strategy. Every new automator needs to answer questions like that: “How do you choose which test cases to automate?” While everything regarding testing is very specific, we may try to give an answer through general idea supplemented with concrete examples. Cross-posting my […] ...

GUI Automation Components

Posted by Albert Gareev on Sep 09, 2014
0
To perform testing, GUI automation scripts need to encapsulate the following components: Test data, used for input and verification. Service functionalities, like reporting, data retrieval, etc. GUI mapping – set of logical names of GUI controls mapped to their physical properties. GUI operation, i.e. recognition of controls, sending commands, retrieving property values. Test instructions (test […] ...

Scalability

Posted by Albert Gareev on Aug 29, 2014
0
Scalability is an integral characteristic comprising the following: How well volume of test cases can be extended without linear (or even geometric) growth of efforts for creation and maintenance. How well the same testing coverage can be applied to different environments. How well framework of the solution supports creation of integration of tests, by reusing […] ...

Maintainability

Posted by Albert Gareev on Aug 22, 2014
0
Maintainability is an integral characteristic comprising the following: Updates in test logic and/or GUI mapping caused by the changes in the application under test. Design or refactoring of automation for the purpose of expansion of coverage. Expansion or update of data set. Execution, testing, and debugging of testing scripts.   New environment setup Data changes […] ...

Reliability – Performance

Posted by Albert Gareev on Aug 15, 2014
0
Performance of automated tests must be optimized by design and implementation, to achieve best coverage in shortest time possible within the constraints of the context. This should be achieved by use of configurable synchronization parameters, proper design of automated test cases and test scenarios. Failure criteria: Execution of knowingly failed test cases, i.e. execution of […] ...

Reliability – Attendance and Robustness

Posted by Albert Gareev on Aug 08, 2014
0
Attendance Automated tests must be designed for unattended execution. Failure criteria: Test script may stop or break execution at any moment. Manual action required to continue the execution. Full restart is required if execution was stopped. Upon error or failure execution is skipped to the next script, leaving gaps in coverage. Scripts don’t log errors […] ...

Reliability – Operation

Posted by Albert Gareev on Aug 01, 2014
0
Automated tests must be delivered as a software product which can be used (operated) by testers and other team members without special technical skills in programming or automation tools. Operation requirements for all setup and preparation procedures: Must require only general computer skills; Must follow a single standard; Must use centralized and unified configuration file(s). […] ...

Business Value – Usability

Posted by Albert Gareev on Jul 25, 2014
0
Automated tests, as a product, should be available for use by any tester (and other team members) with minimal technical skills required.   This requirement heavily applies to the following procedures: Automation setup before execution. Launching execution of scripts. Reviewing test execution reports.   This requirement somewhat applies to the following procedures: Performing project-specific environment […] ...

Business Value – Transparency

Posted by Albert Gareev on Jul 18, 2014
0
Transparency of coverage should include written information about the following: Implemented Test Coverage – coverage of each and every automated test case, including test purpose and main execution steps. Executed Test Coverage – automatically generated test execution report, with each and every automated test case that was executed, including test purpose, all execution steps, and […] ...

Business Value – Creation Effort

Posted by Albert Gareev on Jul 11, 2014
0
Creation effort maps directly to costs – one can compare creation expenses with potential savings. Creation effort is an integral characteristic comprising the following: Test design or re-design of manual test cases to convert from human judgment to mechanistic verification. Automation design and development, including creation of scripts and any reusable components. Swiftness and efficiency […] ...

Business Value – Verification Levels

Posted by Albert Gareev on Jul 04, 2014
1
The automation is fundamentally limited to mechanical verification (aka Checking) – through comparison of values only. However, with proper design, automation may simulate many of usage scenarios, which will trigger possible problems either detectable through comparison or directly impacting the flow of test execution. Combination of cheap high volume of automated tests with human attention and […] ...

Business Value – Purpose

Posted by Albert Gareev on Jun 27, 2014
1
By industry standard definitions, test automation is created with the purpose of extending coverage, saving in test execution time, allowing for increase of test iterations, earlier detection of defects, and everything altogether. Therefore, only created and successfully used on a regular basis test automation should be considered as adding value. Coverage assessment must be done […] ...

Automation Requirements, Revisited – 2014

Posted by Albert Gareev on Jun 24, 2014
0
As I’m finding myself, I’ve been writing about QA Test Automation quite extensively in the years around 2010. No surprise – it was my main specialty. Then I was extending and sharpening my skillset in the field of QA Testing, and writing more and more about my new interests. Yet it wasn’t a transition – […] ...

Designing Data-Driven Test Cases

Posted by Albert Gareev on Jul 19, 2013
1
As I presented in my last post, a lot of test coverage is either data-specific or triggered by data combinations. This is an easy kind of tests to automate assuming well-known and explicit inputs and outputs. Below I enlist most common patterns and test ideas. General Patterns All major business requirements (the ones that are […] ...

Performance is an end-to-end thing

Posted by Albert Gareev on Aug 06, 2012
0
Some time ago I had a hard time discussing performance problem with application developer. My challenge was to overcome a “block” set by the developer: “this is a one-page code, it’s a very simple logic, and I see no reasons to spend time evaluating its performance”. I didn’t have more than 5-7 minutes for the […] ...

Comparison Rules: Mind Map

Posted by Albert Gareev on Jul 06, 2010
0
Reference Page: Comparison Rules ...

Test Automation Problems – Page Update

Posted by Albert Gareev on May 24, 2010
0
Test Automation Problems series in my blog has received its own page. I also added Recommended Reading section to provide a few links to external resources. Don’t miss the latest article by Linda Wilkinson. ...

Not a Load Test Plan template

Posted by Albert Gareev on Mar 23, 2010
1
Do you know enough about your test environment when designing a Load Test plan? What can you do and what would you do to provide stakeholders with the most valuable information in the context and at the moment? How do you use their feedback in your testing work? ...

Test Automation Problems (5) – Integration – ...

Posted by Albert Gareev on Sep 15, 2009
0
Parent page: Test Automation Problems Support and Feedback  Business logic investigation, test case design, and test script programming are distinct activities. A professional automation engineer is capable of carrying out the all three, however the overall progress will slowdown. Since test automation project often starts when automated testing results are already needed there is no […] ...

Test Automation Problems (5) – Integration – ...

Posted by Albert Gareev on Sep 14, 2009
0
Parent page: Test Automation Problems Requirements  Trying to do something without understanding what you are supposed to get often produces unexpected results. With regards to computer software the expectations must be even more precise and specific, as all the functionalities are built from atomic instructions executed by a machine. Test Automation, as a software development […] ...

Test Automation Problems (4) – Implementation ...

Posted by Albert Gareev on Aug 13, 2009
0
Parent page: Test Automation Problems Robustness  Automated Testing is about delivering test results – on time, within the Test Phase and reliable enough, with no manual retesting. There is no or minimal time reserved for scripts fixing, debugging and rerunning during the Test Cycle. If Automated Testing didn’t work then QA resources have to perform […] ...

Test Automation Problems (4) – Implementation ...

Posted by Albert Gareev on Aug 12, 2009
0
Parent page: Test Automation Problems Verification  Manual verification is complicated and informal. For test automation purposes, making it logical and specific requires time and effort of a seasoned professional. Workarounds like “picture-based” verification do not require formal logic but do not save man hours at the end due to high maintenance cost and manual validation […] ...

Test Automation Problems (4) – Implementation ...

Posted by Albert Gareev on Aug 11, 2009
0
Parent page: Test Automation Problems Coverage  How thoroughly script goes through the test case flow and how many test cases are automated per test plan?   Points of consideration  1)     Scattered vs. in-depth functionality coverage  Each functionality module of the application requires initial effort spent on GUI mapping and common functions development. However, this effort is […] ...

Test Automation Problems (4) – Implementation ...

Posted by Albert Gareev on Aug 10, 2009
0
Parent page: Test Automation Problems Framework  There are 3 most popular approaches in implementation of Test Automation, and none of them isn’t ideally suitable for any environment. Approach and problems  1)     Linear scripts  A  single test case could be automated fairly quickly (especially using recording) but robustness without error-handling is very questionable  GUI mapping is […] ...

Test Automation Problems (3) – Resource Allocation

Posted by Albert Gareev on Jul 29, 2009
1
Parent page: Test Automation Problems Resource Conversion When starting up in-house Test Automation, companies often look for internal resources. While the benefits are obvious, there are some snags on this path. Problems and impact  1)     Converting manual testing QA resources to test automation  Strong programming skills and technical mindset are mandatory requirements, and must be considered […] ...

Test Automation Workarounds

Posted by Albert Gareev on Jul 24, 2009
1
General Recommendation Object-Oriented GUI Recognition method should be used whenever possible. If an object is completely supported by testing tool then wrapping functions (provided by the testing tool) should be used to interact with the object. If an object is partially supported by testing tool then object’s native methods and properties should be used when […] ...

Test Automation Problems (2) – Technical Setup

Posted by Albert Gareev on Jul 08, 2009
0
Parent page: Test Automation Problems Software  Technical problems related to Test Automation Tools, and platform-product-tool compatibilities. Problems Test Automation Tool does not support Product’s platform (operating system) completely or partially Test Automation Tool does not support Product’s GUI (for Functional/Regression Test Automation) completely or partially (this may include total absence of support for the particular […] ...

Test Automation Problems (1) – Project Planning and ...

Posted by Albert Gareev on Jul 01, 2009
0
Parent page: Test Automation Problems “Heavyweight” planning  Traditional “Waterfall” type approach: planning of automation development activities months and months ahead, creating project phases, extensive documentation, trying to predict future coverage. Problems and impact  Long-term budgeting and resource allocation planning is barely possible for automation projects as it is a secondary deliverable vs. the main software […] ...

QA Test Automation Requirements (5) – ...

Posted by Albert Gareev on Jun 10, 2009
0
Original date: 12 May 2009, 5:00pm Quality Assurance Functional/Regression Automated Testing and Test Automation Requirements: Usability, Maintainability, Scalability, and Robustness. Maintainability Requirements Matrix Setup – What level of convenience you want to have setting-up the scripts? If Test Data file and path are hard-coded in the script then pointing to another data file requires code change. […] ...

QA Test Automation Requirements (4) – Scalability

Posted by Albert Gareev on Jun 09, 2009
0
Original date: 6 May 2009, 12:30pm Quality Assurance Functional/Regression Automated Testing and Test Automation Requirements: Usability, Maintainability, Scalability, and Robustness. Scalability Requirements Matrix Platform – What applications should be able to handle the Framework? The business need for automation of QA/testing activities appears when testing volume, turnover and coverage requirements make it beneficial enough. Volume ...
Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported
This work by Albert Gareev is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported.