Business Value – Verification Levels
The automation is fundamentally limited to mechanical verification (aka Checking) – through comparison of values only. However, with proper design, automation may simulate many of usage scenarios, which will trigger possible problems either detectable through comparison or directly impacting the flow of test execution. Combination of cheap high volume of automated tests with human attention and judgment helps achieving best verification levels.
|
Verification |
Description |
Assessment |
| Static GUI checkpoint | Merely verification of presence of a GUI control. | Not really useful on itself; GUI is not a functionality.
Although fairly easy to create with recording, big number of static checkpoints significantly increases maintenance cost. |
| Dynamic GUI checkpoint | Verification through input; includes both verification of presence and attempt to use the control. | The most typical coverage in isolated functional testing. The value can be greatly increased with verification of in- and out-flow of the data with scenarios.
Automation must aim to cover all GUI controls through dynamic checkpoints, but manage to avoid duplication of code. |
| Business Rule verification | Verification of behaviour defined by the business rules. Includes verification of presence, verification of values according to comparison rules, and triggering actions.
Examples: Input in one field makes another field enabled or disabled for input; input in one field changes prepopulated values in other fields. |
Further increases value of automated tests. As implementation of triggers requires logical branches in the code, the common trap is implementation of multiple versions of scripts for the same functionality – which increases creation and maintenance costs. |
| Usage Scenario Verification | Combination of scripts enabling coverage of real usage scenarios with all their dependencies.
Requires running of test cases in a high volume.
|
This coverage is especially valuable due to wide range of indirect verifications, that don’t have to be specifically coded. This happens because important issues always eventually will impact the “down flow”, and minor issues tend to accumulate, which also makes them eventually visible.
Accountability and observation are required from testers to pay attention and investigate indirect evidence of a problem, but this always results in a big catch.
|
| File Comparison | This is a non-GUI verification method that is applied to big volume of application data. Either scripts capture GUI data (tables or records, onscreen reports), or application produces file itself.
During test preparation, the data files are to be thoroughly verified manually. At subsequent test executions, current file (“actual result”) is compared with the original file (“expected result”). All discrepancies to be manually investigated. |
The strength of the approach is that it enables verification of data in big volumes with very simple implementation effort.
However, the applicability is limited: any single change in structure or contents of the file will cause comparison failures. In case of large number of variations of “expected result” files, requires significant manual effort for preparation and verification of files.
|
| File Verification | File verification is a mixture of textual comparison and parsing – more advanced approach to file comparison.
Enables parameterization of “expected result” file, and even implementation of some simple logic executed during parsing and comparison. |
This approach retains all strengths and addresses the weakness.
However, it requires greater investment into implementation of verification engine. |
One response to "Business Value – Verification Levels"
Thanks, Albert, for this very interesting analysis.