Transparency of coverage should include written information about the following:

• Implemented Test Coverage – coverage of each and every automated test case, including test purpose and main execution steps.
• Executed Test Coverage – automatically generated test execution report, with each and every automated test case that was executed, including test purpose, all execution steps, and pass/fail status.

Failure criteria

• Absence of accurate (enough detailed and up to date) information of test coverage of automated test cases.
• Execution report is not scalable.
• Execution report does not provide complete test reproduction steps helping to investigate found problems.
• Absence of test execution report, or the report fails to satisfy the following requirements:
• Each Test Case executed has individual report, which can be stored, emailed, embedded to a document.
• All parameters and environment settings passed on for the execution.
• All execution steps, as operations performed with GUI controls or test data.
• Pass/Fail conclusions.