CRUSSPIC STMPL my matrix
In my test lead role, on each project I need to assess existing test coverage, or implement test coverage, or both. What I found helpful is a systematic approach: model, categorize and classify, analyze. My initial model is largely based on Rapid Testing Heuristics. Then, based on the product/project context, traceability matrix unfolds to something like below.
Functional Capability & Consistency
Business Rules Coverage
- Use all data fields
- Use only mandatory data fields
- All data field dependencies
- All workflows within the Business Function
- All workflow triggers and dependencies
- All pre-conditions
- All post-conditions
- All hard-coded business conditions
- All required pre-populated data in DB
- All required pre-populated data in integrated applications
Time factor
- All time/date based conditions
- All time/date based triggers and actual execution of triggered jobs / services
- All time/date based dependencies of data values
Structure and Platform
- Consistency within Business Function and with other Business Functions
- Assessment of complexity of the structure
- Identifying generic (external library / framework) components within Business Function
- Identifying generic (internal library / framework) components within Business Function
- Identifying custom/unique components within Business Function
- Assessment of consistency of the structure and used components within the Business Function and with other Business Functions
Data domain
- Explicit input
- Defaulted input
- Implicit input
- System input
- Input from integrated products
- Other input
- Interrupted input
- Failures in input
- Output to screen
- Output to other Business Functions
- Output to DB
- Output to system
- Output to files
- Output to printers
- Output to integrated products
- Output to other destinations
- Interrupted output
- Failures in output
- Special input
- Restricted characters
- Special characters
- Mixed input
- High-precision data
Boundaries
- Numeric
- String
- Date
- List
- Set
- Tree
Non-business data
- Counters
- Buffers
- Clipboard
Formatting rules
- Date
- Currency
- %
- Sorting order for lists, sets, and trees
Representation
On-screen representation (GUI) – connected with “Operations” and “Communication” categories
- Structure
- Layout
- Formatting
On-paper representation (reports, etc)
- Structure
- Layout
- Formatting
Consistency
- Consistency with the purpose
- Verify Business Function is consistent with the Business Purpose
- Historical Consistency
- Review consistency of the goal(s) and logic of the Business Function with the previous versions
- SLA/Claims Consistency
- Review consistency of the implemented Business Function with the promised features in SLA/Contract, other official claims
State
- State transition rules
- State transition workflow
Operations
- Consistency of operations within Business Function and with similar Business Function inside the product
- Consistency of operations with similar Business Function in other comparable or competitor products
- Compliance of operations with SLA/Contractual obligations
- Compliance of operations with Statutory Regulations
- Operation by Mouse
- Operation by Keyboard
- Operation by other devices
- Performance / responsiveness of input operations
- Printing operations
- Output operations with other devices
- Performance / responsiveness of output operations
Communication
- Help/Hints
- Labeling
- Prompts
- Confirmations
- Errors
- Other communication
- Timing / Responsiveness of communication
- Accuracy of communication
- Consistency of communication with the purpose and situation in the context
- Consistency of communication with the Image of the organization
Multi-client operation
Shared read / write access
- Business Rules
- Limitations
Concurrent (simultaneous) access
- Business Rules
- Limitations
- Race conditions
Security
- Authentication
- Authorization
- Transmission
- Traffic
- Encryption
- Fake Client
- Session hijacking
- XML/HTML/JS Injection
- Failure Testing
- Exceptions
- Timeouts
- Denial of Service
- Out of resources
Penetration testing
- Incomplete data
- Invalid data
- Business Rules violation
- Non-validated injections
Supportability and Compatibility
- Functional Test coverage on supported browsers
- Behavior on officially not supported browsers
- Versions of OS
- Compatibility with other client software
Web Page Validation – http://validator.w3.org
- HTML
- CSS
- JS
- Browser settings
- Cookies
- Cache
- JavaScript
- View mode
- Certificates
Load/Performance Testing
Benchmark tests
- Production workload scenarios
- Heavy operations
- Package operations
- End-user performance tests
- Hybrid (server load + end-user) tests
Stability tests
- Endurance-under-load tests
- System stress tests
- Volume tests
Function-targeting tests
- Burst
- Stress
- Spike