preload

CRUSSPIC STMPL my matrix

Posted by Albert Gareev on Jun 24, 2013 | Categories: Heuristics

In my test lead role, on each project I need to assess existing test coverage, or implement test coverage, or both. What I found helpful is a systematic approach: model, categorize and classify, analyze. My initial model is largely based on Rapid Testing Heuristics. Then, based on the product/project context, traceability matrix unfolds to something like below.

Functional Capability & Consistency

Business Rules Coverage

  • Use all data fields
  • Use only mandatory data fields
  • All data field dependencies
  • All workflows within the Business Function
  • All workflow triggers and dependencies
  • All pre-conditions
  • All post-conditions
  • All hard-coded business conditions
  • All required pre-populated data in DB
  • All required pre-populated data in integrated applications

Time factor

  • All time/date based conditions
  • All time/date based triggers and actual execution of triggered jobs / services
  • All time/date based dependencies of data values

Structure and Platform

  • Consistency within Business Function and with other Business Functions
  • Assessment of complexity of the structure
  • Identifying generic (external library / framework) components within Business Function
  • Identifying generic (internal library / framework) components within Business Function
  • Identifying custom/unique components within Business Function
  • Assessment of consistency of the structure and used components within the Business Function and with other Business Functions

Data domain

  • Explicit input
  • Defaulted input
  • Implicit input
  • System input
  • Input from integrated products
  • Other input
  • Interrupted input
  • Failures in input
  • Output to screen
  • Output to other Business Functions
  • Output to DB
  • Output to system
  • Output to files
  • Output to printers
  • Output to integrated products
  • Output to other destinations
  • Interrupted output
  • Failures in output
  • Special input
  • Restricted characters
  • Special characters
  • Mixed input
  • High-precision data

Boundaries

  • Numeric
  • String
  • Date
  • List
  • Set
  • Tree

Non-business data

  • Counters
  • Buffers
  • Clipboard

Formatting rules

  • Date
  • Currency
  • %
  • Sorting order for lists, sets, and trees

Representation

On-screen representation (GUI) – connected with “Operations” and “Communication” categories

  • Structure
  • Layout
  • Formatting

On-paper representation (reports, etc)

  • Structure
  • Layout
  • Formatting

Consistency

  • Consistency with the purpose
  • Verify Business Function is consistent with the Business Purpose
  • Historical Consistency
  • Review consistency of the goal(s) and logic of the Business Function with the previous versions
  • SLA/Claims Consistency
  • Review consistency of the implemented Business Function with the promised features in SLA/Contract, other official claims

State

  • State transition rules
  • State transition workflow

Operations

  • Consistency of operations within Business Function and with similar Business Function inside the product
  • Consistency of operations with similar Business Function in other comparable or competitor products
  • Compliance of operations with SLA/Contractual obligations
  • Compliance of operations with Statutory Regulations
  • Operation by Mouse
  • Operation by Keyboard
  • Operation by other devices
  • Performance / responsiveness of input operations
  • Printing operations
  • Output operations with other devices
  • Performance / responsiveness of output operations

Communication

    • Help/Hints
    • Labeling
    • Prompts
    • Confirmations
    • Errors
    • Other communication
    • Timing / Responsiveness of communication
    • Accuracy of communication
    • Consistency of communication with the purpose and situation in the context
    • Consistency of communication with the Image of the organization

Multi-client operation

Shared read / write access

  • Business Rules
  • Limitations

Concurrent (simultaneous) access

  • Business Rules
  • Limitations
  • Race conditions

Security

  • Authentication
  • Authorization
  • Transmission
  • Traffic
  • Encryption
  • Fake Client
  • Session hijacking
  • XML/HTML/JS Injection
  • Failure Testing
  • Exceptions
  • Timeouts
  • Denial of Service
  • Out of resources

Penetration testing

  • Incomplete data
  • Invalid data
  • Business Rules violation
  • Non-validated injections

Supportability and Compatibility

  • Functional Test coverage on supported browsers
  • Behavior on officially not supported browsers
  • Versions of OS
  • Compatibility with other client software

Web Page Validation – http://validator.w3.org

  • HTML
  • CSS
  • JS
  • Browser settings
  • Cookies
  • Cache
  • JavaScript
  • View mode
  • Certificates

Load/Performance Testing

Benchmark tests

  • Production workload scenarios
  • Heavy operations
  • Package operations
  • End-user performance tests
  • Hybrid (server load + end-user) tests

Stability tests

  • Endurance-under-load tests
  • System stress tests
  • Volume tests

Function-targeting tests

  • Burst
  • Stress
  • Spike


Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported
This work by Albert Gareev is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported.