Unit Test Design vs. Unit Function Design
Design first
Before producing any code we need to think of an algorithm first. This is a design phase.
Design could be narrow – in this case all the details are created “on the fly”, while coding; or it could be very detailed, with functional diagrams and pseudo-code.
Unit Function Design
Unit function design aims creation of a code performing that function. Typically, it starts with a skeleton of the main logic, and gets enhanced with data handling operations. As the next step, error-handling and logging operations are added, as well as code handling special case.
Note. Sometimes, design and coding of functionality involves only first two steps and that often ends up with a variety of bugs found in testing phase.
Unit Test Design
Unit test design begins with a review of a code-under-test.
Review includes analysis of design patterns, code-path analysis, data and conditional analysis. Often, even at this stage we may encounter bugs, that require full or partial re-writing (or re-design) of a function.
If no obvious problems were encountered, design of test code might be started.
Design of test code aims acquiring of information about behavior of the code-under-test. With each call we supply input data and verify returned data against expected result provided by an oracle. If mismatch is encountered either test code or unit code is changed to meet validation criteria.
Note. Thus, by designing, programming and executing unit tests developers test, re-design, re-factor and, finally, enhance the unit function code.
Unit Test Code to Unit Function Code Ratio
ArraySort Example
Straight logic: 24 code lines
Public Function ArraySort(ByRef dvArray, ByVal boolAscending) Dim Iter, Jter, aValue For Jter = 0 To UBound(dvArray) For Iter = 0 To UBound(dvArray)-1 aValue = dvArray(Iter) If aValue > dvArray(Iter+1) Then If boolAscending Then dvArray(Iter) = dvArray(Iter+1) dvArray(Iter+1) = aValue Else 'do nothing End If Else If boolAscending Then 'do nothing Else dvArray(Iter) = dvArray(Iter+1) dvArray(Iter+1) = aValue End If End If Next Next ArraySort = dvArray End Function
Negative cases considered based on code review: 4 more lines added
Public Function ArraySort(ByRef dvArray, ByVal boolAscending) Dim Iter, Jter, aValue If Not isArray(dvArray) Then dvArray = Array(dvArray) ArraySort = dvArray End If For Jter = 0 To UBound(dvArray) For Iter = 0 To UBound(dvArray)-1 aValue = dvArray(Iter) If aValue > dvArray(Iter+1) Then If boolAscending Then dvArray(Iter) = dvArray(Iter+1) dvArray(Iter+1) = aValue Else 'do nothing End If Else If boolAscending Then 'do nothing Else dvArray(Iter) = dvArray(Iter+1) dvArray(Iter+1) = aValue End If End If Next Next ArraySort = dvArray End Function
Unit Test Code: 36 lines
dvArray = Array(2,3,5,1,9,-1) Call ArraySort(dvArray, True) If dvArray(0) <> -1 Then Log.Error("ArraySort failed") End If If dvArray(5) <> 9 Then Log.Error("ArraySort failed") End If If dvArray(3) <> 3 Then Log.Error("ArraySort failed") End If dvArray = 1 Call ArraySort(dvArray, True) If dvArray(0) <> 1 Then Log.Error("ArraySort failed") End If dvArray = "" Call ArraySort(dvArray, True) If dvArray(0) <> "" Then Log.Error("ArraySort failed") End If dvArray = ArraySort(sEmpty, True) If dvArray(0) <> "" Then Log.Error("ArraySort failed") End If dvArray = Array(2,3,5,1,9,-1) Call ArraySort(dvArray, False) If dvArray(5) <> -1 Then Log.Error("ArraySort failed") End If If dvArray(0) <> 9 Then Log.Error("ArraySort failed") End If If dvArray(3) <> 2 Then Log.Error("ArraySort failed") End If
Result: not only test code is 30% bigger, it took longer time to create it.
Unit Test Automation Paradox
Design-time Testing
Testing as tracing and investigation of a code-under-test is performed manually; all bugs found are fixed simultaneously with this process.
Code Path Coverage
Code path coverage is reached because function calls and input data are designed based on the manual code review.
Automated Regression Test Paradox
Once the code has been re-factored (i.e. changed while maintaining its functionality), its internal logical paths are changed. While it’s still supposed to return the same results, which were validated before, new code path coverage is not assured!