Design first

Before producing any code we need to think of an algorithm first. This is a design phase.
Design could be narrow - in this case all the details are created “on the fly”, while coding; or it could be very detailed, with functional diagrams and pseudo-code.

Unit Function Design

Unit function design aims creation of a code performing that function. Typically, it starts with a skeleton of the main logic, and gets enhanced with data handling operations. As the next step, error-handling and logging operations are added, as well as code handling special case.

Note. Sometimes, design and coding of functionality involves only first two steps and that often ends up with a variety of bugs found in testing phase.

Unit Test Design

Unit test design begins with a review of a code-under-test.
Review includes analysis of design patterns, code-path analysis, data and conditional analysis. Often, even at this stage we may encounter bugs, that require full or partial re-writing (or re-design) of a function.

If no obvious problems were encountered, design of test code might be started.
Design of test code aims acquiring of information about behavior of the code-under-test. With each call we supply input data and verify returned data against expected result provided by an oracle. If mismatch is encountered either test code or unit code is changed to meet validation criteria.

Note. Thus, by designing, programming and executing unit tests developers test, re-design, re-factor and, finally, enhance the unit function code.

Unit Test Code to Unit Function Code Ratio

ArraySort Example

Straight logic: 24 code lines

Public Function ArraySort(ByRef dvArray, ByVal boolAscending)
 Dim Iter, Jter, aValue
 For Jter = 0 To UBound(dvArray)
  For Iter = 0 To UBound(dvArray)-1
   aValue = dvArray(Iter)
   If aValue > dvArray(Iter+1) Then
    If boolAscending Then
     dvArray(Iter) = dvArray(Iter+1)
     dvArray(Iter+1) = aValue
    Else
     'do nothing
    End If
   Else
    If boolAscending Then
     'do nothing
    Else
     dvArray(Iter) = dvArray(Iter+1)
     dvArray(Iter+1) = aValue
    End If
   End If
  Next
 Next
 ArraySort = dvArray
End Function

Negative cases considered based on code review: 4 more lines added

Public Function ArraySort(ByRef dvArray, ByVal boolAscending)
  Dim Iter, Jter, aValue
  If Not isArray(dvArray) Then
    dvArray = Array(dvArray)
    ArraySort = dvArray
  End If
 For Jter = 0 To UBound(dvArray)
  For Iter = 0 To UBound(dvArray)-1
   aValue = dvArray(Iter)
   If aValue > dvArray(Iter+1) Then
    If boolAscending Then
     dvArray(Iter) = dvArray(Iter+1)
     dvArray(Iter+1) = aValue
    Else
     'do nothing
    End If
   Else
    If boolAscending Then
     'do nothing
    Else
     dvArray(Iter) = dvArray(Iter+1)
     dvArray(Iter+1) = aValue
    End If
   End If
  Next
 Next
 ArraySort = dvArray
End Function

Unit Test Code: 36 lines

  dvArray = Array(2,3,5,1,9,-1)
  Call ArraySort(dvArray, True)
  If dvArray(0) <> -1 Then
    Log.Error("ArraySort failed")
  End If
  If dvArray(5) <> 9 Then
    Log.Error("ArraySort failed")
  End If
  If dvArray(3) <> 3 Then
    Log.Error("ArraySort failed")
  End If
  dvArray = 1
  Call ArraySort(dvArray, True)
  If dvArray(0) <> 1 Then
    Log.Error("ArraySort failed")
  End If
  dvArray = ""
  Call ArraySort(dvArray, True)
  If dvArray(0) <> "" Then
    Log.Error("ArraySort failed")
  End If
  dvArray = ArraySort(sEmpty, True)
  If dvArray(0) <> "" Then
    Log.Error("ArraySort failed")
  End If
  dvArray = Array(2,3,5,1,9,-1)
  Call ArraySort(dvArray, False)
  If dvArray(5) <> -1 Then
    Log.Error("ArraySort failed")
  End If
  If dvArray(0) <> 9 Then
    Log.Error("ArraySort failed")
  End If
  If dvArray(3) <> 2 Then
    Log.Error("ArraySort failed")
  End If

Result: not only test code is 30% bigger, it took longer time to create it.

Unit Test Automation Paradox

Design-time Testing

Testing as tracing and investigation of a code-under-test is performed manually; all bugs found are fixed simultaneously with this process.

Code Path Coverage

Code path coverage is reached because function calls and input data are designed based on the manual code review.

Automated Regression Test Paradox

Once the code has been re-factored (i.e. changed while maintaining its functionality), its internal logical paths are changed. While it’s still supposed to return the same results, which were validated before, new code path coverage is not assured!