UnitTesting

This page is still a work in progress.

Unit Testing is about caring for the code you write. It is about breaking the code down to testable units. When code is easy to test it is also easy to maintain and improve. Unit testing is about quality assurance and confidence in the code you write. It is about investing a little time upfront to gain on your investment when the project grows or has been put on ice for a while. Unit Testing is basically putting together the simplest "best practises" rules you find about coding and make those rules second nature to you while you code.

What Unit Testing is Not

It is not the silver bullet the "inventors" like it to be. Unit testing GUI applications can be a real challenge and sometimes impossible.

Why Should I Adopt Unit Testing?

Every time you make a test you assure that some piece of your code works as intended. By following some simple rules you can make a simple automatic testing engine to make sure your code will work as you intend. If it does not you have to do improvements.

With current and later releases of AutoIt, if it does not then you either have to decide to keep a version of AutoIt where all your tests do pass. Or you will have an incredible easy time locating the areas of your code where you have to do updates.

The Simplest Framework Possible

Let's create a "unittest.au3" file and save it in an include directory. A good thing to know is that you can have your own include directory.

The UTAssert function

Now open your unittest.au3 file and add the simplest possible assert function.

Func UTAssert(Const $bool, Const $msg = "Assert Failure", Const $erl = @ScriptLineNumber)
    If NOT $bool Then 
        ConsoleWrite("(" & $erl & ") := " & $msg & @LF)
    EndIf
    
    Return $bool
EndFunc

A Sample Project Using UTAssert

To make good use of our unit testing framework we should make sure our code is nicely divided into well defined functions. If you are a hard core unit tester you should even know that the philosophy is to write the test code first and then fill in the code to make the tests work.

We need a nice little project showing us what to do. Let's start with a really easy text book example. We invent the function Add. Add can take two arguments. But if the first one is an array then the second one is not needed. If two values are provided they are added and the sum is returned. If an array is provided all elements in the array is added together. So a test setup would look something like this:

The first tests

UTAssert(Add(1) = 1)
UTAssert(Add(1,1) = 2)
UTAssert(Add(-1,1) = 0)
UTAssert(Add(-1,-1) = -2)

That was the easy part. We should add as many tests as we need to make sure the code we write will be working under all expected curcumstances.

Now we have to cover the array requirement for the code.

Local $arg1[9] = [1, 2, 3, 4, 5, 6, 7, 8, 9]
UTAssert(Add($arg1) = 45)

At this point none of the tests will pass because we have not written the function yet.

The function Under Test

Func Add(Const $arg1, Const $arg2 = 0)
    Local $ret
    
    If Not IsArray($arg1) Or Not IsArray($arg2) Then 
        $ret = $arg1 + $arg2
    EndIf
      
    Return $ret
EndFunc

Running Our First Tests

Ok, So at this point we fire up the script and look at what it gives us:

>Running:(3.2.2.0):E:\scite\..\autoit-v3.2.2.0\autoit3.exe "E:\CodeX\autoit\au3\au3UnitTest.au3"    
(23) := Assert Failure
+>AutoIT3.exe ended.rc:0

Obviously not so strange that the array test failed as we have not added any array code yet. So, let's continue with our quest.

Adding some more code to fix failed tests

Func Add(const $arg1, const $arg2 = 0)
    Local $ret, $i
    
    If IsArray($arg1) Or IsArray($arg2) Then 
      If IsArray($arg1) Then 
       For $i = 0 to UBound($arg1) - 1
        $ret += Number($arg1[$i])
       Next
      Else
       $ret += Number($arg1)
      EndIf
    Else
        $ret = $arg1 + $arg2
    EndIf
      
    Return $ret
EndFunc

Another run with the tests we have created and it turned out to be good. No red lines to jump to. That is really nice.

Reconsider Our Scenarios

Now it's time to ask. Did I write a test case for every possible use case I can think of? No, actually I did not. I'm missing several scenarios.

Adding More Tests

<

>
Local $arg1[9] = [1, 2, 3, 4, 5, 6, 7, 8, 9]

UTAssert(Add($arg1, 10) = 55)

UTAssert(Add($arg1, $arg1) = 90) ;NOTE*

• Now this was not part of the original specification. Should I use it like this or should I return an error? If I return an error how will UTAssert react to it?

Running With the Added Tests

Running this will show us that our function needs more work.

>Running:(3.2.2.0):E:\scite\..\autoit-v3.2.2.0\autoit3.exe "E:\CodeX\autoit\au3\au3UnitTest.au3"    
(31) := Assert Failure
(32) := Assert Failure
+>AutoIT3.exe ended.rc:0

Adding a bit of code again

So, we take a look at our code. After considering our code we find that both failures are easy to fix. All we have to do is to duplicate the code

If IsArray($arg1) Then 
   For $i = 0 to UBound($arg1) - 1
	  $ret += Number($arg1[$i])
   Next
Else
   $ret += Number($arg1)
EndIf

And change the variable name $arg1 to $arg2.

Identifying Duplicate Code Blocks.

But hey, any part of the code that is a duplicate like that should be refactored to its own function. That function should be tested separately with unit tests.

Func ArgSum(Const $arg)
    Local $ret

    If IsArray($arg) Then 
      For $i = 0 to UBound($arg) - 1
        $ret += Number($arg[$i])
      Next
    Else
      $ret += Number($arg)
    EndIf

    Return $ret
EndFunc

Putting This All Together

We have the following code:

;#include <unittest.au3>

Local $arg1[9] = [1, 2, 3, 4, 5, 6, 7, 8, 9]

UTAssert(Add(39, 40) = 79)

UTAssert(Add($arg1) = 45)

UTAssert(Add($arg1, 10) = 55)

UTAssert(Add($arg1, $arg1) = 90) ; NOTE* Look above in the Wiki for the note about this test

; The function under test.
Func Add(Const $arg1, Const$arg2 = 0)
	Return ArgSum($arg1) + ArgSum($arg2)
EndFunc   ;==>Add

Func ArgSum(Const $arg1)
	Local $ret
  
	If IsArray($arg1) Then
		For $i = 0 To UBound($arg1) - 1
			$ret += Number($arg1[$i])
		Next
	Else
		$ret = Number($arg1)
	EndIf
  
	Return $ret
EndFunc   ;==>ArgSum

Func UTAssert(Const $bool, Const $msg = "Assert Failure", Const $erl = @ScriptLineNumber)
	If Not $bool Then
		ConsoleWrite("(" & $erl & ") := " & $msg & @LF)
	EndIf
  
	Return $bool
EndFunc   ;==>UTAssert

How About Some Automation

AutoIt is all about automation. So a unit test tool for AutoIt should be able to help us out. The big question is can it be simpler than it already is? Probably not much while you write the code. But when you want to try your code against a new release of AutoIt then there are lots of things we can automate.

Automatically Running the Function We Are Working In

Now that is a possibility. Say, all you have to do to run the function you are working in is to hit some key combination. To achieve this we would need:

• A method of identifying the function we are working in.
• A small program to write some code including the module we are working in and call the appropriate function.

As it is, the SciTE4Autoit distribution has a way of identifying the function your caret is located in. You can observe this in the status bare on the left side. This information is also available through a variable.

A Simple Runner Application

At this point we need a simple runner application. It has to:

• Accept function to run and script location.
• Wrap up the code to run the test function associated with the function.

;#include <unittest.au3>

Func UTAssert(Const $bool, Const $msg = "Assert Failure", Const $erl = @ScriptLineNumber, Const $error = @error, Const $extended = @extended)
    If NOT $bool Then 
        ConsoleWrite("(" & $erl & ") := " & $msg & @LF)
    EndIf
      
    If $error <> 0 Then SetError($error, $extended, $error)
    
    Return $bool
EndFunc

Func ParseCmdLine(ByRef $arr, ByRef $CmdLine)
#cs;$arr will be formated according to this
   ;$arr[0] = Array items Containing data. On top of that is maintenance items
   ;$arr[1 ... n] = File names
   ;$arr[n .... $arr[0]] = Function names
   ;$arr[$arr[0] + 1 = File names count
   ;This is given by [0] and previous $arr[$arr[0] + 2 = Function names count
#ce   
    Local $paths = 1
   
    ReDim $arr[$CmdLine[0] + 2]
   
    For $i = 0 to $CmdLine[0]
      $arr[$i] = $CmdLine[$i]
    Next
  
    $arr[$arr[0] + 1] = $paths
EndFunc

Func err(Const $msg, Const $nr, Const $terminate = 0, Const $erl = @ScriptLineNumber)
   dbg($msg, $nr, 0, $erl)
   IF $terminate Then Exit
EndFunc

Func dbg(Const $msg, Const $error = @error, Const $extended = @extended, Const $erl = @ScriptLineNumber)
	ConsoleWrite("(" & $erl & ") : = (" & $error & ")(" & $extended & ") " & $msg & @LF)
  
	If $error <> 0 Then SetError($error, $extended, $error)
  
	Return $error
EndFunc   ;==>dbg

Func testParseCmdLine()
   Local $cmds[4] = [3, "testFunc1", "c:\test", "testFunc2"]
   
   Local $arr[4]
   
   ParseCmdLine($arr, $cmds)
   
   UTAssert(IsArray($arr))
   
   UTAssert($arr[0] = 3)
   
   UTAssert($arr[$arr[0] +1] = 1)
   
   UTAssert($arr[0] - $arr[$arr[0] +1] = 2)
   
   UTAssert($arr[1] = "c:\test")
   
   ;TODO: Should we anticipate the sequence of functions to call?
   
   UTAssert($arr[1] = "testFunc1")
   
   UTAssert($arr[1] = "testFunc2")
 EndFunc
 
; =====================================================================
; Selftesting part
; =====================================================================
If StringInStr(@ScriptName, "au3UTRunner.au3") Then 
   testParseCmdLine()
EndIf

Creating a UnitTest Runner

Now here is an interesting topic. What is a test runner? The simplest approach again is just to use SciTE and watch for lines starting with parenthesis wrapped around a number. If done like this we need a script calling all of our test functions. Actually not a bad solution. It does not give you any statistics. And we just love to know how many tests we pass each time.

But as we work on our UDFs we can do something like this.

#include-once

#include "myFuncLib.au3"

#include "UnitTest.au3"

If StringInStr(@ScriptName, "filename") Then 
    testMyFunc1()
EndFunc
  
Func testMyFunc1()
    UTAssert(1 = 1, "Now this is a solution")
EndFunc

In SciTE all we have to do now is press F5 and look for those pesky error lines. Maybe not the cleanest possible but it is simple and it works.