Unit Testing is about caring for the code you write. It is about breaking the code down to testable units. When code is easy to test it is also easy to maintain and improve. Unit testing is about quality assurance and confidence in the code you write. It is about investing a little time upfront to gain on your investment when the project grows or has been put on ice for a while. Unit Testing is basically putting together the simplest "best practises" rules you find about coding and make those rules second nature to you while you code.
- 1 What Unit Testing is Not
- 2 Why Should I Adopt Unit Testing?
- 3 The Simplest Framework Possible
- 3.1 The UTAssert function
- 3.2 A Sample Project Using UTAssert
- 3.3 Running Our First Tests
- 3.4 Adding some more code to fix failed tests
- 3.5 Reconsider Our Scenarios
- 3.6 Putting This All Together
- 3.7 How About Some Automation
- 4 Creating a UnitTest Runner
What Unit Testing is Not
It is not the silver bullet the "inventors" like it to be. Unit testing GUI applications can be a real challenge and sometimes impossible.
Why Should I Adopt Unit Testing?
Every time you make a test you assure that some piece of your code works as intended. By following some simple rules you can make a simple automatic testing engine to make sure your code will work as you intend. If it does not you have to do improvements.
With current and later releases of AutoIt, if it does not then you either have to decide to keep a version of AutoIt where all your tests do pass. Or you will have an incredible easy time locating the areas of your code where you have to do updates.
The Simplest Framework Possible
Let's create a "unittest.au3" file and save it in an include directory. A good thing to know is that you can have your own include directory.
The UTAssert function
Now open your unittest.au3 file and add the simplest possible assert function.
Func UTAssert(Const $bool, Const $msg = "Assert Failure", Const $erl = @ScriptLineNumber) If NOT $bool Then ConsoleWrite("(" & $erl & ") := " & $msg & @LF) EndIf Return $bool EndFunc
A Sample Project Using UTAssert
To make good use of our unit testing framework we should make sure our code is nicely divided into well defined functions. If you are a hard core unit tester you should even know that the philosophy is to write the test code first and then fill in the code to make the tests work.
We need a nice little project showing us what to do. Let's start with a really easy text book example. We invent the function Add. Add can take two arguments. But if the first one is an array then the second one is not needed. If two values are provided they are added and the sum is returned. If an array is provided all elements in the array is added together. So a test setup would look something like this:
The first tests
UTAssert(Add(1) = 1) UTAssert(Add(1,1) = 2) UTAssert(Add(-1,1) = 0) UTAssert(Add(-1,-1) = -2)
That was the easy part. We should add as many tests as we need to make sure the code we write will be working under all expected curcumstances.
Now we have to cover the array requirement for the code.
Local $arg1 = [1, 2, 3, 4, 5, 6, 7, 8, 9] UTAssert(Add($arg1) = 45)
At this point none of the tests will pass because we have not written the function yet.
The function Under Test
Func Add(Const $arg1, Const $arg2 = 0) Local $ret If Not IsArray($arg1) Or Not IsArray($arg2) Then $ret = $arg1 + $arg2 EndIf Return $ret EndFunc
Running Our First Tests
Ok, So at this point we fire up the script and look at what it gives us:
>Running:(22.214.171.124):E:\scite\..\autoit-v126.96.36.199\autoit3.exe "E:\CodeX\autoit\au3\au3UnitTest.au3" (23) := Assert Failure +>AutoIT3.exe ended.rc:0
Obviously not so strange that the array test failed as we have not added any array code yet. So, let's continue with our quest.
Adding some more code to fix failed tests
Func Add(const $arg1, const $arg2 = 0) Local $ret, $i If IsArray($arg1) Or IsArray($arg2) Then If IsArray($arg1) Then For $i = 0 to UBound($arg1) - 1 $ret += Number($arg1[$i]) Next Else $ret += Number($arg1) EndIf Else $ret = $arg1 + $arg2 EndIf Return $ret EndFunc
Another run with the tests we have created and it turned out to be good. No red lines to jump to. That is really nice.
Reconsider Our Scenarios
Now it's time to ask. Did I write a test case for every possible use case I can think of? No, actually I did not. I'm missing several scenarios.
Adding More Tests
> Local $arg1 = [1, 2, 3, 4, 5, 6, 7, 8, 9] UTAssert(Add($arg1, 10) = 55) UTAssert(Add($arg1, $arg1) = 90) ;NOTE*
- Now this was not part of the original specification. Should I use it like this or should I return an error? If I return an error how will UTAssert react to it?
Running With the Added Tests
Running this will show us that our function needs more work.
>Running:(188.8.131.52):E:\scite\..\autoit-v184.108.40.206\autoit3.exe "E:\CodeX\autoit\au3\au3UnitTest.au3" (31) := Assert Failure (32) := Assert Failure +>AutoIT3.exe ended.rc:0
Adding a bit of code again
So, we take a look at our code. After considering our code we find that both failures are easy to fix. All we have to do is to duplicate the code
If IsArray($arg1) Then For $i = 0 to UBound($arg1) - 1 $ret += Number($arg1[$i]) Next Else $ret += Number($arg1) EndIf
And change the variable name $arg1 to $arg2.
Identifying Duplicate Code Blocks.
But hey, any part of the code that is a duplicate like that should be refactored to its own function. That function should be tested separately with unit tests.
Func ArgSum(Const $arg) Local $ret If IsArray($arg) Then For $i = 0 to UBound($arg) - 1 $ret += Number($arg[$i]) Next Else $ret += Number($arg) EndIf Return $ret EndFunc
Putting This All Together
We have the following code:
;#include <unittest.au3> Local $arg1 = [1, 2, 3, 4, 5, 6, 7, 8, 9] UTAssert(Add(39, 40) = 79) UTAssert(Add($arg1) = 45) UTAssert(Add($arg1, 10) = 55) UTAssert(Add($arg1, $arg1) = 90) ; NOTE* Look above in the Wiki for the note about this test ; The function under test. Func Add(Const $arg1, Const$arg2 = 0) Return ArgSum($arg1) + ArgSum($arg2) EndFunc ;==>Add Func ArgSum(Const $arg1) Local $ret If IsArray($arg1) Then For $i = 0 To UBound($arg1) - 1 $ret += Number($arg1[$i]) Next Else $ret = Number($arg1) EndIf Return $ret EndFunc ;==>ArgSum Func UTAssert(Const $bool, Const $msg = "Assert Failure", Const $erl = @ScriptLineNumber) If Not $bool Then ConsoleWrite("(" & $erl & ") := " & $msg & @LF) EndIf Return $bool EndFunc ;==>UTAssert
How About Some Automation
AutoIt is all about automation. So a unit test tool for AutoIt should be able to help us out. The big question is can it be simpler than it already is? Probably not much while you write the code. But when you want to try your code against a new release of AutoIt then there are lots of things we can automate.
Automatically Running the Function We Are Working In
Now that is a possibility. Say, all you have to do to run the function you are working in is to hit some key combination. To achieve this we would need:
- A method of identifying the function we are working in.
- A small program to write some code including the module we are working in and call the appropriate function.
As it is, the SciTE4Autoit distribution has a way of identifying the function your caret is located in. You can observe this in the status bare on the left side. This information is also available through a variable.
A Simple Runner Application
At this point we need a simple runner application. It has to:
- Accept function to run and script location.
- Wrap up the code to run the test function associated with the function.
;#include <unittest.au3> Func UTAssert(Const $bool, Const $msg = "Assert Failure", Const $erl = @ScriptLineNumber, Const $error = @error, Const $extended = @extended) If NOT $bool Then ConsoleWrite("(" & $erl & ") := " & $msg & @LF) EndIf If $error <> 0 Then SetError($error, $extended, $error) Return $bool EndFunc Func ParseCmdLine(ByRef $arr, ByRef $CmdLine) #cs;$arr will be formated according to this ;$arr = Array items Containing data. On top of that is maintenance items ;$arr[1 ... n] = File names ;$arr[n .... $arr] = Function names ;$arr[$arr + 1 = File names count ;This is given by  and previous $arr[$arr + 2 = Function names count #ce Local $paths = 1 ReDim $arr[$CmdLine + 2] For $i = 0 to $CmdLine $arr[$i] = $CmdLine[$i] Next $arr[$arr + 1] = $paths EndFunc Func err(Const $msg, Const $nr, Const $terminate = 0, Const $erl = @ScriptLineNumber) dbg($msg, $nr, 0, $erl) IF $terminate Then Exit EndFunc Func dbg(Const $msg, Const $error = @error, Const $extended = @extended, Const $erl = @ScriptLineNumber) ConsoleWrite("(" & $erl & ") : = (" & $error & ")(" & $extended & ") " & $msg & @LF) If $error <> 0 Then SetError($error, $extended, $error) Return $error EndFunc ;==>dbg Func testParseCmdLine() Local $cmds = [3, "testFunc1", "c:\test", "testFunc2"] Local $arr ParseCmdLine($arr, $cmds) UTAssert(IsArray($arr)) UTAssert($arr = 3) UTAssert($arr[$arr +1] = 1) UTAssert($arr - $arr[$arr +1] = 2) UTAssert($arr = "c:\test") ;TODO: Should we anticipate the sequence of functions to call? UTAssert($arr = "testFunc1") UTAssert($arr = "testFunc2") EndFunc ; ===================================================================== ; Selftesting part ; ===================================================================== If StringInStr(@ScriptName, "au3UTRunner.au3") Then testParseCmdLine() EndIf
Creating a UnitTest Runner
Now here is an interesting topic. What is a test runner? The simplest approach again is just to use SciTE and watch for lines starting with parenthesis wrapped around a number. If done like this we need a script calling all of our test functions. Actually not a bad solution. It does not give you any statistics. And we just love to know how many tests we pass each time.
But as we work on our UDFs we can do something like this.
#include-once #include "myFuncLib.au3" #include "UnitTest.au3" If StringInStr(@ScriptName, "filename") Then testMyFunc1() EndFunc Func testMyFunc1() UTAssert(1 = 1, "Now this is a solution") EndFunc
In SciTE all we have to do now is press F5 and look for those pesky error lines. Maybe not the cleanest possible but it is simple and it works.