Jump to content

Test Harness...?


Recommended Posts

Hey All,

I've searched the forums a lot and have been unsuccessful in finding any answers to the questions I'm going to ask. Please forgive me if I just didn't search well enough.

I would like to use AutoIt3 to do some automated testing (as I'm sure a lot of people are doing). So far AutoIt3 has been working great for my needs. Now I am trying to handle error cases in the program I'm testing (note: not errors in AU3 but in the application I'm testing). The problem I have is that once an AU3 script starts it runs to the end and there doesn't seem to be a way to monitor for crashes of the application I'm testing. In this case I typically get a crash dialog or error message. I would like to have an AU3 script that is much like a Test Harness, which calls other scripts to run various tests as well as continuously monitors for crash dialogs. Eventually I would also like this Test Harness script to kill a sub-script if it's detected a crash state.

I did see AdLibEnable but it seemed like it might be a bit too narrowly focused to handle error reporting and potentially re-setting up an environment after a crash has been detected. Maybe I am incorrect though?

I tried to do the following as a test with a known crashing condition in the application I'm testing:

Run("C:\Program Files\AutoIt3\AutoIt3.exe BVT.au3 Play")
While 1
    Sleep(100)
    $bErrorRaised = DetectErrorDialogs ()
    If $bErrorRaised = True Then
        ExitLoop
    EndIf
WEnd

But what happened is it treated the Run more like a RunWait, which made it so that my DetectErrorDialogs function was a bit pointless.

Any ideas? Any suggestions?

Link to comment
Share on other sites

A good idea but then you forget one thing the program is Single-Threaded so then you need two programs with the main, Test Harness if you will run first then the next program, Test. Then you get into if the Test Harness sense a crash what shall it do? I can name three things it could do, Send a error code to the program (however your program now must watch for outside message), try to correct the problem in it like a backup program (however you can only do this if you sense the program ahead of the crashing, rather considerably ahead), find the problem and just correct that (you need a very fast test harness if you ever wanna complete against a system of the same, finding that one problem might just take to much time).

If you AdlibEnable it your dooming your program to fail because AdlibEnable pauses your program to run that code, those not doing what you want it to in a timely manner. The best way to find a error "then" fix it would be to enable a debugger to watch it and if a crash occurs write the error to a file and then restart it

0x576520616C6C206469652C206C697665206C69666520617320696620796F75207765726520696E20746865206C617374207365636F6E642E

Link to comment
Share on other sites

I did see AdLibEnable but it seemed like it might be a bit too narrowly focused to handle error reporting and potentially re-setting up an environment after a crash has been detected. Maybe I am incorrect though?

It depends on how you set up your adlib. The automation framework we developed at work utilizes a popup handler library. We register a callback function and, optionally, a window title context. Once started, the adlib actually calls our function that checks for windows that meets the dialog criteria. If it does, the associated callback function is called. The callback functions accept one parameter that holds a window handle. Inside the function, we can do whatever we want with the window or process or kick-off recovery scenarios, etc.

We extend this further by scripting application-specific popup handler libraries, which reference the base library. That way, we can define some global scenarios that always need to be handled across all tests by referencing the app-specific library, but then define some contextual handlers in the test itself ( i.e., a dialog that needs to be handled in a different manner for this test ).

TestHarness.au3

#include "PopUpHandler.au3"

PopUpHandler_Register( "OnPopUpDefaultHandler" )
PopUpHandler_Register( "OnPopUpExceptionHandler", "ERROR!" )
PopUpHandler_Start()

Dim $pid = Run("C:\Program Files\AutoIt3\AutoIt3.exe BVT.au3")
While ( ProcessExists( $pid ) )
    Sleep(100)  
WEnd

Func OnPopUpDefaultHandler( $hWnd )
    ControlClick( $hWnd, "", "Button1" )
    ConsoleWrite( "OnPopUpDefaultHandler: Handled Default PopUp Dialog" & @CRLF )
    ConsoleWrite( "OnPopUpDefaultHandler: Moving on...who cares!" & @CRLF )
EndFunc

Func OnPopUpExceptionHandler( $hWnd )
    ControlClick( $hWnd, "", "Button1" )
    ConsoleWrite( "OnPopUpExceptionHandler: Handled Exception PopUp Dialog" & @CRLF )
    ConsoleWrite( "OnPopUpExceptionHandler: Doing some backup stuff here" & @CRLF )
EndFunc

BVT.au3

; This represents an error occuring in the AUT 
MsgBox(0, 'DEFAULT', "An error occured")

; This represents another error occuring in the AUT ( perhaps a crash )
MsgBox(0, 'ERROR!', "A different error occured")

PopUpHandler.au3

#cs ----------------------------------------------------------------------------
    Name:           PopUpHandler.au3
    Author:         Zach Fisher and Shane Sloan
    Created:        2008-10-1
    Description:    Handles PopUps. Registers the PopUp Handler Functions the
                    user chooses
#ce ----------------------------------------------------------------------------

#include-once

Opt( "MustDeclareVars", 1 )

Global $g_objPopUpHandlers = ObjCreate( "Scripting.Dictionary" )

;===============================================================================
;
; Function Name:    PopUpHandler_Register
; Description:      Set the function that handles PopUp windows 
; Parameter(s):     $strFunctionName = Name of the function to call
;                                       The function should accept 1 argument ( hWnd )
;                   $strWinTitle    = ( Optional )If a PopUp with this title
;                                       is observed during monitoring, then the
;                                       provided function will be called
; Requirement(s):     True = Success; False = Failed
; Return Value(s):  Nothing
; Author:           Zach Fisher
;
;===============================================================================
;
Func PopUpHandler_Register( $strFunctionName, $strWinTitle = "POPUPDEFAULT" )
    If Not ( $g_objPopUpHandlers.Exists( $strWinTitle )) Then
        $g_objPopUpHandlers.Add( $strWinTitle, $strFunctionName )
    Else
        $g_objPopUpHandlers( $strWinTitle ) = $strFunctionName
    EndIf
    ConsoleWrite( "Added pop-up handler for """ & $strWinTitle & """" & @CRLF)
EndFunc


;===============================================================================
;
; Function Name:    PopUpHandler_Start
; Description::     Starts the popup monitor
; Parameter(s):     None
; Requirement(s):     None
; Return Value(s):  None
; Author(s):        Zach Fisher
;
;===============================================================================
;
Func PopUpHandler_Start()
    AdlibEnable( "CheckForPopUps" )
    ConsoleWrite( "PopUp Handling has started" & @CRLF)
EndFunc


;===============================================================================
;
; Function Name:    PopUpHandler_Stop
; Description::     Stops the popup monitor
; Parameter(s):     None
; Requirement(s):     None
; Return Value(s):  None
; Author(s):        Zach Fisher
;
;===============================================================================
;
Func PopUpHandler_Stop()
    AdlibDisable()
    ConsoleWrite( "PopUp Handling has stopped" & @CRLF)
EndFunc


;===============================================================================
;
; Function Name:    CheckForPopUps
; Description::     Checks for the presence of pop up dialogs
; Parameter(s):     None
; Requirement(s):     None
; Return Value(s):  Nothing 
; Author:           Zach Fisher
;
;===============================================================================
;
Func CheckForPopUps( )
    Local $strWinList
    $strWinList = WinList( "[CLASS:#32770]" )
    ProcessWinList( $strWinList )
EndFunc


;===============================================================================
;
; Function Name:    ProcessWinList
; Description::     Determine if a window meets the criteria for being a pop-up.
;                   If so, call the appropriate handler
; Parameter(s):     $strWinList = Array of windows from WinList
; Requirement(s):     None
; Return Value(s):  None
; Author(s):        AutoIt Help File & Zach Fisher
;
;===============================================================================
;
Func ProcessWinList( $strWinList ); Private Function, used by CheckForPopUps
    Local $i
    For $i = 1 to $strWinList[0][0]
        If ( $strWinList[$i][0] <> "" AND _IsVisible( $strWinList[$i][1] )) then    
            If ( $g_objPopUpHandlers.Exists( $strWinList[$i][0] )) then
                Call ( $g_objPopUpHandlers( $strWinList[$i][0] ), $strWinList[$i][1] )
            Else
                Call ( $g_objPopUpHandlers( "POPUPDEFAULT" ), $strWinList[$i][1] )
            EndIf 
        EndIf
    Next
EndFunc


;===============================================================================
;
; Function Name:      _IsVisible 
; Description::     Determines the visibility of the window
; Parameter(s):     $handle = handle to window
; Return Value(s):  1 = True, 0 = False
;
;===============================================================================
;
Func _IsVisible($handle)
  If BitAnd( WinGetState($handle), 2 ) Then 
    Return 1
  Else
    Return 0
  EndIf
EndFunc
Edited by zfisherdrums
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...