PopUp Checker

[zfisherdrums]

Writing a Test Automation Framework using AutoIt. Needed a way to handle unexpected popups belonging to the application under test.

This currently checks for #32770 class windows and JIT Debug windows. I will be implementing Dr. Watson handling sometime.

Attached is a script that produces a buggy application. Download it and compile it. It will be used in the code example below.

BuggyApp.au3

expandcollapse popup

[indent]#include <A3LWinAPI.au3>

Global $APP_PID = -1

;===============================================================================
;
; Function Name:    CheckForPopUps
; Description::     Checks for the presence of pop up dialogs belonging to the AUT
; Parameter(s):     None
; Requirement(s):     A3LWinAPI.au3
; Return Value(s):  0 = no popups, popup text if popup occurred   
;
;===============================================================================
;
Func CheckForPopUps( )
    Local $returnVal = 0
    Local $windows = _API_EnumWindows()
    Local $TEST_PID, $winTitle, $winText
    For $i = 1 To $windows[0][0]        
    ; #32770 Class windows are often pop-ups
        If $windows[$i][1] = "#32770" Then
        ; App Pop-Ups should share PID with application under test
            $TEST_PID = WinGetProcess( $windows[$i][0] )
            $winTitle = WinGetTitle( $windows[$i][0] )
            $winText = WinGetText( $windows[$i][0] )
            Select
                Case $TEST_PID = $APP_PID 
                ; Splash screen does not return window text, so ignore it
                    If $winText <> "" then
                        $winText = StringReplace( $winText, @LF, " | " )
                        $returnVal = $winText
                        HandlePopUp( $windows[$i][0] )
                        ExitLoop
                    EndIf                   
                Case $winTitle = "Visual Studio Just-In-Time Debugger"
                    HandlePopUp( "Visual Studio Just-In-Time Debugger" )
                    $returnVal = $winTitle
                    ExitLoop
                Case Else
                ; Do nothing
            EndSelect
        EndIf
    Next
    Return $returnVal
EndFunc


;===============================================================================
;
; Function Name:    HandlePopUp   
; Description::     Takes care of those bothersome error dialogs
; Parameter(s):     $id = Window identifier ( HWND, Title, etc )
; Requirement(s):  
; Return Value(s): 
; Author(s):       
;
;===============================================================================
;
Func HandlePopUp( $id )
; We can define routines for specific errors in here
; For this example, we closing any and all error dialogs
    ConsoleWrite( "Closing popup." & @CRLF)
    WinClose( $id )
EndFunc

AdlibEnable( "CheckForPopUps", 1000 )

$APP_PID = Run( "BuggyApp.exe" )
While ProcessExists( $APP_PID )
    ControlFocus( "Form1", "", "Edit1" )
    ControlSend( "Form1", "", "Edit1", "Testing the application" & @LF )
    Sleep(1000)
Wend[/indent]

EDIT: Updated for clarity

Edited November 1, 2007 by zfisherdrums

[JohnBailey]

I don't understand your example

[zfisherdrums]

I don't understand your example

I've updated the top post to provide a better example.

[JohnBailey]

I've updated the top post to provide a better example.

Thanks for the revision.

I get the general idea better now. However, if I'm creating the msgbox why don't I just handle it there? I think I'm missing some thing crucial.

For instance, I was thinking this would fix this stuff and it does, but it closes for some reason. In the BuggyApp change the loop to the following and then test

While 1
    Local $array
    Sleep(100)
    $random = Random( 0, 50, 1 )
    If $random = 5 Then
        $array[2] = 'error'
    EndIf
WEnd

EDIT:

Why did you want to use AdlibEnable instead of just include it in the loop?

Edited November 1, 2007 by JohnBailey

[JohnBailey]

What do you think of the following suggestions?

1. Get rid of the Adlib or give an option (I show an option for it)

2. Have the HandlePopUp contain a parameter for reading back the Popup Win's Text

BuggyApp

#include <GUIConstants.au3>

Opt("GUIOnEventMode", 1)
#Region ### START Koda GUI section ### Form=
$Form1 = GUICreate("Form1", 399, 376, 193, 115)
GUISetOnEvent($GUI_EVENT_CLOSE, "Form1Close")
$Edit1 = GUICtrlCreateEdit("", 0, 16, 393, 337, BitOR($ES_AUTOVSCROLL,$ES_AUTOHSCROLL,$ES_WANTRETURN))
GUISetState(@SW_SHOW)
#EndRegion ### END Koda GUI section ###

While 1
    Local $array
    Sleep(2000)
    ControlSend( "Form1", "", "Edit1", "Testing the application" & @LF )
    $random = Random( 0, 50, 1 )
    If $random = 5 Then
        $array[2] = 'error'
    EndIf
WEnd
Func Form1Close()
    Exit
EndFunc

CheckForPopUps Example

expandcollapse popup

#include <A3LWinAPI.au3>

Global $APP_PID = -1

Local $ErrorMsg = 'Used Adlib, so unkown return'
;AdlibEnable( "CheckForPopUps", 1000 )

$APP_PID = Run( "BuggyApp.exe" )
While ProcessExists( $APP_PID )
    Local $ErrorMsg = CheckForPopUps($APP_PID)
    Sleep(100)
Wend
MsgBox(0,'PopUpChecker','Program checking for errored and closed, so I am closing too.'&@LF&@LF&'Error msg was: '&$ErrorMsg&@LF&@LF&'I will close in ten seconds or click ok',10)

;===============================================================================
;
; Function Name:       CheckForPopUps
; Description::     Checks for the presence of pop up dialogs belonging to the AUT
; Parameter(s):     $_APP_PID   -   var     - Optional: Set the PID to watch for popups from.
;                                                   'useGlobal' = (default) you can use the Global $APP_PID (best used if using Adlib directly calling this function)
;
; Requirement(s):     A3LWinAPI.au3
; Return Value(s):   0 = no popups, popup text if popup occurred  
;
;===============================================================================
;
Func CheckForPopUps($_APP_PID='useGlobal')
    
    If $_APP_PID = 'useGlobal' Then; I did this so you could use it more than once in a script or totally remove the Global and Adlib
        $_APP_PID = $APP_PID
    EndIf
    
    Local $returnVal = 0
    Local $windows = _API_EnumWindows()
    Local $TEST_PID, $winTitle, $winText
    For $i = 1 To $windows[0][0]        
   ; #32770 Class windows are often pop-ups
        If $windows[$i][1] = "#32770" Then
       ; App Pop-Ups should share PID with application under test
            $TEST_PID = WinGetProcess( $windows[$i][0] )
            $winTitle = WinGetTitle( $windows[$i][0] )
            $winText = WinGetText( $windows[$i][0] )
            Select
                Case $TEST_PID = $_APP_PID
               ; Splash screen does not return window text, so ignore it
                    If $winText <> "" then
                        $winText = StringReplace( $winText, @LF, " | " )
                        $returnVal = $winText
                        HandlePopUp( $windows[$i][0],$returnVal)
                        ExitLoop
                    EndIf                   
                Case $winTitle = "Visual Studio Just-In-Time Debugger"
                    HandlePopUp( "Visual Studio Just-In-Time Debugger" )
                    $returnVal = $winTitle
                    ExitLoop
                Case Else
               ; Do nothing
            EndSelect
        EndIf
    Next
    Return $returnVal
EndFunc


;===============================================================================
;
; Function Name:     HandlePopUp  
; Description::     Takes care of those bothersome error dialogs
; Parameter(s):     $id = Window identifier ( HWND, Title, etc )
;                       $_popupMsg  - string    -Optional: Allows you to see what the Popup Win's Text is/was
;                                                   'default' = (default) just the default
; Requirement(s):  
; Return Value(s):
; Author(s):      
;
;===============================================================================
;
Func HandlePopUp( $id ,$_popupMsg='default')
; We can define routines for specific errors in here
; For this example, we closing any and all error dialogs
    ConsoleWrite( "Closing popup." & @CRLF)
    ConsoleWrite( "Popup Msg: "& $_popupMsg & @CRLF)
    WinClose( $id )
EndFunc

BTW: I think the idea is neat

Edited November 1, 2007 by JohnBailey

[zfisherdrums]

Hello JohnBailey,

Thanks for your contributions and your questions. You help me understand the places where my explanations break down.

First, let me say that "BuggyApp" is scripted to represent an application that, for whatever reason, is throwing all sorts of exceptions. These exceptions sometimes manifest themselves as dialog boxes, message boxes, JIT/MFC/Dr.Watson windows, etc. In that example, these exceptions appear as a MessageBox. The name "BuggyApp" may be misleading. I'm using it to represent an application that SHOULD work but is not for reasons I, the tester, cannot predict in my script.

Secondly, the example that utilizes CheckForPopUp() simulates testing "BuggyApp". We must 'pretend' that the ControlSend() statement represents an interesting test against that application. Perhaps we're typing into an edit box, perhaps we're logging in, perhaps we're navigating screens; all these tests are being represented by the ControlSend() statement. That is why I placed it there and not in "BuggyApp" itself.

So what happens if, in the course of testing an application, an exception occurs? What if the test is part of an unattended automated regression testing suite? Without a method to handle that exception the test may continue in a corrupted state, report a false positive/false negative, or even worse - freeze. One way to handle that contingency is embedding these checks inside our tests. That may help for the exceptions we wish to target, but what about new exceptions or the same exception with new text? Now we're back into our test and hunting down all actions that MIGHT bump up against such a scenario. Our test would become overrun with our own expection handling especially if we're running a lengthy system-level scenario.

Another way to handle this is to break out the exception handling into a function. This helps as it clears up all that unwieldy code. But we are still having to explicitly call that function during the test (example below).

Do This
Check for exception
Now do this
Check for exception
Now this
Check for exception
...

In the example above, the test's intent is obscured by our exception handling. So a solution is to have this exception handling occur inside an adlib function. Now the intent of my test is clear while I still enjoy the benefit of having something in place to handle unexpected errors. The challenge, though, is when a dialog is expected by the test. For example, a FileDialog appears when saving a document. We now have a context where a popup would be expected AND acceptable. To handle these contexts, I use a HandlePopUp() function. In my larger implementations, this function does accept the window text and uses a Switch block to determine the appropriate action ( including logging, screenshots, closing application, etc ).

That said, you posit an interesting solution for developers to consider inasmuch as you suggest placing the test INSIDE the application. Having visibility into the application code underscores how valuable it is for testers to have code-level access. I would love it if the applications I tested encapsulated some of the more mundane tests and opened themselves up for our low-level scrutiny, but that's for another post/blog.

Edited November 1, 2007 by zfisherdrums

[JohnBailey]

Hello JohnBailey,

Thanks for your contributions and your questions. You help me understand the places where my explanations break down.

First, let me say that "BuggyApp" is scripted to represent an application that, for whatever reason, is throwing all sorts of exceptions. These exceptions sometimes manifest themselves as dialog boxes, message boxes, JIT/MFC/Dr.Watson windows, etc. In that example, these exceptions appear as a MessageBox. The name "BuggyApp" may be misleading. I'm using it to represent an application that SHOULD work but is not for reasons I, the tester, cannot predict in my script.

---quote trimmed---

WOW! haha that helped me understand what your intentions are/were

Thank you!

I really like the whole idea and how you're going about it. Seems as though I missed the primary thrust of your idea. Trimming down a large "debug" related script is difficult and now I understand.

Thanks again

[JohnBailey]

How do I get the Icon "state" of a msgbox?

Basically, I want to "filter-out" nonerror Msgboxes. I think there's a way to get the msgbox icon state/type