Automation for Microsoft Office via Microsoft Active Accessibility in AutoIt

[Fritsche]

Some weeks ago I got the task to automatic create a lot of databases with the Access 2003 Source Code Control Add-in in a nightly build. First I use the AutoIt Functions like WinActivate and ControlClick. But many Controls in MS-Office, such the menubar (MsoCommandBar) or buttons (bosa_sdm_Mso96) are not visible to those commands. So I use many Send or ControlSend commands. Finally the Build runs not stable. After 10 or 30 minutes it blocks with unreproducible errors.

Then I rewrite the scripts by use of the Microsoft Active Accessibility technology. Now I can address the MS-Office controls by name. I can click at a menu item by use of an object model. The problems are disappeared.

I attached a library with some examples. For now I use a small COM-Wrapper for the Win-API Calls because the Win-API returns a COM-Object (IAccessible) in their parameter list and I can´t implement that in AutoIt. May be someone out there have the knowledge to replace it with pure AutoIt Code.

If you find it useful, please feel free to extend the library or the examples.

Best Regards

ActiveAccessibilityForMsOffice.zip

Edited September 25, 2007 by Fritsche

[ptrex]

@Fritsche

Very nice

I like to see new techniques being introduces in AU3 and especially COM related ones.

Good job.

regards

ptrex

[MadBoy]

Some weeks ago I got the task to automatic create a lot of databases with the Access 2003 Source Code Control Add-in in a nightly build. First I use the AutoIt Functions like WinActivate and ControlClick. But many Controls in MS-Office, such the menubar (MsoCommandBar) or Buttons (bosa_sdm_Mso96) are not visible to those commands. So I use many Send or ControlSend commands. Finally the Build runs not stable. After 10 or 30 minutes it blocks with unreproducible errors.

Then I rewrite the scripts by use of the Microsoft Active Accessibility technologie. Now I can adress the MS-Office controls by name. I can click at a menuitem by use of an objectmodel. The problems are disappeared.

I attached a library with some examples. For now I use a small COM-Wrapper for the Win-API Calls because the Win-Api returns a COM-Object (IAccessible) in their parameterlist und I can´t implement that in AutoIt. May be someone out there have the knowledge to replace it with pure AutoIt Code.

If you find it useful, please feel free to extend the library or the examples.

Best Regards

I haven't tested your method but it seems nice as it's autoit based. However most if not all Office applications, settings can be changed using Resource Kit for 200,XP,2003, 2007. Have you tried them and those don't do the job? Sorry if it's something more you wanted to achive.

[Fritsche]

I haven't tested your method but it seems nice as it's autoit based. However most if not all Office applications, settings can be changed using Resource Kit for 200,XP,2003, 2007. Have you tried them and those don't do the job? Sorry if it's something more you wanted to achive.

Sorry, but I can´t see how the Resource Kit can help me by the task that I have described in the opening posting.

[taralex]

I tried your funcitons, but even the example failed to work, the error reads:

C:\Program Files\AutoIt3\Include\ActiveAccessibility_Library.au3 (160) : ==> Variable must be of type "Object".:

[yehia]

do this means we r able to control windowless controls?

[zfisherdrums]

do this means we r able to control windowless controls?

Depends. If the windowless controls implement IAccessible, then it may very well be possible. To determine if this is applicable in your situation, download Microsoft's Accessibility Tools, AccExplorer32 in particular. Using that tool will enable you to determine which controls respond to assistive technology (AT) and which do not.

Zach Fisher...

Edited January 5, 2009 by zfisherdrums

[yehia]

ahm sorry but how to do that i did a test on the control and thats what i got

but as i can see the accexplorer got the value of the edit object called history

so i guess it could be done

can u write an example for this?

thanks alot for the support

[yehia]

can anyone show me how to get the value of an accessible object?

[jkorsak]

This worked out great for me in Microsoft Access. I have an mde interface controlled by an outside company that will no do any work toward automation and will not release an mdb to me. Unfortunately, telling them that do not care about their clients may not have been the best tactic. This was the only way short of mousemove and mouseclick that I could get automation to function. Plus, they made it so that after a button click process completes the only notification of completion was text changing in a locked textbox on a subform which was not recognized by any control level Windows API methods I tried. I could get a handle to the control, could change the text of the control but any attempt to read the text always returned an empty string. I know the method I was using was somewhat functional since I could read text in from a control on a vb form using the same function calls.

Walking the tree (path) from the parent window is sort of a pain. A button on a form in MS Access is 7 children below the main Access window. Using the accexplorer you have to click on parent to get to the parent object and keep going until you find a visible, selectable window. I found the inspect32.exe tool in the AA tools to be more useful than accexplorer:

http://download.microsoft.com/download/7/3...9/inspect32.exe

inspect32.exe gives you an ancestor section at the end that shows all of the required steps to build the path to get from the top level window to the control of interest.

I did have to modify a few lines of code in ActiveAccessibility_Library.au3 to get it to work for me. First, in MSAA_GetObjectByPath:

If ((($Path[$PathIndex][0] = $CurrentChild.accName) AND (StringLen($Path[$PathIndex][0]) = StringLen($CurrentChild.accName))) Or _

(StringLen($Path[$PathIndex][2]) > 0 And $Path[$PathIndex][2] = $CurrentChild.accName)) Then

For some reason without the explicit check on the string length an unnamed child with the correct type would match $Path[$PathIndex][0]. My length check is probably not a good fix but it worked for me.

Second, also in MSAA_GetObjectByPath in the "If IsObj($CurrentChild) Then" conditional:

The "else" section does some checking if the child is found but not an object (I think). If I had a bad object name this just kept throwing up an error so I commented it out during testing. I am not sure when this section would be used, I need to look into the IAccessible interface details more.

In any case, this was a great find that really saved the day for me. Thank you very much.

[jkorsak]

Also, here is an example from my script on pushing a MS Access button:

expandcollapse popup

Dim $i      ; $Path depth counter for convenience when adjusting the path
Dim $iAAccess; IAccessible root object, in this case the MS Access window - for debugging
Dim $iAText ; IAccessible object for the Msgtxt field - for debugging
Dim $iAQuit ; IAccessible object for the Quit button - for debugging

; Build the IAccessible tree to the subform containing the buttons
$i = 0
$Path[$i][0] = $MsAccess_WindowTitle_Main
$Path[$i][1] = $MsAA_ROLE_CLIENT
$Path[$i][2] = ""
$i = $i + 1
$Path[$i][0] = "Workspace"
$Path[$i][1] = $MsAA_ROLE_WINDOW
$Path[$i][2] = ""
$i = $i + 1
$Path[$i][0] = "Workspace"
$Path[$i][1] = $MsAA_ROLE_CLIENT
$Path[$i][2] = ""
$i = $i + 1
$Path[$i][0] = $MsAccess_WindowTitle_Subform
$Path[$i][1] = $MsAA_ROLE_WINDOW
$Path[$i][2] = ""
$i = $i + 1
$Path[$i][0] = $MsAccess_WindowTitle_Subform
$Path[$i][1] = $MsAA_ROLE_CLIENT
$Path[$i][2] = ""
$i = $i + 1

; Get the IAccessible object for the Quit button
$Path[$i][0] = "Quit"
$Path[$i][1] = $MsAA_ROLE_PUSHBUTTON
$Path[$i][2] = ""

;Debug Checks START
;Check to see if we can get the IAccessible object for the Access window
;Check to see if the path is right for the Quit button
;First get the IAccessible object for the Main Access Window
$iAAccess = MSAA_GetAccessibleObject($MsAccess_WindowTitle_Main)
    If Not IsObj($iAAccess) then
        msgbox(0,"Error","$iAAccess is not an object",15)
        Exit(1)
    EndIf

;Now get the IAccessible object for the Quit button
$iAQuit = MSAA_GetObjectByPath($Path, $iAAccess, $iAChildID)
    If Not IsObj($iAQuit) then
        msgbox(0,"Error","$iAQuit is not an object",15)
        Exit(1)
    EndIf
;Debug Checks END

MSAA_DoDefaultActionByPath($MsAccess_WindowTitle_Main, $Path)

I used the debug checks section when I was figuring out how the paths worked to make sure I was getting an object to return. That section and the corresponding variables should be removed in production.

Here is the inspect32.exe output for the Quit button:

How found:  Mouse move (518,502)
    hwnd=0x000E047C 32bit class="OFormSub" style=0x56000000 ex=0x0
Info:   IAcc = 0x0016EE18 VarChild:[VT_I4=0x0]
Impl:   Remote native IAccessible
Annotation ID:  [not supported]

Name:   "Quit"
Value:  none [mnfd]
Role:   push button
State:  focusable
Location:   {l:467, t:487, w:92, h:28}
Description:    ""
Kbshortcut: none [null]
DefAction:  "Press"
Parent: "SIMS FAR v1.8":client
Help:   none [mnfd]
Help Topic: [:0] [Error - Empty string - Should return S_FALSE/NULL instead?]
ChildCount: 0
Window: 0x0020051C class="OForm" style=0x56CA0000 ex=0x140
Children:   Container has no children
Selection:  none [empty]
Ancestors:  "SIMS FAR v1.8" : client : focused,focusable
    "SIMS FAR v1.8" : window : focused,moveable,focusable
    "Workspace" : client : focusable
    "Workspace" : window : focusable
    "SIMS FAR" : client : focusable
    "SIMS FAR" : window : sizeable,moveable,focusable
    "Desktop" : client : normal
    "Desktop" : window : normal
    [ No Parent ]

[rainbowg]

Some weeks ago I got the task to automatic create a lot of databases with the Access 2003 Source Code Control Add-in in a nightly build. First I use the AutoIt Functions like WinActivate and ControlClick. But many Controls in MS-Office, such the menubar (MsoCommandBar) or buttons (bosa_sdm_Mso96) are not visible to those commands. So I use many Send or ControlSend commands. Finally the Build runs not stable. After 10 or 30 minutes it blocks with unreproducible errors.

Then I rewrite the scripts by use of the Microsoft Active Accessibility technology. Now I can address the MS-Office controls by name. I can click at a menu item by use of an object model. The problems are disappeared.

I attached a library with some examples. For now I use a small COM-Wrapper for the Win-API Calls because the Win-API returns a COM-Object (IAccessible) in their parameter list and I can´t implement that in AutoIt. May be someone out there have the knowledge to replace it with pure AutoIt Code.

If you find it useful, please feel free to extend the library or the examples.

Best Regards

[rainbowg]

Hi Fritsche,

Thank you for your sharing, it's very nice!

But, now I am confused about Microsoft Active Accessibility technology, I tried to learn it from MSDN.

Could you give me some advices?

Best Regards

[rainbowg]

For example, how to use this function IAccessible.accLocation