IUIAutomation MS framework automate chrome, FF, IE, ....

[junkew]

Differs due to many different factors like platform version. As inspect.exe is the main ms tool that's the best default tool to check with. And then you also have different tools using different ways of identifying. The uia is a good base from Microsoft but commercial tools like ranorex or testcomplete can give different insights and some helpfull tools look with ocr or ai imaging basically a smart way of bitmap analyzing.

[dev_automation_call]

Hi,

Okay thats really useful. Sooo after trying everything above i still have the issue.  Inspect.exe does show me different trees based on the different OS versions. Not sure what to try next since it seems on windows server more elements get found(I dont want to use any other libs e.g ranorex or img search). I tried swapping out OLEACC.dll and UIAutomationCore.dll (UIA docs references these as core to the functionality of the lib) in system 32 to use the windows server dll's as a last ditch attempt. But this only broke chrome in the way that no tabs load anymore and u get the aw,snap message everytime 😂.

[dev_automation_call]

Actually i did find the item in the tree, the tree as mentioned is slightly different. This is the major difference 

Provider Description: MSAA Proxy (IAccessible2) (unmanaged:UIAutomationCore.DLL)]" - Windows Pro (Not working)

Provider Description: MSAA Proxy (unmanaged:UIAutomationCore.DLL)]" - Windows server (Working)

Can i get rid of IAccessible2 somehow haha

[junkew]

Why do you want the trees to be the same? Finding elements on different platforms or versions needs a certain strategy.

[NassauSky]

Hi @junkew ,

In version V0-70, has anyone questioned the output of ex1_Treewalker? Is it common for the clipping of the first letter or more of the Window in the console output?  ie  <askbar>, <eamViewer Panel>, <ite [6 of 6]> 

Would you recommend I make any code adjustments to accommodate for this?

Thanks!

 

**** Deal with browser windows ****
location: 0<askbar><Taskbar><Shell_TrayWnd><000100AE>
location: 0<eamViewer Panel><TeamViewer Panel (minimized)><TV_ControlWinMinimized><000303DA>
location: 46<ite [6 of 6]><Z:\OfficeCheck\Alt\ex1_Treewalker.au3 - SciTE-Lite [6 of 6]><SciTEWindow><00090034>
location: 8<Google Chrome><Google - Google Chrome><Chrome_WidgetWin_1><000C06A8>

**** Continue with finding 2 browser windows ****
location: 0<askbar><Taskbar><Shell_TrayWnd><000100AE>
location: 0<eamViewer Panel><TeamViewer Panel (minimized)><TV_ControlWinMinimized><000303DA>
location: 46<ite [6 of 6]><Z:\OfficeCheck\Alt\ex1_Treewalker.au3 - SciTE-Lite [6 of 6]><SciTEWindow><00090034>
location: 8<Google Chrome><Google - Google Chrome><Chrome_WidgetWin_1><000C06A8>
Yes found first browser 

**** Deal with 2nd browser window****
location: 0<askbar><Taskbar><Shell_TrayWnd><000100AE>
location: 0<eamViewer Panel><TeamViewer Panel (minimized)><TV_ControlWinMinimized><000303DA>
location: 46<ite [6 of 6]><Z:\OfficeCheck\Alt\ex1_Treewalker.au3 - SciTE-Lite [6 of 6]><SciTEWindow><00090034>
location: 8<Google Chrome><Google - Google Chrome><Chrome_WidgetWin_1><000C06A8>
location: 121<Microsoft? Edge><IUIAutomation MS framework automate chrome, FF, IE, .... - Page 45 - AutoIt Example Scripts - AutoIt Forums - Profile 1 - Microsoft? Edge><Chrome_WidgetWin_1><001201FC>
Yes found second browser 
>Exit code: 0

 

[junkew]

No it was never questioned but if you see a fix please post Just the example code so others can benefit.

[Viszna]

I'm using UIAWrapper.au3 and sometimes I get this error:

"C:\AUTOMATY\UIAWrappers.au3" (1674) : ==> Array variable has incorrect number of subscripts or subscript dimension range exceeded.:
$x = Int($t[1] + ($t[3] / 2))
$x = Int($t[1] + (^ ERROR
->14:05:04 AutoIt3.exe ended.rc:1

The biggest problem is catching the error, the script runs ~10 hours every day.
The function is called dozens of times a day.
The error occurs 2 a month, but it is irritating because it interrupts the script.

Excerpt from UIAWrappers.au3

1668        Local $t
1669        $t = StringSplit(_UIA_getPropertyValue($obj2ActOn, $UIA_BoundingRectanglePropertyId), ";")
1670    ;~  If no parameter detect the middle
1671        If $p1 = 0 Then
1672    ;~  consolewrite(_UIA_getPropertyValue($obj, $UIA_BoundingRectanglePropertyId) & $t[1] & ";" & $t[2] & ";" & $t[3] & ";" & $t[4] & @crlf)
1673    ;~  _winapi_mouse_event($MOUSEEVENTF_ABSOLUTE + $MOUSEEVENTF_MOVE,$t[1],$t[2])
1674            $x = Int($t[1] + ($t[3] / 2))
1675            $y = Int($t[2] + $t[4] / 2)
1676        Else
1677    ;~      calculate location relative in the element found
1678            $x = Int($t[1] + $p1)
1679            $y = Int($t[2] + $p2)
1680        EndIf

Maybe it's worth adding a verification of the $t variable on line 1670 before further operations on the $t array?

I added an error trap - now I'm waiting for it to appear and check what goes to $t.

[junkew]

Most likely $obj2acton is not on the screen or not found and as such you do not get a rectangle. 

So check if the object is there besides that the uiwrappers could be improved regarding errors handling.