Jump to content

Recommended Posts

Posted

I work with "Microsoft Dynamics NAV 2018" and often need to get data out to various places.

With the normal Autoit commands I can get some of the data out, but not stuff in tables, non edits fields etc.
With the various UIA spy tools I can see that the text is there but can't figure out how to get it all out to a string or an array.

I basically want to activate a window and press a button and all the text is read out (into a string or array), kinda like the autoit info tool does it..

Is it possible at all to read through the entire tree and get all, say the:
$UIA_ValueValuePropertyId, $UIA_LegacyIAccessibleNamePropertyId, $UIA_LegacyIAccessibleValuePropertyId
values...

I don't need to interact in any way (click buttons etc.) in the window, just get the text..
Can anybody guide me or maybe have a simple example because the UIA info in the various threads, I'm sorry to say, goes right over my head...

 

Regards, Martin

Posted

If you have luck you can type in your addressbar

javascript:alert(document.body.innerText)

"I'm sorry to say, goes right over my head..."

Nice moment to start with example 1-10 in the examples section on IUIAutomation.

The alternative is the WebDriver examples but also that will have a steep learning curve.

 

Posted

Hi, thanks for the reply.

The app is a windows application and not browser based..

I've tried the various examples and they work of course, but I cant really follow..
Is there an example of just reading text, not pushing buttons or any thing? I think i've looked at all of them but might have missed something..

"Normal" Autoit code I can usually figure out, but been away from it for some years and due to personal issues the more complicated stuff is hard for me to grasp these days..

Posted
  1. Study with inspect.exe or other spy tools referenced in FAQ 31 your tree and see if its worth the effort.
    If all data is there you can walk the tree in AutoIt in different ways
  2. Best file to study: ex1_Treewalker.au3
  3. especiall findthemall function is then usefull to start with
  4. If ordering is important you probably have to create the walking over the tree yourself which can be hard an challenging certainly when you have to filter out unwanted information.

As you are saying tables it still can be done but gets harder as then you have to navigate the cells by row and column which is most likely possible.

Its possible when its shown to the spying tools but can be hard as UIA is a big model to understand.

 

Posted (edited)
  On 1/10/2022 at 7:53 PM, eatfishdk said:

The app is a windows application and not browser based.

Expand  

You could try:

WinGetText()

or the examples/solutions mentioned by @junkew or ....

Try this new UDF:

 

Edited by mLipok

Signature beginning:
Please remember: "AutoIt"..... *  Wondering who uses AutoIt and what it can be used for ? * Forum Rules *
ADO.au3 UDF * POP3.au3 UDF * XML.au3 UDF * IE on Windows 11 * How to ask ChatGPT for AutoIt Codefor other useful stuff click the following button:

  Reveal hidden contents

Signature last update: 2023-04-24

Posted
  On 1/10/2022 at 8:08 PM, junkew said:
  1. Study with inspect.exe or other spy tools referenced in FAQ 31 your tree and see if its worth the effort.
    If all data is there you can walk the tree in AutoIt in different ways
  2. Best file to study: ex1_Treewalker.au3
  3. especiall findthemall function is then usefull to start with
  4. If ordering is important you probably have to create the walking over the tree yourself which can be hard an challenging certainly when you have to filter out unwanted information.

As you are saying tables it still can be done but gets harder as then you have to navigate the cells by row and column which is most likely possible.

Its possible when its shown to the spying tools but can be hard as UIA is a big model to understand.

Expand  

All the data (text) I need is visible to the spy tools, so it's there. I'm not worried about getting to much data as that is relatively easy to manage if I just get a string or array out with the help of UIA...
But I'll try the different examples and see if I can figure it out.

Thanks again for the reply.

 

 

Posted
  On 1/10/2022 at 9:30 PM, mLipok said:

You could try:

WinGetText()

or the examples/solutions mentioned by @junkew or ....

Try this new UDF:

 

Expand  

Wingettext and the other methods in "plain" autoit doesn't give me all the text, just a small subset..
I'll might take a look at the udf you linked, but a lot of the text is hidden because the size of the window so need to scroll to see it all.. 

But thanks for the suggestion!

 

Posted (edited)

Have you tried CTRL-A then CTRL-C ?

Edited by Werty

Some guy's script + some other guy's script = my script!

Posted
  On 1/11/2022 at 11:29 AM, Werty said:

Have you tried CTRL-A then CTRL-C ?

Expand  

Erhm.. Its a joke right?

The text I need are in boxes like this:
image.png.f72aecd9379c31381d84e25183a97d8d.png

If copy/paste was possible, I've wouldn't have looked at UIA... 

And no, you can't right click on these fields and copy... But they are available through the UIA spy tools..

 

Posted

Here you have the findthemall function which you can tweak to get more/less properties printed.

  • Below example just dumps name and class
    • you can just extend this line with your properites 
      ConsoleWrite(
      "Title is: " & _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) & @TAB 
      & "Class=" & _UIA_getPropertyValue($oUIElement, $uia_classnamepropertyid) & @TAB
      & "Value=" & _UIA_getPropertyValue($oUIElement, $UIA_ValueValuePropertyId) & @TAB
      & "Accessible name=" & _UIA_getPropertyValue($oUIElement, $UIA_LegacyIAccessibleNamePropertyId) & @TAB
      & "Accessible value=" & _UIA_getPropertyValue($oUIElement, $UIA_LegacyIAccessibleValuePropertyId) & @TAB
      & @CRLF)

       

  • With UIA_DumpThemAll you get all properties which is overkill so you have to tweak
  • When the class or controltype is different it could be you have to look for other properties to get the text thats relevant for you
Func findThemAll($oElementStart, $TreeScope)
    Local $hTimer = TimerInit()
;~  Get result with findall function alternative could be the treewalker
    Dim $pCondition, $pTrueCondition
    Dim $pElements, $iLength

    $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition)
    $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition)
;~  $oCondition1 = _AutoItObject_WrapperCreate($aCall[1], $dtagIUIAutomationCondition)
;~ Tricky to search all descendants on html objects or from desktop/root element
    $oElementStart.FindAll($TreeScope, $oCondition, $pElements)

    $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray)

    $oAutomationElementArray.Length($iLength)
    For $i = 0 To $iLength - 1; it's zero based
        $oAutomationElementArray.GetElement($i, $UIA_pUIElement)
        $oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
        ConsoleWrite("Title is: " & _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) & @TAB & "Class=" & _UIA_getPropertyValue($oUIElement, $uia_classnamepropertyid) & @CRLF)
        
;~          if _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId)="<your object of interest name>" then
;~              $t = StringSplit(_UIA_getPropertyValue($oUIElement, $UIA_BoundingRectanglePropertyId), ";")
;~              _UIA_DrawRect($t[1], $t[3] + $t[1], $t[2], $t[4] + $t[2])
;~              consolewrite(_UIA_getAllPropertyValues($UIA_oUIElement) & @CRLF)           
;~          endif
        
    Next

    Local $fDiff = TimerDiff($hTimer)
    Consolewrite("Findthemall took: " & $fDiff & " milliseconds" & @CRLF & @CRLF)

EndFunc   ;==>findThemAll

 

Posted
  On 1/11/2022 at 11:48 AM, junkew said:

Here you have the findthemall function which you can tweak to get more/less properties printed.

  • Below example just dumps name and class
    • you can just extend this line with your properites 
      ConsoleWrite(
      "Title is: " & _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) & @TAB 
      & "Class=" & _UIA_getPropertyValue($oUIElement, $uia_classnamepropertyid) & @TAB
      & "Value=" & _UIA_getPropertyValue($oUIElement, $UIA_ValueValuePropertyId) & @TAB
      & "Accessible name=" & _UIA_getPropertyValue($oUIElement, $UIA_LegacyIAccessibleNamePropertyId) & @TAB
      & "Accessible value=" & _UIA_getPropertyValue($oUIElement, $UIA_LegacyIAccessibleValuePropertyId) & @TAB
      & @CRLF)

       

  • With UIA_DumpThemAll you get all properties which is overkill so you have to tweak
  • When the class or controltype is different it could be you have to look for other properties to get the text thats relevant for you
Func findThemAll($oElementStart, $TreeScope)
    Local $hTimer = TimerInit()
;~  Get result with findall function alternative could be the treewalker
    Dim $pCondition, $pTrueCondition
    Dim $pElements, $iLength

    $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition)
    $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition)
;~  $oCondition1 = _AutoItObject_WrapperCreate($aCall[1], $dtagIUIAutomationCondition)
;~ Tricky to search all descendants on html objects or from desktop/root element
    $oElementStart.FindAll($TreeScope, $oCondition, $pElements)

    $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray)

    $oAutomationElementArray.Length($iLength)
    For $i = 0 To $iLength - 1; it's zero based
        $oAutomationElementArray.GetElement($i, $UIA_pUIElement)
        $oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
        ConsoleWrite("Title is: " & _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) & @TAB & "Class=" & _UIA_getPropertyValue($oUIElement, $uia_classnamepropertyid) & @CRLF)
        
;~          if _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId)="<your object of interest name>" then
;~              $t = StringSplit(_UIA_getPropertyValue($oUIElement, $UIA_BoundingRectanglePropertyId), ";")
;~              _UIA_DrawRect($t[1], $t[3] + $t[1], $t[2], $t[4] + $t[2])
;~              consolewrite(_UIA_getAllPropertyValues($UIA_oUIElement) & @CRLF)           
;~          endif
        
    Next

    Local $fDiff = TimerDiff($hTimer)
    Consolewrite("Findthemall took: " & $fDiff & " milliseconds" & @CRLF & @CRLF)

EndFunc   ;==>findThemAll

 

Expand  

Thank you very much, I'll have a look at it later today!

 

Posted

@junkew Just wanted to say thank you for getting me started!

I've finally got some code that dumps all the stuff I need from the program.
Now I just need to sort the stuff I need to get out and make it a bit faster! 

Again! Thanks alot!'

Regards, Martin

Posted

Speed is probably due to the true condition which basically means return all objects which can be a huge array

$UIA_oUIAutomation.CreateTrueCondition($pTrueCondition)
    $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition)
;~  $oCondition1 = _AutoItObject_WrapperCreate($aCall[1], $dtagIUIAutomationCondition)
;~ Tricky to search all descendants on html objects or from desktop/root element
    $oElementStart.FindAll($TreeScope, $oCondition, $pElements)

These conditions you can make with and, or logic.

See below a filter on ControlType and Name. Probably in your scenario I would filter on controltype and or condition (createOrCondition)

Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  Local $pCondition2 ; $UIA_NamePropertyId is LOCALIZED and maybe CASE SENSITIVE
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "File", $pCondition2 ) ; File <<<<<<<<<<<<<<<<<<<<
  If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pCondition2 OK" & @CRLF )

  ; And condition
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF

That's where I frequently use the findThemAll function for to tweak the filters.

The alternative for findall are the treewalkers where you can fully walk the element tree yourself but it depends on what you feel is most easiest to handle.

References

  • See for example the different examples 1-25 that try to explain all the details

 

  • @LarsJmade some wonderfull example threads and some tooling like UIASpy 

See for example

 

Posted

Hi

I've made some good progress and are getting the text I need.
I'm going to look into those conditions when I have some more time, had some issues getting them to work right but thats me not really getting it 🙂
Looking at the posts/examples right now.

But I got it from 8-9 seconds to about one (theres a ton of stuff in the ui..) so thats pretty good for right now!

Again thanks alot @junkew for the help!

 

  • 1 year later...
Posted
  On 1/11/2022 at 11:48 AM, junkew said:
Func findThemAll($oElementStart, $TreeScope)
Expand  

What should be passed as $TreeScope ?

Signature beginning:
Please remember: "AutoIt"..... *  Wondering who uses AutoIt and what it can be used for ? * Forum Rules *
ADO.au3 UDF * POP3.au3 UDF * XML.au3 UDF * IE on Windows 11 * How to ask ChatGPT for AutoIt Codefor other useful stuff click the following button:

  Reveal hidden contents

Signature last update: 2023-04-24

Posted (edited)
  On 11/19/2023 at 2:44 PM, mLipok said:

What should be passed as $TreeScope ?

Expand  

I think I got it please anybody confirm that one of this following const should be used

; enum TreeScope
Global Const $TreeScope_Element     =  1
Global Const $TreeScope_Children    =  2
Global Const $TreeScope_Descendants =  4
Global Const $TreeScope_Subtree     =  7
Global Const $TreeScope_Parent      =  8
Global Const $TreeScope_Ancestors   = 16

 

Edited by mLipok

Signature beginning:
Please remember: "AutoIt"..... *  Wondering who uses AutoIt and what it can be used for ? * Forum Rules *
ADO.au3 UDF * POP3.au3 UDF * XML.au3 UDF * IE on Windows 11 * How to ask ChatGPT for AutoIt Codefor other useful stuff click the following button:

  Reveal hidden contents

Signature last update: 2023-04-24

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...