Jump to content

Recommended Posts

Posted

as always

Awesome .. tool !
 

Signature beginning:
Please remember: "AutoIt"..... *  Wondering who uses AutoIt and what it can be used for ? * Forum Rules *
ADO.au3 UDF * POP3.au3 UDF * XML.au3 UDF * IE on Windows 11 * How to ask ChatGPT for AutoIt Codefor other useful stuff click the following button:

  Reveal hidden contents

Signature last update: 2023-04-24

Posted

Nothing appears when I start UIASpy.au3 and the script ends -> I've to comment out

If WinExists( "UIASpy - UI Automation Spy Tool" ) Then Return

otherwise it returns always.

 

Nice work. I never worked with UI but maybe somedays...

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Posted

Looks nice, gave no gui.

For whatever reason the - is not fine in winexist (i assume its interpreted as a regex)

changed -minus to a dot for a quick fix

Func UIASpy_Gui()
    ; Only one UIASpy at a time
    consolewrite("hello : " & WinExists( "UIASpy . UI Automation Spy Tool") & @CRLF)
    If WinExists( "UIASpy . UI Automation Spy Tool" ) Then Return
    consolewrite("hello 2")

 

Posted (edited)

Was not working for me. I'm like ?!?!?. lol

;~  If WinExists( "UIASpy - UI Automation Spy Tool" ) Then Return
    If WinExists( "[TITLE:UIASpy - UI Automation Spy Tool;CLASS:AutoIt v3 GUI;]" ) Then Return

that did it. Had the browser open in the same title  =)

PS: I just now got to read your posting @UEZ.

PS2: ..since it take a while to load:

;~  If WinExists( "UIASpy - UI Automation Spy Tool" ) Then Return
    If WinExists( "[TITLE:UIASpy - UI Automation Spy Tool;CLASS:AutoIt v3 GUI;]" ) Then Return
    If WinExists( "[TITLE:UIASpy - UI Automation Spy Tool;CLASS:AutoIt v3;]" ) Then Return
    SplashTextOn("UIASpy - UI Automation Spy Tool", "..loading..")
    
    ... bunch of code ...
    
    ; Show GUI
    GUISetState( @SW_SHOW, $hGui )
    SplashOff()

would be cool too :)

Edited by argumentum

Follow the link to my code contribution ( and other things too ).
FAQ - Please Read Before Posting.
autoit_scripter_blue_userbar.png

Posted (edited)
;~  If WinExists( "UIASpy - UI Automation Spy Tool" ) Then Return
    If Not _Singleton(@ScriptName, 1) Then Return

might help, too.

 

Further, the red rectangle flickers.

Edited by UEZ

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Posted

So far I've commented out that run-once check. But please do not start UIASpy more than once. It'll be a mess. New zip in first post.

While I've developed this script I've primarily focused on the UI Automation code. That is, I've focused on detecting all UI elements correctly, inserting the elements into the treeview, and calculating and displaying detail information in the listview. And not at least optimized the code for performance and speed. Any issues with that part of the code?

CUIAutomation2.au3 is copied directly from junkew's UI Automation thread. To be able to detect all elements in Windows 10 Start menu two new elements is added:

Global Const $UIA_SemanticZoomControlTypeId=50039
Global Const $UIA_AppBarControlTypeId=50040

That's the reason for the update of the file.

The red element rectangle and startup splash texts has got less attention. With regard to the red element rectangle, it's incredibly hard to draw without a little flicker. It's certainly not enough just to delete the old rectangle once and to draw the new rectangle once. This may easily end with a half-deleted old rectangle and a half-drawn new rectangle.

If eg. the target window is restored from a minimized state, the Windows OS will do a lot of redrawing of the window. This may easily delete the red rectangle. You have to continue updating the rectangle until the redrawing of the window is finished.

I would prefer not to write multiple thousand line code just to draw the red rectangle. And I think the focus on the UI Automation code is the proper focus for a script like this. It's a tool a developer can use while developing UI Automation code.

So far, I'll only promise that I'll look at the problem that the red rectangle is not properly drawn on a high resolution display.

Posted

High resolution display
I've multiplied left, top, right and bottom coordinates of the element rectangle with the DPI scale factor in DrawElemRect() near bottom of UIASpy_Detect.au3. Since I don't have a high resolution display I've not been able to test the code. But the rectangle is still drawn correct on my standard screen (DPI scale factor = 1).

Danp2 (or other with a high resolution display) will you test the code and check if the red rectangle is drawn correctly?

Replace the existing version of UIASpy_Detect.au3 with this version:

UIASpy_Detect.au3

Posted (edited)
  On 12/3/2018 at 6:36 PM, LarsJ said:

are you able to do a test?

Expand  

Yes I did. I don't understand much, but it runs. If there is anything in particular you'd want me to try out, let me know. :) 

PS: I put one of my monitors at 100% and the other at 150%. Your code behaved well. No complains.

PS2: pressing F4 while in the 150% monitor it gave me this:
image.thumb.png.cb83d4ff9592eb6d06b212310c686461.png
..so a picture is like 1000 words. ..if this is what you're looking to test out. It does behave as expected at 100%.

Edited by argumentum
clarify

Follow the link to my code contribution ( and other things too ).
FAQ - Please Read Before Posting.
autoit_scripter_blue_userbar.png

Posted

argumentum, This is exactly the information I'm looking for, thank you very much.

There was one thing I forgot to tell. Calculation of the DPI scale factor is not performed dynamically. It's only performed once when UIASpy starts up. If the scale setting is changed on the display, UIASpy must be restarted to calculate the new DPI scale factor.

I'll try to review the code once more during the day and maybe return with a new test later this evening.

Posted

Screen scaling
On Windows 10 a screen scaling different from 100% will likely mean that the red element rectangle is drawn at a wrong position and with a wrong size. The reason for this seems to be that the rectangle returned by $UIA_BoundingRectanglePropertyId is calculated relatively to the logical screen size that matches the scaling factor, and not to the physical size (in pixels) of the screen.

There may be several reasons for a screen scaling different from 100%. One reason may be a high resolution screen where the scaling as default is set to eg. 150%. Another reason may be a normal screen where the scaling manually is set to eg. 125%.

In the first case, a correct drawing of the rectangle may depend on whether or not the target application is DPI aware. In the last case, the rectangle will probably always be mistakenly drawn.


On Windows 7 the element rectangle is still drawn correct even if the scaling is set to eg. 125%. The physical size of the screen seems to be taken into account when $UIA_BoundingRectanglePropertyId is calculated.


Because there are many different situations it's tricky to handle the screen scaling properly. To get around the problem I've decided to add an Options menu where you have to select the scaling factor if it's not 100%.


If $UIA_BoundingRectanglePropertyId is used to eg. set the mouse position to perform a click on the element, the scaling factor must be included to correctly calculate the mouse position.

If the scaling is different from 100% the rectangle values of $UIA_BoundingRectanglePropertyId are simply multiplied with the scaling factor.

In this situation both the original rectangle as returned by $UIA_BoundingRectanglePropertyId and the scaled rectangle are displayed in the right pane listview.

New zip-file in bottom of first post.

Posted (edited)

@LarsJ Your tools just help me to solve one problem.

Now I know what was wrong.

Thanks for this great tool.
 

I check many things with this tool and I have some questions or even feature request,

Here is a screenshot which will help me to describe what I mean.

image.thumb.png.f444a1783bd58b37959ee01071786542.png

 

These rose fields are of course cut out for security reasons and GDPR rules.

My problem was that I was checking for WinExist('****','Weryfikacja danych') but I do not know that there are 3 different windows that match this "pattern", as they were obscured by other "INNER WINDOW/PANEL" (btw. how they should be properly called ?) placed inside the same application (inside MAIN WINDOW).

To understand what I mean I change each INNER window size and position to show what I mean in this following screenshot.

image.png.58a0d8ff249d96b44d7eb5f841ee1147.png

 

QUESTION 1:
How I can save entire structure to file ? (when I try ten I get only info about current selected element , but not all sub elements).


QUESTION 2:
Do you think an option to save a selected Treview node, with all sub nodes, to an XML file would be  usefull ?
I can take this subtask as a my part of contribution to your project, if you would accept this offer.

Because I need a tool to compare what change is made in some "UIA"  structures when I change some feautres on CRM WINDOW (which use my Clients), and find out a way to automate some "events" but firstly I must to know which element is changing by user interaction.


QUESTION 3:
Would you move this project to GitHub ?

QUESTION 4:
When I use 

image.png.010eebc19a0afdac5399857b2ab1cac0.png

as a effect, right panel change  his own content - like this:

image.png.8702be2bcc353b506df60af0cebfc6a2.png

is it intentional ?

QUESTION 5:
Do you think that this project could be moved here ? maybe in a near future.

https://www.autoitscript.com/forum/forum/49-autoit-projects-and-collaboration/

QUESTION 6:

Is there a way to use this Tool to search for a specyfic word in this entire structure ?
"Find" and "Find Next" feature would be great.

Regards,

mLipok

 

EDIT:
QUESTION 7:
I was trying to get UIA structure from some menu....

image.png.a4bd07445eb4f81b6b9b859aa20d8c2f.png

I found that this is imposible because , when I open menu in my CRM DOCUMENT Editor then I can't use "Update all chlids of elements", as using this options causes the menu to be closed in the CRM Editor.

So I thought about new feature:
Could you add an additional option "Update all chlids of elements after X seconds" where X should be each time indicated by user ?

I was even do some experiment ....

Case $idUpdateAllChildsOfElement ; Update all childs of element
                                Do
                                    $iUpdateWaitBefor = Number(InputBox('Wait time','Provide amount of seconds to wait before update',0))
                                Until IsInt($iUpdateWaitBefor)
                                Sleep($iUpdateWaitBefor*1000)

                                $idTVExpand = 0 ; Disable $idTVExpand
                                _GUICtrlTreeView_BeginUpdate( $hTV )
                                UIASpy_DelChilds( $iIdx )
                                $iCntElems = 0
                                UIASpy_AddChilds( $iIdx, True )
                                DllCall( "user32.dll", "lresult", "SendMessageW", "hwnd", $hTV, "uint", $TVM_ENSUREVISIBLE, "wparam", 0, "handle", $aElems[$iIdx][4] )
                                _GUICtrlTreeView_EndUpdate( $hTV )
                                $idTVExpand = $idTVExpand0
                                ToolTip("")
                                If $iUpdateWaitBefor > 0 Then MsgBox(0, 'Info', 'Structure is updated')

but this is not working as I intent - I can't get SubItem structure for this Menu - the same problem is with getting NOTEPAD "File" menu structure.

 

Edited by mLipok

Signature beginning:
Please remember: "AutoIt"..... *  Wondering who uses AutoIt and what it can be used for ? * Forum Rules *
ADO.au3 UDF * POP3.au3 UDF * XML.au3 UDF * IE on Windows 11 * How to ask ChatGPT for AutoIt Codefor other useful stuff click the following button:

  Reveal hidden contents

Signature last update: 2023-04-24

Posted

Question 1
"Copy structure to clipboard" does not calculate the element subtree. The procedure to copy the entire subtree structure to clipboard is (right click the treeview item):
1. Update all childs of element
2. Copy structure to clipboard

Question 2
I rarely use the XML format. Usually you can do the same with a plain text file, and with much faster code. But if you'll write the code and it's not too slow, I'll add it to the zip-file in first post.

Question 3
GitHub. No.

Question 4
Show structure in listview. This is intentional.
1. It takes some time to calculate the structure of a subtree with 1000 elements or more. The calculation is complete when the structure appears in the listview.
2. Perhaps you only need a single or a few rows in the structure. You can select these rows in the listview and use "Copy all or selected items to clipboard" in the "Right pane" main menu to copy only those few rows.

Question 6
A Search feature is planned for the next version.

Question 7
Use the function keys to handle a menu. Here's an example with Notepad File menu:
Run UIASpy.au3. Open Notepad. Click File menu. Place mouse cursor over first menu item. Press F1. Press F2. Switch to UIASpy. It should look like this:
YDCmj8g.png
Left: F1,  Middle: F2,  Right: Treeview (red items because the corresponding UI elements doesn't exist, the menu is closed)

Question 5
Move example to "Projects and collaboration" forum.
I think such an example is too small for "Projects and collaboration". But a more general topic, eg. "UI Automation Updates" might be interesting to start up under "Projects and collaboration". This can include a discussion of a spy tool.

First of all, an update of the code in CUIAutomation2.au3 (constants and definitions) is required. CUIAutomation2.au3 contains Windows 7 code. To support Windows 8 and Windows 10 probably 50 - 100 new interfaces, methods, properties and constants are added which are not included in CUIAutomation2.au3.

An UI Automation thread in "Projects and collaboration" can certainly be interesting.

Posted (edited)

@LarsJ Thanks for all your answers.
I know this is not thread for teaching how to use UIA at all.

..but it would be helpfull to show one example how to use UIASpy for create simple notepad automation.

So here is what I propose.

In your previous post you answer to my Q7 and as an example you post this following screenshot

YDCmj8g.png

 

Question/Request:

Could you create few next screenshot which will shows how to get structur for "Edit1" , "FileName" and "SaveButton"

Then would you create an AU3 example to show how to use this information to create a simple script to fill up "Edit1" with HelloWorld, click "Save As..." enter filename and finally save it to disc.

It could then be considered a complementary example of how to use UIASpy to create a simple notepad automation.

Regards,

mLipok

 

Edited by mLipok

Signature beginning:
Please remember: "AutoIt"..... *  Wondering who uses AutoIt and what it can be used for ? * Forum Rules *
ADO.au3 UDF * POP3.au3 UDF * XML.au3 UDF * IE on Windows 11 * How to ask ChatGPT for AutoIt Codefor other useful stuff click the following button:

  Reveal hidden contents

Signature last update: 2023-04-24

Posted (edited)

Are you refering to my Request ?

EDIT... or to one of my last questions  which I ask recently in this topic ?
or just generally ...

Edited by mLipok

Signature beginning:
Please remember: "AutoIt"..... *  Wondering who uses AutoIt and what it can be used for ? * Forum Rules *
ADO.au3 UDF * POP3.au3 UDF * XML.au3 UDF * IE on Windows 11 * How to ask ChatGPT for AutoIt Codefor other useful stuff click the following button:

  Reveal hidden contents

Signature last update: 2023-04-24

Posted
  On 12/17/2018 at 7:48 PM, junkew said:

 

@mLipok in the uia thread notepad and calc examples are available.

Expand  

As I stated before.

  Quote

I know this is not thread for teaching how to use UIA at all.

Expand  

My concept was to show how to use UIASpy in order to create "working" Notepad automation, to show corelation with using UIASpy tool and final simple code.

 

Signature beginning:
Please remember: "AutoIt"..... *  Wondering who uses AutoIt and what it can be used for ? * Forum Rules *
ADO.au3 UDF * POP3.au3 UDF * XML.au3 UDF * IE on Windows 11 * How to ask ChatGPT for AutoIt Codefor other useful stuff click the following button:

  Reveal hidden contents

Signature last update: 2023-04-24

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...