Jump to content
Sign in to follow this  
toto22

SOLVED: Get Text from UI

Recommended Posts

toto22
Posted (edited)

Please help me to get text from another program using UI

I need my program to extract symbol and its description from TC2000, basically a "element title". I want it to return "INNT Innovate Biopharmaceuticals, Inc Healthcare Sector(D) Biotechnology" (or any other text if symbol changes)

 

my wrapper is:

Mouse position is retrieved 1194-405
At least we have an element title: [INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
] class: [TextBlock]

Having the following values for all properties: 
Title is: <INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
>   Class   := <TextBlock>  controltype:= <UIA_TextControlTypeId>   ,<50020>    , (0000C364)    989;356;406;183
*** Parent Information top down ***
2: Title is: <TC2000 ® Version 17.0.6435.29524>    Class   := <Window> controltype:= <UIA_WindowControlTypeId> ,<50032>    , (0000C370)    51;24;1712;1124
"Title:=TC2000 ® Version 17.0.6435.29524;controltype:=UIA_WindowControlTypeId;class:=Window""  
1: Title is: <> Class   := <WorkspaceTabControl>    controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    59;123;1696;1017
"Title:=;controltype:=UIA_CustomControlTypeId;class:=WorkspaceTabControl""  
0: Title is: <> Class   := <WebChart>   controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    675;125;1080;612
"Title:=;controltype:=UIA_CustomControlTypeId;class:=WebChart"" 


;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:=TC2000 ® Version 17.0.6435.29524;controltype:=UIA_WindowControlTypeId;class:=Window")    ;TC2000 ® Version 17.0.6435.29524
_UIA_setVar("oP2","Title:=;controltype:=UIA_CustomControlTypeId;class:=WorkspaceTabControl")    ;
_UIA_setVar("oP3","Title:=;controltype:=UIA_CustomControlTypeId;class:=WebChart")   ;

;~ $oUIElement=_UIA_getObjectByFindAll("INNT
InnovateBiopharmaceuticals,Inc
HealthcareSector(D)
Biotechnology
.mainwindow", "title:=INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
;ControlType:=UIA_TextControlTypeId", $treescope_subtree)
_UIA_setVar("oUIElement","Title:=INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
;controltype:=UIA_TextControlTypeId;class:=TextBlock") ;ControlType:=UIA_TextControlTypeId;classname:=TextBlock")

;~ Actions split away from logical/technical definition above can come from configfiles 

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")

;~_UIA_action($oUIElement","highlight")
;~_UIA_action($oUIElement,"click")
;~_UIA_action("oUIElement","highlight")
;~_UIA_action("oUIElement","click")


;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP2=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=TC2000 ® Version 17.0.6435.29524;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children)   
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_CustomControlTypeId;class:=WorkspaceTabControl", $treescope_children)    
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_CustomControlTypeId;class:=WebChart", $treescope_children)   
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll("INNT
InnovateBiopharmaceuticals,Inc
HealthcareSector(D)
Biotechnology
.mainwindow", "title:=INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
;ControlType:=UIA_TextControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
;ControlType:=UIA_TextControlTypeId", $treescope_subtree)
;~_UIA_action($oUIElement,"highlight")
_UIA_action($oUIElement,"click")


*** Detailed properties of the highlighted element ***
UIA_title:= <INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
>
UIA_text:= <INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
>
UIA_regexptitle:= <INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
>
UIA_class:= <TextBlock>
UIA_regexpclass:= <TextBlock>
UIA_iaccessiblechildId:= <0>
UIA_id:= <Watermark>
UIA_handle:= <0>
UIA_RuntimeId:= <7;1244;18409771>
UIA_BoundingRectangle:= <989;356;406;183>
UIA_ProcessId:= <1244>
UIA_ControlType:= <50020>
UIA_LocalizedControlType:= <text>
UIA_Name:= <INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <False>
UIA_IsEnabled:= <True>
UIA_AutomationId:= <Watermark>
UIA_ClassName:= <TextBlock>
UIA_ClickablePoint:= <1192;447>
 

 

Edited by toto22

Share this post


Link to post
Share on other sites
toto22

I can do it in C#. However I cant figure it out in Autoit. Please help

C# code

FlaUI.Core.Application app = FlaUI.Core.Application.Attach(System.Diagnostics.Process.GetProcessesByName("TMain")[0].Id);
            using (var automation = new UIA3Automation())
            {
                var window = app.GetMainWindow(automation);
                Console.WriteLine(window.Title);
                
                Price = "";
                do
                {
                    try
                    {
                        var child = window.FindFirstChild(cf => cf.ByAutomationId("WorkspaceManager"));
                        var child2 = child.FindFirstChild(cf => cf.ByClassName("WebChart"));
                        var child3 = child2.FindFirstChild(cf => cf.ByAutomationId("Watermark"));

                        //var pattern1 = child3.Patterns.Value.Pattern;

                        var pattern1 = child3.Properties.Name.Value;
                        Symbol2 = child3.Properties.Name.Value;

                        int index = Symbol2.IndexOf(" ");
                        if (index >= 0)
                            Symbol2 = Symbol2.Substring(0, index);
                    }
                    catch (Exception)
                    {

                    }
                } while ( Price == "g");
                //string Symbol2 = Symbol.ToString();
                textBox1.Text = Symbol2;

 

Share this post


Link to post
Share on other sites
Earthshine

so, the code that junkew's spy tool returns doesn't work? it's this part

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")

 


My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
toto22

Hello and thank you. It does set focus but how do I get my text?

Share this post


Link to post
Share on other sites
Earthshine
Posted (edited)

@junkew is your person for this one but I think it has to do with

UIA_IsTextPatternAvailable:= <False>

 

if you can make a C# dll that can do it, you can call it from AutoIt, no?

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
toto22

I'm so confused with Autoit UI commands

Share this post


Link to post
Share on other sites
Earthshine
Posted (edited)

well, to be fair, this is the IUIAutomation wrapper written by @junkew you are using, no? He will probably be around and see this. He's who I would ask.

also, what you use in C# is none other than a wrapper. which is ok.

Edited by Earthshine
  • Like 1

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
Earthshine

let me axe you this... do you have a function like this you could use in those wrappers?

Func _UIA_ControlGetText($hWnd, $controlID)


My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
toto22
Posted (edited)

i'm not sure,

this where im at -.- 

;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

local $oP1 = _UIA_setVar("oP1","Title:=TC2000 ® Version 17.0.6435.29524;controltype:=UIA_WindowControlTypeId;class:=Window")   ;TC2000 ® Version 17.0.6435.29524
local $oP2 = _UIA_setVar("oP2","Title:=;controltype:=UIA_CustomControlTypeId;class:=WorkspaceTabControl")   ;
local $oP3 = _UIA_setVar("oP3","Title:=;controltype:=UIA_CustomControlTypeId;class:=WebChart")
Local $oUIElement=_UIA_getObjectByFindAll($oP3,"controltype:=UIA_TextControlTypeId;class:=TextBlock") ;ControlType:=UIA_TextControlTypeId;classname:=TextBlock")

MsgBox(1,"",$oUIElement)

 

Edited by toto22

Share this post


Link to post
Share on other sites
Earthshine
Posted (edited)

ok, it's junkew's

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
toto22

no, nothing, i dunno if i'm doing it right

Share this post


Link to post
Share on other sites
Earthshine
Posted (edited)

hmmmm, can you try modifying that last one

Local $oUIElement=_UIA_getObjectByFindAll($oP3,"Title:=;controltype:=UIA_TextControlTypeId;class:=TextBlock")
Edited by Earthshine

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
Earthshine
;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

local $oP1 = _UIA_setVar("oP1","Title:=TC2000 ® Version 17.0.6435.29524;controltype:=UIA_WindowControlTypeId;class:=Window")   ;TC2000 ® Version 17.0.6435.29524
local $oP2 = _UIA_setVar("oP2","Title:=;controltype:=UIA_CustomControlTypeId;class:=WorkspaceTabControl")   ;
local $oP3 = _UIA_setVar("oP3","Title:=;controltype:=UIA_CustomControlTypeId;class:=WebChart")
Local $oUIElement=_UIA_getObjectByFindAll($oP3,"Title:=;controltype:=UIA_TextControlTypeId;class:=TextBlock")

MsgBox(1,"",$oUIElement)

try it

  • Like 1

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
toto22

Thank you so much for your help. However, It still returns blank message box. Ill keep playing with it and if I make any progress, ill let you know. Thank you so much for your time and help

  • Like 1

Share this post


Link to post
Share on other sites
Earthshine
Posted (edited)

there is that other one you can try--based on the one you already use though

https://github.com/jvanegmond/au3_uiautomation

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
Earthshine

can you do this?

$oUIElement=_UIA_getObjectByFindAll($oP0, "title:=*;ControlType:=UIA_TextControlTypeId", $treescope_subtree)

 

  • Like 1

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
toto22

you know it does find object and highlights and clicks it. However getting text is a whole different story. there should be a command to show text or header.

Im taking a lil break now. thank you brotha

 

  • Like 1

Share this post


Link to post
Share on other sites
Earthshine
Posted (edited)

ya, try that one from GitHub mate. it also has the gettext function

hey, is it our Jos? 

https://github.com/jvanegmond

Edited by Earthshine
  • Like 1

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
Earthshine
Posted (edited)

I see there is a thread where you can set the text, I bet you can get the text in such a way? 

_UIA_action("204022.mainwindow","setvalue using keys","hello world")

maybe

_UIA_action("YourApp","getValue")

YES, i looked in the UIWrappers.au3, there is an 'getValue' you should be able to use!

 

Edited by Earthshine
  • Like 1

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
Earthshine
Posted (edited)

ok, update, so I used Microsoft Word 2016 as a sample for this code and made a test docx file, with the text in it--"This is a test." and ran the code below and it RETURNED MY TEXT!! using junkew's wrapper, the one you already use.

;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=This is a test.docx - Word;controltype:=UIA_WindowControlTypeId;class:=OpusApp", $treescope_children)
_UIA_Action($oP4,"setfocus")
Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=_WwF", $treescope_children)
_UIA_Action($oP3,"setfocus")
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=This is a test.docx;controltype:=UIA_PaneControlTypeId;class:=_WwB", $treescope_children)
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=This is a test.docx;controltype:=UIA_DocumentControlTypeId;class:=_WwG", $treescope_children)
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=Page 1;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children)
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll("Page1content.mainwindow", "title:=Page 1 content;ControlType:=UIA_EditControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP2, "title:=Page 1 content;ControlType:=UIA_EditControlTypeId", $treescope_subtree)
;~_UIA_action($oUIElement,"highlight")
Local $string = _UIA_action($oUIElement,"getValue")
MsgBox($MB_OK,"", $string)

 

Edited by Earthshine
  • Like 2

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Similar Content

    • caramen
      By caramen
      Is that correct?
      With SimpleSpy i can click on controls without any include/udf ON CHROME?
    • drapdv
      By drapdv
      So, I have this odd issue happening with UIAWrappers.  When I run simplespy.au3 through SciTE, it finds the object I'm trying to work with, no problem. But, when I compile simplespy.exe, it can't find it.  Another script that utilizes UIAWrappers is experiencing the same phenomenon. EDIT: Compiling as 32-bit or 64-bit does not affect this behavior.
      The trouble starts at the very first object I try to get via _UIA_getObjectByFindAll, so I included the $properties2Match array and the object properties that were returned by _UIA_getPropertyValue for that first object both when running from SciTE and when compiled.
      It always finds the matches for the title and control type properties.  But when compiled, it fails to find the Automation ID property.  Any ideas as to why this would be happening?  Hoping junkew will see this and have an idea.
       
      These are the $properties2Match
      30005|Screen/PVN - ###.###.##.## - CDK Drive|0|False
      30003|50032|0|False
      30011|Portal|0|False
       
      Object properties when running from SciTE:
      <propertymatching>       
              j:1 propID:[30005] expValue:[Screen/PVN - ###.###.##.## - CDK Drive]actualValue:[Screen/PVN - ###.###.##.## - CDK Drive]1
              j:2 propID:[30003] expValue:[50032]actualValue:[50032]1
              j:3 propID:[30011] expValue:[Portal]actualValue:[Portal]1  
      </propertymatching>
       
      Object properties when script is compiled:
      <propertymatching>
              j:1 propID:[30005] expValue:[Screen/PVN - ###.###.##.## - CDK Drive]actualValue:[Screen/PVN - ###.###.##.## - CDK Drive]1
              j:2 propID:[30003] expValue:[50032]actualValue:[50032]1
              j:3 propID:[30011] expValue:[Portal]actualValue:[]0  
      </propertymatching>
       
    • RomanY
      By RomanY
      Good evening!
      Can somebody help me with getting text from wpf controls. With a help of inspect.exe I found element selector and now I need to get text from some button and assert that text is  as expected?
       
    • souldjer777
      By souldjer777
      Good Evening Everyone
      Long time since I've been in the forums - missing AutoIt and programming dearly - but on with the question... I hope this one is easy
      I am using SimpleSpy / UIAWrappers.au3 to automate an application. I basically have a "Stop" button that I am waiting to change from UIA_IsEnabled:= <False> to UIA_IsEnabled:= <True>
      How do I check for the UIA_IsEnabled condition ?  Of course SimpleSpy gives all the details of GUI elements... but what code do I use to just check the GUI "Stop" buttons UIA_IsEnabled condition?
      Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Stop;ControlType:=UIA_ButtonControlTypeId", $treescope_subtree)
      Thanks!
    • careca
      By careca
      Hey there, i have the code from simplespy but have no idea about what to use, got the UIA_V0_64 files.
      I look at the examples but i get lost, i ask for guideance in this matter.
      Best regards.

       
      To get to this window simply run:
      control.exe /name Microsoft.DefaultPrograms /page pageFileAssoc
      select a filetype and click "change program"
×