Jump to content
Sign in to follow this  
toto22

SOLVED: Get Text from UI

Recommended Posts

Please help me to get text from another program using UI

I need my program to extract symbol and its description from TC2000, basically a "element title". I want it to return "INNT Innovate Biopharmaceuticals, Inc Healthcare Sector(D) Biotechnology" (or any other text if symbol changes)

 

my wrapper is:

Mouse position is retrieved 1194-405
At least we have an element title: [INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
] class: [TextBlock]

Having the following values for all properties: 
Title is: <INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
>   Class   := <TextBlock>  controltype:= <UIA_TextControlTypeId>   ,<50020>    , (0000C364)    989;356;406;183
*** Parent Information top down ***
2: Title is: <TC2000 ® Version 17.0.6435.29524>    Class   := <Window> controltype:= <UIA_WindowControlTypeId> ,<50032>    , (0000C370)    51;24;1712;1124
"Title:=TC2000 ® Version 17.0.6435.29524;controltype:=UIA_WindowControlTypeId;class:=Window""  
1: Title is: <> Class   := <WorkspaceTabControl>    controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    59;123;1696;1017
"Title:=;controltype:=UIA_CustomControlTypeId;class:=WorkspaceTabControl""  
0: Title is: <> Class   := <WebChart>   controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    675;125;1080;612
"Title:=;controltype:=UIA_CustomControlTypeId;class:=WebChart"" 


;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:=TC2000 ® Version 17.0.6435.29524;controltype:=UIA_WindowControlTypeId;class:=Window")    ;TC2000 ® Version 17.0.6435.29524
_UIA_setVar("oP2","Title:=;controltype:=UIA_CustomControlTypeId;class:=WorkspaceTabControl")    ;
_UIA_setVar("oP3","Title:=;controltype:=UIA_CustomControlTypeId;class:=WebChart")   ;

;~ $oUIElement=_UIA_getObjectByFindAll("INNT
InnovateBiopharmaceuticals,Inc
HealthcareSector(D)
Biotechnology
.mainwindow", "title:=INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
;ControlType:=UIA_TextControlTypeId", $treescope_subtree)
_UIA_setVar("oUIElement","Title:=INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
;controltype:=UIA_TextControlTypeId;class:=TextBlock") ;ControlType:=UIA_TextControlTypeId;classname:=TextBlock")

;~ Actions split away from logical/technical definition above can come from configfiles 

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")

;~_UIA_action($oUIElement","highlight")
;~_UIA_action($oUIElement,"click")
;~_UIA_action("oUIElement","highlight")
;~_UIA_action("oUIElement","click")


;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP2=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=TC2000 ® Version 17.0.6435.29524;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children)   
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_CustomControlTypeId;class:=WorkspaceTabControl", $treescope_children)    
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_CustomControlTypeId;class:=WebChart", $treescope_children)   
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll("INNT
InnovateBiopharmaceuticals,Inc
HealthcareSector(D)
Biotechnology
.mainwindow", "title:=INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
;ControlType:=UIA_TextControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
;ControlType:=UIA_TextControlTypeId", $treescope_subtree)
;~_UIA_action($oUIElement,"highlight")
_UIA_action($oUIElement,"click")


*** Detailed properties of the highlighted element ***
UIA_title:= <INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
>
UIA_text:= <INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
>
UIA_regexptitle:= <INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
>
UIA_class:= <TextBlock>
UIA_regexpclass:= <TextBlock>
UIA_iaccessiblechildId:= <0>
UIA_id:= <Watermark>
UIA_handle:= <0>
UIA_RuntimeId:= <7;1244;18409771>
UIA_BoundingRectangle:= <989;356;406;183>
UIA_ProcessId:= <1244>
UIA_ControlType:= <50020>
UIA_LocalizedControlType:= <text>
UIA_Name:= <INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <False>
UIA_IsEnabled:= <True>
UIA_AutomationId:= <Watermark>
UIA_ClassName:= <TextBlock>
UIA_ClickablePoint:= <1192;447>
 

 

Edited by toto22

Share this post


Link to post
Share on other sites

I can do it in C#. However I cant figure it out in Autoit. Please help

C# code

FlaUI.Core.Application app = FlaUI.Core.Application.Attach(System.Diagnostics.Process.GetProcessesByName("TMain")[0].Id);
            using (var automation = new UIA3Automation())
            {
                var window = app.GetMainWindow(automation);
                Console.WriteLine(window.Title);
                
                Price = "";
                do
                {
                    try
                    {
                        var child = window.FindFirstChild(cf => cf.ByAutomationId("WorkspaceManager"));
                        var child2 = child.FindFirstChild(cf => cf.ByClassName("WebChart"));
                        var child3 = child2.FindFirstChild(cf => cf.ByAutomationId("Watermark"));

                        //var pattern1 = child3.Patterns.Value.Pattern;

                        var pattern1 = child3.Properties.Name.Value;
                        Symbol2 = child3.Properties.Name.Value;

                        int index = Symbol2.IndexOf(" ");
                        if (index >= 0)
                            Symbol2 = Symbol2.Substring(0, index);
                    }
                    catch (Exception)
                    {

                    }
                } while ( Price == "g");
                //string Symbol2 = Symbol.ToString();
                textBox1.Text = Symbol2;

 

Share this post


Link to post
Share on other sites

so, the code that junkew's spy tool returns doesn't work? it's this part

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")

 


My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites

@junkew is your person for this one but I think it has to do with

UIA_IsTextPatternAvailable:= <False>

 

if you can make a C# dll that can do it, you can call it from AutoIt, no?

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites

well, to be fair, this is the IUIAutomation wrapper written by @junkew you are using, no? He will probably be around and see this. He's who I would ask.

also, what you use in C# is none other than a wrapper. which is ok.

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites

let me axe you this... do you have a function like this you could use in those wrappers?

Func _UIA_ControlGetText($hWnd, $controlID)


My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites

i'm not sure,

this where im at -.- 

;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

local $oP1 = _UIA_setVar("oP1","Title:=TC2000 ® Version 17.0.6435.29524;controltype:=UIA_WindowControlTypeId;class:=Window")   ;TC2000 ® Version 17.0.6435.29524
local $oP2 = _UIA_setVar("oP2","Title:=;controltype:=UIA_CustomControlTypeId;class:=WorkspaceTabControl")   ;
local $oP3 = _UIA_setVar("oP3","Title:=;controltype:=UIA_CustomControlTypeId;class:=WebChart")
Local $oUIElement=_UIA_getObjectByFindAll($oP3,"controltype:=UIA_TextControlTypeId;class:=TextBlock") ;ControlType:=UIA_TextControlTypeId;classname:=TextBlock")

MsgBox(1,"",$oUIElement)

 

Edited by toto22

Share this post


Link to post
Share on other sites

hmmmm, can you try modifying that last one

Local $oUIElement=_UIA_getObjectByFindAll($oP3,"Title:=;controltype:=UIA_TextControlTypeId;class:=TextBlock")
Edited by Earthshine

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

local $oP1 = _UIA_setVar("oP1","Title:=TC2000 ® Version 17.0.6435.29524;controltype:=UIA_WindowControlTypeId;class:=Window")   ;TC2000 ® Version 17.0.6435.29524
local $oP2 = _UIA_setVar("oP2","Title:=;controltype:=UIA_CustomControlTypeId;class:=WorkspaceTabControl")   ;
local $oP3 = _UIA_setVar("oP3","Title:=;controltype:=UIA_CustomControlTypeId;class:=WebChart")
Local $oUIElement=_UIA_getObjectByFindAll($oP3,"Title:=;controltype:=UIA_TextControlTypeId;class:=TextBlock")

MsgBox(1,"",$oUIElement)

try it


My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites

Thank you so much for your help. However, It still returns blank message box. Ill keep playing with it and if I make any progress, ill let you know. Thank you so much for your time and help

Share this post


Link to post
Share on other sites

can you do this?

$oUIElement=_UIA_getObjectByFindAll($oP0, "title:=*;ControlType:=UIA_TextControlTypeId", $treescope_subtree)

 


My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites

you know it does find object and highlights and clicks it. However getting text is a whole different story. there should be a command to show text or header.

Im taking a lil break now. thank you brotha

 

Share this post


Link to post
Share on other sites

I see there is a thread where you can set the text, I bet you can get the text in such a way? 

_UIA_action("204022.mainwindow","setvalue using keys","hello world")

maybe

_UIA_action("YourApp","getValue")

YES, i looked in the UIWrappers.au3, there is an 'getValue' you should be able to use!

 

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites

ok, update, so I used Microsoft Word 2016 as a sample for this code and made a test docx file, with the text in it--"This is a test." and ran the code below and it RETURNED MY TEXT!! using junkew's wrapper, the one you already use.

;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=This is a test.docx - Word;controltype:=UIA_WindowControlTypeId;class:=OpusApp", $treescope_children)
_UIA_Action($oP4,"setfocus")
Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=_WwF", $treescope_children)
_UIA_Action($oP3,"setfocus")
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=This is a test.docx;controltype:=UIA_PaneControlTypeId;class:=_WwB", $treescope_children)
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=This is a test.docx;controltype:=UIA_DocumentControlTypeId;class:=_WwG", $treescope_children)
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=Page 1;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children)
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll("Page1content.mainwindow", "title:=Page 1 content;ControlType:=UIA_EditControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP2, "title:=Page 1 content;ControlType:=UIA_EditControlTypeId", $treescope_subtree)
;~_UIA_action($oUIElement,"highlight")
Local $string = _UIA_action($oUIElement,"getValue")
MsgBox($MB_OK,"", $string)

 

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Sayed
      Hi there, 
      I'm new in AutoIt forms and using AutoIt to automate desktop application (able to automate the application normally but facing issue when I've to re-run the application twice within the same script...so need help in this please)
      here is the steps then followed by the issue in a brief : 
      1- run application . 
      2- do some actions (click menus,activate windows,set texts..)
      3- close the application. 
      4- run the application again & access the same controls.
      5- open the same windows again (like step 2)
      6- perform some validations (by getting texts from some text boxes)
      7- close the application again (and repeat 1-7 for 15 times in average )
      The issue 
      * all controls are accessible in the first run and actions done successfully on controls (for steps 1-3) BUT from the second run of the application from step-4 it's able to set focus only the main application window.
      Note: only unique properties used to while mapping the controls. 
      Error that appear in the console :
      UIAWrappers.au3" (1673) : ==> Array variable has incorrect number of subscripts or subscript dimension range exceeded.: $x = Int($t[1] + ($t[3] / 2)) $x = Int($t[1] + (^ ERROR  
      Simple spy code  of one of the controls that has this strange issue(menubar&view menu Item): 
      ;~ *** Standard code maintainable *** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) _UIA_setVar("oP1","Title:=XXX;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app") ;main app form xxx _UIA_setVar("oP2","Title:=menuStrip1;controltype:=UIA_MenuBarControlTypeId;class:=WindowsForms10.Window.8.app") ;menuStrip1 ;~ $oUIElement=_UIA_getObjectByFindAll("View.mainwindow", "title:=View;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree) _UIA_setVar("oUIElement","Title:=View;controltype:=UIA_MenuItemControlTypeId;class:=") ;ControlType:=UIA_MenuItemControlTypeId;classname:=") ;~ Actions split away from logical/technical definition above can come from configfiles ;~_UIA_Action("oP1","highlight") _UIA_Action("oP1","setfocus") ;~_UIA_Action("oP2","highlight") _UIA_Action("oP2","setfocus") _UIA_action("oUIElement","highlight") ;~_UIA_action("oUIElement","click")  
       
    • By Se7enstars
      New Modern Design UI in AutoIt in 118 line
      Please watch & like in Youtube: Link
      Full Source Code free: Link
       

    • By adityaparakh
      Hello ,
      I am trying to fetch data and control a Windows Program.
      The data isn't reflected in the Au3Info tool , hence used SimpleSpy UIA , for trying.
      The code is able to fetch only one row at a time.
      Most Strangely , the code doesn't retreive the same specified row consistently.
      If I run the same code , under same circumstance , it is still fetching different rows.
      This was when op2 and op3 were zero. I would not like to use this at all.

      Also , it takes 3 seconds for it search down the hiearchy and reach the row.
      I will need to fetch ALL the rows continuously for 6 hours every second , in a minimized state.
      Can you please help.
      Attached :
      1. AuInfo summary
      2. Code
      3. Simple Spy Summary
      The Actions I am trying to do (in a Minimzed State - not affecting usage of computer for other purposes)
      1. Fetch all Rows (number of rows will not always be constant
      2. Press the Check box for a specific row(s)
      3. Press the squareOff button on top
      Thanks a lot
       
       
      #include <MsgBoxConstants.au3> #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP8=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=MO Trader;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.3e799b_r8_ad1", $treescope_children) _UIA_Action($oP8,"setfocus") Local $oP7=_UIA_getObjectByFindAll($oP8, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3e799b_r8_ad1", $treescope_children) _UIA_Action($oP7,"setfocus") Local $oP6=_UIA_getObjectByFindAll($oP7, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3e799b_r8_ad1", $treescope_children) _UIA_Action($oP6,"setfocus") Local $oP5=_UIA_getObjectByFindAll($oP6, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3e799b_r8_ad1", $treescope_children) _UIA_Action($oP5,"setfocus") Local $oP4=_UIA_getObjectByFindAll($oP5, "Title:=Day Net Position;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3e799b_r8_ad1", $treescope_children) _UIA_Action($oP4,"setfocus") Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3e799b_r8_ad1", $treescope_children) _UIA_Action($oP3,"setfocus") Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=0.0000;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3e799b_r8_ad1", $treescope_children) _UIA_Action($oP2,"setfocus") Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=0.0000;controltype:=UIA_TableControlTypeId;class:=WindowsForms10.Window.8.app.0.3e799b_r8_ad1", $treescope_children) _UIA_Action($oP1,"setfocus") Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=Data Panel;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children) ;~ First find the object in the parent before you can do something Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Row 12;ControlType:=UIA_CustomControlTypeId", $treescope_subtree) _UIA_action($oUIElement,"click") Local $string = _UIA_action($oUIElement,"getValue") MsgBox($MB_OK,"", $string) Local $oUIElement2=_UIA_getObjectByFindAll($oP0, "title:=Row 6;ControlType:=UIA_CustomControlTypeId", $treescope_subtree) _UIA_action($oUIElement2,"click") Local $string2 = _UIA_action($oUIElement2,"getValue") MsgBox($MB_OK,"", $string2)  


      SimpleSpy Row.txt
    • By caramen
      Is that correct?
      With SimpleSpy i can click on controls without any include/udf ON CHROME?
    • By drapdv
      So, I have this odd issue happening with UIAWrappers.  When I run simplespy.au3 through SciTE, it finds the object I'm trying to work with, no problem. But, when I compile simplespy.exe, it can't find it.  Another script that utilizes UIAWrappers is experiencing the same phenomenon. EDIT: Compiling as 32-bit or 64-bit does not affect this behavior.
      The trouble starts at the very first object I try to get via _UIA_getObjectByFindAll, so I included the $properties2Match array and the object properties that were returned by _UIA_getPropertyValue for that first object both when running from SciTE and when compiled.
      It always finds the matches for the title and control type properties.  But when compiled, it fails to find the Automation ID property.  Any ideas as to why this would be happening?  Hoping junkew will see this and have an idea.
       
      These are the $properties2Match
      30005|Screen/PVN - ###.###.##.## - CDK Drive|0|False
      30003|50032|0|False
      30011|Portal|0|False
       
      Object properties when running from SciTE:
      <propertymatching>       
              j:1 propID:[30005] expValue:[Screen/PVN - ###.###.##.## - CDK Drive]actualValue:[Screen/PVN - ###.###.##.## - CDK Drive]1
              j:2 propID:[30003] expValue:[50032]actualValue:[50032]1
              j:3 propID:[30011] expValue:[Portal]actualValue:[Portal]1  
      </propertymatching>
       
      Object properties when script is compiled:
      <propertymatching>
              j:1 propID:[30005] expValue:[Screen/PVN - ###.###.##.## - CDK Drive]actualValue:[Screen/PVN - ###.###.##.## - CDK Drive]1
              j:2 propID:[30003] expValue:[50032]actualValue:[50032]1
              j:3 propID:[30011] expValue:[Portal]actualValue:[]0  
      </propertymatching>
       
×
×
  • Create New...