Jump to content

SOLVED: Get Text from UI


Recommended Posts

Please help me to get text from another program using UI

I need my program to extract symbol and its description from TC2000, basically a "element title". I want it to return "INNT Innovate Biopharmaceuticals, Inc Healthcare Sector(D) Biotechnology" (or any other text if symbol changes)

 

my wrapper is:

Mouse position is retrieved 1194-405
At least we have an element title: [INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
] class: [TextBlock]

Having the following values for all properties: 
Title is: <INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
>   Class   := <TextBlock>  controltype:= <UIA_TextControlTypeId>   ,<50020>    , (0000C364)    989;356;406;183
*** Parent Information top down ***
2: Title is: <TC2000 ® Version 17.0.6435.29524>    Class   := <Window> controltype:= <UIA_WindowControlTypeId> ,<50032>    , (0000C370)    51;24;1712;1124
"Title:=TC2000 ® Version 17.0.6435.29524;controltype:=UIA_WindowControlTypeId;class:=Window""  
1: Title is: <> Class   := <WorkspaceTabControl>    controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    59;123;1696;1017
"Title:=;controltype:=UIA_CustomControlTypeId;class:=WorkspaceTabControl""  
0: Title is: <> Class   := <WebChart>   controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    675;125;1080;612
"Title:=;controltype:=UIA_CustomControlTypeId;class:=WebChart"" 


;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:=TC2000 ® Version 17.0.6435.29524;controltype:=UIA_WindowControlTypeId;class:=Window")    ;TC2000 ® Version 17.0.6435.29524
_UIA_setVar("oP2","Title:=;controltype:=UIA_CustomControlTypeId;class:=WorkspaceTabControl")    ;
_UIA_setVar("oP3","Title:=;controltype:=UIA_CustomControlTypeId;class:=WebChart")   ;

;~ $oUIElement=_UIA_getObjectByFindAll("INNT
InnovateBiopharmaceuticals,Inc
HealthcareSector(D)
Biotechnology
.mainwindow", "title:=INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
;ControlType:=UIA_TextControlTypeId", $treescope_subtree)
_UIA_setVar("oUIElement","Title:=INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
;controltype:=UIA_TextControlTypeId;class:=TextBlock") ;ControlType:=UIA_TextControlTypeId;classname:=TextBlock")

;~ Actions split away from logical/technical definition above can come from configfiles 

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")

;~_UIA_action($oUIElement","highlight")
;~_UIA_action($oUIElement,"click")
;~_UIA_action("oUIElement","highlight")
;~_UIA_action("oUIElement","click")


;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP2=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=TC2000 ® Version 17.0.6435.29524;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children)   
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_CustomControlTypeId;class:=WorkspaceTabControl", $treescope_children)    
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_CustomControlTypeId;class:=WebChart", $treescope_children)   
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll("INNT
InnovateBiopharmaceuticals,Inc
HealthcareSector(D)
Biotechnology
.mainwindow", "title:=INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
;ControlType:=UIA_TextControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
;ControlType:=UIA_TextControlTypeId", $treescope_subtree)
;~_UIA_action($oUIElement,"highlight")
_UIA_action($oUIElement,"click")


*** Detailed properties of the highlighted element ***
UIA_title:= <INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
>
UIA_text:= <INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
>
UIA_regexptitle:= <INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
>
UIA_class:= <TextBlock>
UIA_regexpclass:= <TextBlock>
UIA_iaccessiblechildId:= <0>
UIA_id:= <Watermark>
UIA_handle:= <0>
UIA_RuntimeId:= <7;1244;18409771>
UIA_BoundingRectangle:= <989;356;406;183>
UIA_ProcessId:= <1244>
UIA_ControlType:= <50020>
UIA_LocalizedControlType:= <text>
UIA_Name:= <INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <False>
UIA_IsEnabled:= <True>
UIA_AutomationId:= <Watermark>
UIA_ClassName:= <TextBlock>
UIA_ClickablePoint:= <1192;447>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <0>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_FrameworkId:= <WPF>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <False>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <False>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <False>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <False>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <3>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleName:= <INNT
 Innovate Biopharmaceuticals, Inc
Healthcare Sector(D)
Biotechnology
>
UIA_LegacyIAccessibleRole:= <41>
UIA_LegacyIAccessibleState:= <0>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:1244,providerId:0x0 Main(parent link):Unidentified Provider (managed:MS.Internal.Automation.ElementProxy, PresentationCore, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <True>

 

Edited by toto22
Link to comment
Share on other sites

I can do it in C#. However I cant figure it out in Autoit. Please help

C# code

FlaUI.Core.Application app = FlaUI.Core.Application.Attach(System.Diagnostics.Process.GetProcessesByName("TMain")[0].Id);
            using (var automation = new UIA3Automation())
            {
                var window = app.GetMainWindow(automation);
                Console.WriteLine(window.Title);
                
                Price = "";
                do
                {
                    try
                    {
                        var child = window.FindFirstChild(cf => cf.ByAutomationId("WorkspaceManager"));
                        var child2 = child.FindFirstChild(cf => cf.ByClassName("WebChart"));
                        var child3 = child2.FindFirstChild(cf => cf.ByAutomationId("Watermark"));

                        //var pattern1 = child3.Patterns.Value.Pattern;

                        var pattern1 = child3.Properties.Name.Value;
                        Symbol2 = child3.Properties.Name.Value;

                        int index = Symbol2.IndexOf(" ");
                        if (index >= 0)
                            Symbol2 = Symbol2.Substring(0, index);
                    }
                    catch (Exception)
                    {

                    }
                } while ( Price == "g");
                //string Symbol2 = Symbol.ToString();
                textBox1.Text = Symbol2;

 

Link to comment
Share on other sites

so, the code that junkew's spy tool returns doesn't work? it's this part

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")

 

My resources are limited. You must ask the right questions

 

Link to comment
Share on other sites

@junkew is your person for this one but I think it has to do with

UIA_IsTextPatternAvailable:= <False>

 

if you can make a C# dll that can do it, you can call it from AutoIt, no?

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Link to comment
Share on other sites

well, to be fair, this is the IUIAutomation wrapper written by @junkew you are using, no? He will probably be around and see this. He's who I would ask.

also, what you use in C# is none other than a wrapper. which is ok.

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Link to comment
Share on other sites

i'm not sure,

this where im at -.- 

;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

local $oP1 = _UIA_setVar("oP1","Title:=TC2000 ® Version 17.0.6435.29524;controltype:=UIA_WindowControlTypeId;class:=Window")   ;TC2000 ® Version 17.0.6435.29524
local $oP2 = _UIA_setVar("oP2","Title:=;controltype:=UIA_CustomControlTypeId;class:=WorkspaceTabControl")   ;
local $oP3 = _UIA_setVar("oP3","Title:=;controltype:=UIA_CustomControlTypeId;class:=WebChart")
Local $oUIElement=_UIA_getObjectByFindAll($oP3,"controltype:=UIA_TextControlTypeId;class:=TextBlock") ;ControlType:=UIA_TextControlTypeId;classname:=TextBlock")

MsgBox(1,"",$oUIElement)

 

Edited by toto22
Link to comment
Share on other sites

;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

local $oP1 = _UIA_setVar("oP1","Title:=TC2000 ® Version 17.0.6435.29524;controltype:=UIA_WindowControlTypeId;class:=Window")   ;TC2000 ® Version 17.0.6435.29524
local $oP2 = _UIA_setVar("oP2","Title:=;controltype:=UIA_CustomControlTypeId;class:=WorkspaceTabControl")   ;
local $oP3 = _UIA_setVar("oP3","Title:=;controltype:=UIA_CustomControlTypeId;class:=WebChart")
Local $oUIElement=_UIA_getObjectByFindAll($oP3,"Title:=;controltype:=UIA_TextControlTypeId;class:=TextBlock")

MsgBox(1,"",$oUIElement)

try it

My resources are limited. You must ask the right questions

 

Link to comment
Share on other sites

I see there is a thread where you can set the text, I bet you can get the text in such a way? 

_UIA_action("204022.mainwindow","setvalue using keys","hello world")

maybe

_UIA_action("YourApp","getValue")

YES, i looked in the UIWrappers.au3, there is an 'getValue' you should be able to use!

 

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Link to comment
Share on other sites

ok, update, so I used Microsoft Word 2016 as a sample for this code and made a test docx file, with the text in it--"This is a test." and ran the code below and it RETURNED MY TEXT!! using junkew's wrapper, the one you already use.

;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=This is a test.docx - Word;controltype:=UIA_WindowControlTypeId;class:=OpusApp", $treescope_children)
_UIA_Action($oP4,"setfocus")
Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=_WwF", $treescope_children)
_UIA_Action($oP3,"setfocus")
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=This is a test.docx;controltype:=UIA_PaneControlTypeId;class:=_WwB", $treescope_children)
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=This is a test.docx;controltype:=UIA_DocumentControlTypeId;class:=_WwG", $treescope_children)
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=Page 1;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children)
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll("Page1content.mainwindow", "title:=Page 1 content;ControlType:=UIA_EditControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP2, "title:=Page 1 content;ControlType:=UIA_EditControlTypeId", $treescope_subtree)
;~_UIA_action($oUIElement,"highlight")
Local $string = _UIA_action($oUIElement,"getValue")
MsgBox($MB_OK,"", $string)

 

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...