Sign in to follow this  
Followers 0
souldjer777

Control Application WindowsForms10 Internet Explorer_Server .HTM

7 posts in this topic

#1 ·  Posted (edited)

Good Morning Guru's,

I have an exe that calls a .dll which contains several .htm files.  It's bascially an htm inside of an exe frame.  If I inspect the application with the "AutoIt v3 Window Info" utility I see...

Outside EXE...

>>>> Window <<<<
Title:    User Client
Class:    WindowsForms10.Window.8.app.0.378734a

(Everything Else is N/A)

Inside the frame of the EXE...

>>>> Window <<<<
Title:    User Client
Class:    WindowsForms10.Window.8.app.0.378734a

>>>> Control <<<<
Class:    Internet Explorer_Server
Instance:    1
ClassnameNN:    Internet Explorer_Server1
Name:    
Advanced (Class):    [CLASS:Internet Explorer_Server; INSTANCE:1]

I HAD to use another application called "AccViewer.exe" or  Accessible Info Viewer to see more detail...

Class(NN): Internet Explorer_Server1

Example of some nested htm files within the "User Client.exe"...

res://CliHTMLRes.dll/Console.htm

res://CliHTMLRes.dll/Home.htm

res://CliHTMLRes.dll/Navigation.htm

Win Title:(Blank - not available)
Text:(Blank - not available)

Class(NN): Internet Explorer_Server1

Name: res://CliHTMLRes.dll/Console.htm

Description: MSAAHTML Registered Handler

I don't know how to control the application with the "embedded" htm files.  I don't want to resort to throwing key strokes to the application - that's a shot in the dark.

I've had no luck with trying to access the embedded _IEAttach - Getting 0 and 7 errors

#include <IE.au3>
#include <Array.au3>

ShellExecute ( "C:\Program Files\User Client.exe" )
MsgBox (0, "Executed", "")
WinWait ("User Client")
MsgBox (0, "Waiting", "")
WinActivate ("User Client")
MsgBox (0, "Activated", "")
$hWnd = WinGetHandle("User Client") ; got this from window info
MsgBox (0, "Info", $hWnd)
$oIE = _IEAttach ($hWnd, "embedded",1)
MsgBox (0, "IEAttach", $oIE & " " & @ERROR )
$oLinks = _IEBodyReadText($oIE)
MsgBox (0, "Read", $oLinks)

I'm still looking into it... but losing patience so I thought I'd reach out :)  Thanks Everyone!

:ILA:

PS... have already tried

$oIE = _IEAttach("User Client", "embedded")

$oIE = _IEAttach ($hWnd, "embedded",1)

$oIE = _IEAttach('[CLASS:Internet Explorer_Server; INSTANCE:1]','embedded')

Edited by souldjer777

"Maybe I'm on a road that ain't been paved yet. And maybe I see a sign that ain't been made yet"
Song Title: I guess you could say
Artist: Middle Class Rut

Share this post


Link to post
Share on other sites



See examples section on iuiautomation and check what simple spy sees

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

Alright, I downloaded your Example.zip and UIA_V0_4.zip from...

 

'?do=embed' frameborder='0' data-embedContent>>

 

Put the UIA files in the "Includes" directory and ran the simplespy.au3

 

Got a very long list and it's detecting the controls :sorcerer:

 

Not sure if I should post my results here... results are insane :idiot:

Mouse position is retrieved 1276-488
At least we have an element [About][]
Having the following values for all properties: 
Title is: <About>   Class   := <>   controltype:= <UIA_EditControlTypeId>   ,<50004>    , (0000C354)    
*** Parent Information ***
Title is: <About>   Class   := <>   controltype:= <UIA_HyperlinkControlTypeId>  ,<50005>    , (0000C355)    
*** Detailed properties of the highlighted element ***
UIA_AcceleratorKeyPropertyId :=
UIA_AccessKeyPropertyId :=Alt+B
UIA_AriaPropertiesPropertyId :=
UIA_AriaRolePropertyId :=
UIA_AutomationIdPropertyId :=
UIA_BoundingRectanglePropertyId :=1260;479;32;15
UIA_ClassNamePropertyId :=
UIA_ClickablePointPropertyId :=
UIA_ControllerForPropertyId :=
UIA_ControlTypePropertyId :=50004
UIA_CulturePropertyId :=0
UIA_DescribedByPropertyId :=
UIA_DockDockPositionPropertyId :=5
UIA_ExpandCollapseExpandCollapseStatePropertyId :=3
UIA_FlowsToPropertyId :=
UIA_FrameworkIdPropertyId :=
UIA_GridColumnCountPropertyId :=0
UIA_GridItemColumnPropertyId :=0
UIA_GridItemColumnSpanPropertyId :=1
UIA_GridItemContainingGridPropertyId :=
UIA_GridItemRowPropertyId :=0
UIA_GridItemRowSpanPropertyId :=1
UIA_GridRowCountPropertyId :=0
UIA_HasKeyboardFocusPropertyId :=False
UIA_HelpTextPropertyId :=
UIA_IsContentElementPropertyId :=True
UIA_IsControlElementPropertyId :=True
UIA_IsDataValidForFormPropertyId :=False
UIA_IsDockPatternAvailablePropertyId :=False
UIA_IsEnabledPropertyId :=True
UIA_IsExpandCollapsePatternAvailablePropertyId :=False
UIA_IsGridItemPatternAvailablePropertyId :=False
UIA_IsGridPatternAvailablePropertyId :=False
UIA_IsInvokePatternAvailablePropertyId :=True
UIA_IsItemContainerPatternAvailablePropertyId :=False
UIA_IsKeyboardFocusablePropertyId :=False
UIA_IsLegacyIAccessiblePatternAvailablePropertyId :=True
UIA_IsMultipleViewPatternAvailablePropertyId :=False
UIA_IsOffscreenPropertyId :=False
UIA_IsPasswordPropertyId :=False
UIA_IsRangeValuePatternAvailablePropertyId :=False
UIA_IsRequiredForFormPropertyId :=False
UIA_IsScrollItemPatternAvailablePropertyId :=False
UIA_IsScrollPatternAvailablePropertyId :=False
UIA_IsSelectionItemPatternAvailablePropertyId :=False
UIA_IsSelectionPatternAvailablePropertyId :=False
UIA_IsSynchronizedInputPatternAvailablePropertyId :=False
UIA_IsTableItemPatternAvailablePropertyId :=False
UIA_IsTablePatternAvailablePropertyId :=False
UIA_IsTextPatternAvailablePropertyId :=False
UIA_IsTogglePatternAvailablePropertyId :=False
UIA_IsTransformPatternAvailablePropertyId :=False
UIA_IsValuePatternAvailablePropertyId :=True
UIA_IsVirtualizedItemPatternAvailablePropertyId :=False
UIA_IsWindowPatternAvailablePropertyId :=False
UIA_ItemStatusPropertyId :=
UIA_ItemTypePropertyId :=
UIA_LabeledByPropertyId :=
UIA_LegacyIAccessibleChildIdPropertyId :=1
UIA_LegacyIAccessibleDefaultActionPropertyId :=Jump
UIA_LegacyIAccessibleDescriptionPropertyId :=
UIA_LegacyIAccessibleHelpPropertyId :=
UIA_LegacyIAccessibleKeyboardShortcutPropertyId :=Alt+B
UIA_LegacyIAccessibleNamePropertyId :=About
UIA_LegacyIAccessibleRolePropertyId :=42
UIA_LegacyIAccessibleSelectionPropertyId :=
UIA_LegacyIAccessibleStatePropertyId :=6291520
UIA_LegacyIAccessibleValuePropertyId :=res://CliHTMLRes.dll/
UIA_LocalizedControlTypePropertyId :=edit
UIA_MultipleViewCurrentViewPropertyId :=0
UIA_MultipleViewSupportedViewsPropertyId :=
UIA_NamePropertyId :=About
UIA_NativeWindowHandlePropertyId :=0
UIA_OrientationPropertyId :=0
UIA_ProcessIdPropertyId :=2176
UIA_ProviderDescriptionPropertyId :=[pid:2176,hwnd:0x0 Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]
UIA_RangeValueIsReadOnlyPropertyId :=True
UIA_RangeValueLargeChangePropertyId :=0
UIA_RangeValueMaximumPropertyId :=0
UIA_RangeValueMinimumPropertyId :=0
UIA_RangeValueSmallChangePropertyId :=0
UIA_RangeValueValuePropertyId :=0
UIA_RuntimeIdPropertyId :=
UIA_ScrollHorizontallyScrollablePropertyId :=False
UIA_ScrollHorizontalScrollPercentPropertyId :=0
UIA_ScrollHorizontalViewSizePropertyId :=100
UIA_ScrollVerticallyScrollablePropertyId :=False
UIA_ScrollVerticalScrollPercentPropertyId :=0
UIA_ScrollVerticalViewSizePropertyId :=100
UIA_SelectionCanSelectMultiplePropertyId :=False
UIA_SelectionIsSelectionRequiredPropertyId :=False
UIA_SelectionselectionPropertyId :=
UIA_SelectionItemIsSelectedPropertyId :=False
UIA_SelectionItemSelectionContainerPropertyId :=
UIA_TableColumnHeadersPropertyId :=
UIA_TableItemColumnHeaderItemsPropertyId :=
UIA_TableRowHeadersPropertyId :=
UIA_TableRowOrColumnMajorPropertyId :=2
UIA_TableItemRowHeaderItemsPropertyId :=
UIA_ToggleToggleStatePropertyId :=2
UIA_TransformCanMovePropertyId :=False
UIA_TransformCanResizePropertyId :=False
UIA_TransformCanRotatePropertyId :=False
UIA_ValueIsReadOnlyPropertyId :=True
UIA_ValueValuePropertyId :=res://CliHTMLRes.dll/
UIA_WindowCanMaximizePropertyId :=False
UIA_WindowCanMinimizePropertyId :=False
UIA_WindowIsModalPropertyId :=False
UIA_WindowIsTopmostPropertyId :=False
UIA_WindowWindowInteractionStatePropertyId :=0
UIA_WindowWindowVisualStatePropertyId :=0
I'll look over your examples now and see what I can do! Edited by souldjer777

"Maybe I'm on a road that ain't been paved yet. And maybe I see a sign that ain't been made yet"
Song Title: I guess you could say
Artist: Middle Class Rut

Share this post


Link to post
Share on other sites

#4 ·  Posted (edited)

Okay... so I have a slight problem... I have an element in the gui that changes... in particular - it's the percent of the disk that's encrypted.  Is there a wild card I can use to just test against the "Encrypting " portion ?  As you can see it's read as "Encrypting ( 2%)"

Mouse position is retrieved 735-285
At least we have an element [Encrypting ( 2%) ][]
Having the following values for all properties:
Title is: <Encrypting ( 2%) >    Class   := <>    controltype:= <UIA_EditControlTypeId>    ,<50004>    , (0000C354)  

; Condition to find text "Encrypting " UIA_EditControlTypeId
  Local $pCondition, $pCondition1, $pCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_EditControlTypeId, $pCondition1 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Encrypting ", $pCondition2 )
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "Encrypting condition ERR" & @CRLF )
  ConsoleWrite( "Encrypting condition OK" & @CRLF )

Thanks in advance!

Edited by souldjer777

"Maybe I'm on a road that ain't been paved yet. And maybe I see a sign that ain't been made yet"
Song Title: I guess you could say
Artist: Middle Class Rut

Share this post


Link to post
Share on other sites

I don't think you can use wildcards. But you can find all edit elements, and search for the proper one in a loop:

 

; Condition to find all edit elements
Local $pCondition
$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_EditControlTypeId, $pCondition )
If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
ConsoleWrite( "Property condition OK" & @CRLF )

; Find all edit elements
Local $pUIElementArray, $oUIElementArray, $iElements
$oWindow.FindAll( $TreeScope_Descendants, $pCondition, $pUIElementArray ) ; <<<< Use your own $oWindow element <<<<<<<<<<<
$oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
$oUIElementArray.Length( $iElements )
If Not $iElements Then Return ConsoleWrite( "Find edit elements ERR" & @CRLF )
ConsoleWrite( "Find all edit elements OK. Elements: " & $iElements & @CRLF )

; Find proper edit element
Local $pEdit, $oEdit, $sName, $oProperEdit
For $i = 0 To $iElements - 1
  $oUIElementArray.GetElement( $i, $pEdit )
  $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  $oEdit.GetCurrentPropertyValue( $UIA_NamePropertyId, $sName )
  ConsoleWrite( "Name of edit element " & $i & ": " & $sName & @CRLF )
  If StringInStr( $sName, "Encrypting" ) Then
    ConsoleWrite( "This is the proper edit element" & @CRLF )
    $oProperEdit = $oEdit
    ExitLoop
  EndIf
Next

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

Thank you!  I tried it out and it's finding the element even though it's not anywhere on the screen - I'm sure it's something I did... I'm looking into it...  Thanks again!  :)

Update: I know why now - it's detecting hidden elements not even on the screen - elements that are not visible.  How do I exclude those please! 

Edited by souldjer777

"Maybe I'm on a road that ain't been paved yet. And maybe I see a sign that ain't been made yet"
Song Title: I guess you could say
Artist: Middle Class Rut

Share this post


Link to post
Share on other sites

You have to use UIA_IsOffscreenPropertyId :=False

with the wrappers you get something like this

#include "UIAWrappers.au3"

const cmyWindow:="name:=User Client"
const cmyEdit:="name:=encrypting; isoffscreen:=true"

$oWindow=_UIA_getFirstObjectOfElement($UIA_oDesktop, $cMyWindow, $treescope_children)
$oMyEdit=_UIA_getFirstObjectOfElement($oWindow, $cMyEdit, $treescope_children)

if you need more properties to be abstracted extend this array in UIAWrappers

;~ Propertynames to match to numeric values
local $UIA_propertiesSupportedArray[11][3]=[ _
    ["name",$UIA_NamePropertyId], _
    ["title",$UIA_NamePropertyId], _
    ["automationid",$UIA_AutomationIdPropertyId], _
    ["classname", $UIA_ClassNamePropertyId], _
    ["class", $UIA_ClassNamePropertyId], _
    ["iaccessiblevalue",$UIA_LegacyIAccessibleValuePropertyId], _
    ["iaccessiblechildId", $UIA_LegacyIAccessibleChildIdPropertyId], _
    ["controltype", $UIA_ControlTypePropertyId,1], _
    ["processid", $UIA_ProcessIdPropertyId], _
    ["acceleratorkey", $UIA_AcceleratorKeyPropertyId], _
    ["isoffscreen",$UIA_IsOffscreenPropertyId]  _
]

and in cuitautomation2.au3 you can find all property constants 

just below line

;module UIA_PropertyIds

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0