Beginner Question: I'm new to autoit but have a good experience in python with pytest framework. is autoIt with pytest possible?

[autotest410]

The automation for the entire GUI application was done using pywinauto with pytest. The GUI app that we automated is developed by one of the vendors company. Inhouse team started extending the functionality of the GUI. The flow goes this way:

1. Click on an option from drop down menu (this is on the GUI app provided by vendors and I can do in pytest)

2. cmd prmpt opens up and runs bat file in my C drive (No need to automate this).

3. A window opened up and I came to know that it is a javaw app after inspecting it. I would like to automate this window whose ClassName is SunAWTFrame and ControlType is UIA_WindowControlTypeId.

I tried navigating through the available options but I couldn't find the locators. I found AutoIt is a promising solution to automate.

My approach for this on a high level would be to 

1. Launch the extended java windows app from pytest framework

2. Run the .au3 which create and read the application object, perform the steps as per the test cases, and save the output on the application to a json file. Validate the json file with the expected results in pytest framework.

Below are the questions going on in my mind at the moment:

1. Is my approach feasible?

2. How to access the locators on the extended java application? I saw that it is possible by JAB so I installed it but I couldn't find clear steps to proceed further. It would be really great if you could help me with an appropriate link.

3. I'm not sure about code to create an application object.

Code level approach:

def test_extended_java_app(self, expected_data_extraction):

    ClientAuth().authorized_launch() # This will launch and login to the core application
    Extendedapp.open_from_drop_down() # This will launch the extended java app
    autoit.Run('extended_java_app_steps_set_1.au3')
    f = open('data.json')
    actual_json = json.load(f)

    # Assertions
    assert expected_data_extraction == actual_json

extended_java_app_steps_set_1.au3 should contain the code to perform below actions:

1. Read the application object (Not sure of the implementation. Help or any reference is needed as I'm not launching the application from autoIt.)

2. Enter input data from the first row of an excel and click on search > export the resulted output to a json file or excel file

I'm open to take feedback on my approach so please feel free to propose different ways of tackling this problem. Kindly help me out with the bolded items and so the needful.

 

[autotest410]

The automation for the entire GUI application was done using pywinauto with pytest. The GUI app that we automated is developed by one of the vendors company. Inhouse team started extending the functionality of the GUI. The flow goes this way:

1. Click on an option from drop down menu (this is on the GUI app provided by vendors and I can do in pytest)

2. cmd prmpt opens up and runs bat file in my C drive (No need to automate this).

3. A window opened up and I came to know that it is a javaw app after inspecting it. I would like to automate this window whose ClassName is SunAWTFrame and ControlType is UIA_WindowControlTypeId.

I tried navigating through the available options but I couldn't find the locators. I found AutoIt is a promising solution to automate.

My approach for this on a high level would be to 

1. Launch the extended java windows app from pytest framework

2. Run the .au3 which create and read the application object, perform the steps as per the test cases, and save the output on the application to a json file. Validate the json file with the expected results in pytest framework.

Below are the questions going on in my mind at the moment:

1. Is my approach feasible?

2. How to access the locators on the extended java application? I saw that it is possible by JAB so I installed it but JavaFerret was crashing. Using Access Bridge Explorer and seeing the below properties and I'm not sure about the property to use. Can I use Bounds property, if so how to use it?

3. I'm not sure about code to create an application object.

Code level approach:

def test_extended_java_app(self, expected_data_extraction):

    ClientAuth().authorized_launch() # This will launch and login to the core application
    Extendedapp.open_from_drop_down() # This will launch the extended java app
    autoit.Run('extended_java_app_steps_set_1.au3')
    f = open('data.json')
    actual_json = json.load(f)

    # Assertions
    assert expected_data_extraction == actual_json

extended_java_app_steps_set_1.au3 should contain the code to perform below actions:

1. Read the application object (Not sure of the implementation. Help or any reference is needed as I'm not launching the application from autoIt.)

2. Enter input data from the first row of an excel and click on search > export the resulted output to a json file or excel file

I'm open to take feedback on my approach so please feel free to propose different ways of tackling this problem. Kindly help me out with the bolded items and so the needful.

[Jos]

Please stick to one thread with your question!

[autotest410]

@JosI thought I posted in a wrong forum and I didn't see option to delete a post so ended up in creating another post. My sincere apologizes about it.

Edited October 21, 2022 by autotest410

[BigDaddyO]

is it possible?  probably, but you won't know for sure until you download and install what may be required.  If this is a Java app, then it will most likely require some extended UDF's like UIAutomation but it should be possible.

Install AutoIt and the Scite Editor

The UIAutomation framework: 

Ensure you download Microsoft Inspect  and simplespy which have links in the below page

 

 

"I prefer Lars version of IUIAutomation as I have more control, but as you use pytest I'd guess you would do better with the one above"

 

Once you have all that installed, use the 3 spy's to find the controls on the app to automate.

Easiest to use with AutoIt built in controls would be "Au3Info.exe" which is part of the standard AutoIt install.

 

If that doesn't show anything, then you will need to try the UIAutomation spy's

Launch the inspect.exe to see what that shows when hovering over the app controls to automate.  if it identifies them, then you will need to use the IUIAutomation UDF's

[autotest410]

@BigDaddyOThank you very much for replying me back.

AutoIt v3 Window Info and Inspect.exe didn't display anything. 

AutoIt:

The Summary from AutoIt is as follows:

>>>> Window <<<<
Title:    Application Window's Title
Class:    SunAwtFrame
Position:    386, 131
Size:    900, 705
Style:    0x16CA0000
ExStyle:    0x00000100
Handle:    0x000A0C70

>>>> Control <<<<
Class:    
Instance:    
ClassnameNN:    
Name:    
Advanced (Class):    
ID:    
Text:    
Position:    
Size:    
ControlClick Coords:    
Style:    
ExStyle:    
Handle:    

>>>> Mouse <<<<
Position:    894, 364
Cursor ID:    0
Color:    0xFFFFFF

>>>> StatusBar <<<<

>>>> ToolsBar <<<<

>>>> Visible Text <<<<

>>>> Hidden Text <<<<

Inspect.exe

How found:    Mouse move (1150,473)
    hwnd=0x000A0C70 32bit class="SunAwtFrame" style=0x16CA0000 ex=0x100
Name:    "Application's Window Title"
ControlType:    UIA_WindowControlTypeId (0xC370)
LocalizedControlType:    "window"
BoundingRectangle:    {l:386 t:131 r:1286 b:836}
IsEnabled:    true
IsOffscreen:    false
IsKeyboardFocusable:    true
HasKeyboardFocus:    true
AccessKey:    ""
ProcessId:    20120
RuntimeId:    [2A.A0C70]
FrameworkId:    "Win32"
ClassName:    "SunAwtFrame"
NativeWindowHandle:    0xA0C70
ProviderDescription:    "[pid:26160,providerId:0xA0C70 Main:Nested [pid:20120,providerId:0xA0C70 Annotation(parent link):Microsoft: Annotation Proxy (unmanaged:UIAutomationCore.dll); Main:Microsoft: MSAA Proxy (unmanaged:UIAutomationCore.dll)]; Nonclient:Microsoft: Non-Client Proxy (unmanaged:uiautomationcore.dll); Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]"
IsPassword:    false
HelpText:    ""
IsDialog:    false
LegacyIAccessible.ChildId:    0
LegacyIAccessible.DefaultAction:    ""
LegacyIAccessible.Description:    ""
LegacyIAccessible.Help:    ""
LegacyIAccessible.KeyboardShortcut:    ""
LegacyIAccessible.Name:    "Application's window title"
LegacyIAccessible.Role:    client (0xA)
LegacyIAccessible.State:    focused,focusable (0x100004)
LegacyIAccessible.Value:    ""
Transform.CanMove:    true
Transform.CanResize:    false
Transform.CanRotate:    false
Window.CanMaximize:    false
Window.CanMinimize:    true
Window.IsModal:    false
Window.IsTopmost:    false
Window.WindowInteractionState:    ReadyForUserInteraction (2)
Window.WindowVisualState:    Normal (0)
IsAnnotationPatternAvailable:    false
IsDragPatternAvailable:    false
IsDockPatternAvailable:    false
IsDropTargetPatternAvailable:    false
IsExpandCollapsePatternAvailable:    false
IsGridItemPatternAvailable:    false
IsGridPatternAvailable:    false
IsInvokePatternAvailable:    false
IsItemContainerPatternAvailable:    false
IsLegacyIAccessiblePatternAvailable:    true
IsMultipleViewPatternAvailable:    false
IsObjectModelPatternAvailable:    false
IsRangeValuePatternAvailable:    false
IsScrollItemPatternAvailable:    false
IsScrollPatternAvailable:    false
IsSelectionItemPatternAvailable:    false
IsSelectionPatternAvailable:    false
IsSpreadsheetItemPatternAvailable:    false
IsSpreadsheetPatternAvailable:    false
IsStylesPatternAvailable:    false
IsSynchronizedInputPatternAvailable:    false
IsTableItemPatternAvailable:    false
IsTablePatternAvailable:    false
IsTextChildPatternAvailable:    false
IsTextEditPatternAvailable:    false
IsTextPatternAvailable:    false
IsTextPattern2Available:    false
IsTogglePatternAvailable:    false
IsTransformPatternAvailable:    true
IsTransform2PatternAvailable:    false
IsValuePatternAvailable:    false
IsVirtualizedItemPatternAvailable:    false
IsWindowPatternAvailable:    true
IsCustomNavigationPatternAvailable:    false
IsSelectionPattern2Available:    false
FirstChild:    (null) title bar
LastChild:    (null) title bar
Next:    "Beginner Question: I'm new to autoit but have a good experience in python with pytest framework. is autoIt with pytest possible
Previous:    "(Frozen) AutoIt v3 Window Info" window
Other Props:    Object has no additional properties
Children:    (null) title bar
Ancestors:    "Desktop 1" pane
    [ No Parent ]
 

I tried JavaFerret but it was crashing as my Java version is 8 and it supports only Java7 and under. However JavaMonkey worked for me but could access only the tree view as below

As suggested did run simplespy and executed ctrl+w got the below text in the window:

    https://www.autoitscript.com/forum/topic/153520-iuiautomation-ms-framework-automate-chrome-ff-ie/?do=findComment&comment=1156373
At least we have an element title: [Program Manager] class: [Progman]

Having the following values for all properties: 
Title is: <Program Manager>    Class   := <Progman>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    0;0;3840;1080
*** Parent Information top down ***

;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("ProgramManager.mainwindow","title:=Program Manager;classname:=Progman")

;~ Actions split away from logical/technical definition above can come from configfiles 

_UIA_action("ProgramManager.mainwindow","setfocus")

;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("ProgramManager.mainwindow","title:=Program Manager;classname:=Progman")
_UIA_action("ProgramManager.mainwindow","setfocus")

*** Detailed properties of the highlighted element ***
UIA_title:= <Program Manager>
UIA_text:= <Program Manager>
UIA_regexptitle:= <Program Manager>
UIA_class:= <Progman>
UIA_regexpclass:= <Progman>
UIA_iaccessiblechildId:= <0>
UIA_handle:= <65934>
UIA_RuntimeId:= <42;65934>
UIA_BoundingRectangle:= <0;0;3840;1080>
UIA_ProcessId:= <12160>
UIA_ControlType:= <50033>
UIA_LocalizedControlType:= <pane>
UIA_Name:= <Program Manager>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <True>
UIA_IsEnabled:= <True>
UIA_ClassName:= <Progman>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <65934>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_FrameworkId:= <Win32>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <False>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <False>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <False>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <False>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <3>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleName:= <Program Manager>
UIA_LegacyIAccessibleRole:= <10>
UIA_LegacyIAccessibleState:= <1048576>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:28544,providerId:0x1018E Main:Nested [pid:12160,providerId:0x1018E Annotation(parent link):Microsoft: Annotation Proxy (unmanaged:UIAutomationCore.DLL); Main:Microsoft: MSAA Proxy (unmanaged:UIAutomationCore.DLL)]; Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>
UIA_OptimizeForVisualContent:= <False>
UIA_IsObjectModelPatternAvailable:= <False>
UIA_AnnotationAnnotationTypeId:= <60000>
UIA_IsAnnotationPatternAvailable:= <False>
UIA_IsTextPattern2Available:= <False>
UIA_StylesStyleId:= <0>
UIA_StylesFillColor:= <0>
UIA_StylesFillPatternColor:= <0>
UIA_IsStylesPatternAvailable:= <False>
UIA_IsSpreadsheetPatternAvailable:= <False>
UIA_IsSpreadsheetItemPatternAvailable:= <False>
UIA_Transform2CanZoom:= <False>
UIA_IsTransformPattern2Available:= <False>
UIA_LiveSetting:= <0>
UIA_IsTextChildPatternAvailable:= <False>
UIA_IsDragPatternAvailable:= <False>
UIA_DragIsGrabbed:= <False>
UIA_IsDropTargetPatternAvailable:= <False>
UIA_Transform2ZoomLevel:= <1>
UIA_Transform2ZoomMinimum:= <1>
UIA_Transform2ZoomMaximum:= <1>
UIA_IsTextEditPatternAvailable:= <False>
UIA_IsPeripheral:= <False>
UIA_IsCustomNavigationPatternAvailable:= <False>
UIA_PositionInSet:= <0>
UIA_SizeOfSet:= <0>
UIA_Level:= <0>
UIA_LandmarkType:= <0>
UIA_FillType:= <0>
UIA_VisualEffects:= <0>
UIA_IsSelectionPattern2Available:= <False>
UIA_Selection2ItemCount:= <0>
UIA_HeadingLevel:= <80050>
UIA_IsDialog:= <False>

I verified whether the simplespy works on other applications or not. I tried on notepad and it's working fine. I believe it has issues with my application so tried Access Bridge Explorer and it gave some what okayish accessible component properties.

Access Bridge Explorer displayed tree view and some accessible controls as below:

Now my question is whether I can automate using the accessible properties above. If yes please suggest me with helpful links. I went through most of the topics but couldn't find anything helpful. 

I'm not sure which properties to use especially in this tree structure. Kindly help me out.

[autotest410]

I finally make this working at least for search and clear functionality. It is a great learning for me.