guestscripter Posted January 13, 2014 Posted January 13, 2014 (edited) I´ve got an already usably working script for entering a reservation in to a Java App, but to give it more than very basic functionality and stability I´m stuck. Is there any way beyond MouseGetCursor and WinActivate, and other than lot´s of Sleep, that I can make this more stable? sometimes there is a popup in the app, which of course the script doesn´t see, and sometimes the script "trips" and enters the wrong data in the wrong field, and that of course causes lots of errors and popups... yes, there´s a "Java UDF" Topic in the example scripts section, but that didn´t work for me, and the development on that seems to have stopped years ago... (unless I´m overlooking another thread somewhere?) A newer thread about, as far as I understand it, automating hard-to-automate controls might be a direction: ?do=embed' frameborder='0' data-embedContent>... Here´s a cut-out of what I´ve built so far: expandcollapse popupGlobal $sWinTitle = "OPERA PMS [Version" Global $wHandle = WinGetHandle($sWinTitle) Global $cHandle = ControlGetHandle($wHandle, "", "") #region Parser ;that´s a separate story #endregion #region Entry _BlockWindow() OperaSend("!.") OperaSend($aRES[$ixGuestSurName],2) OperaSend("{TAB}") OperaSend($aRES[$ixGuestFirstName],2) OperaSend("{TAB}") OperaSend("i{ENTER}") OperaSend("!n") ;etc... _unBlock() #endregion Func OperaSend($sSend, $flag = 0) While Not WinActive($wHandle);force the window to be active WinActivate($wHandle) WEnd If $flag = 2 Then ;$flag = 2 send longer strings of text with clipput+paste Do ClipPut("") ClipPut($sSend) Sleep(128) ;seems to be necessary Until @error = 0 Do $x = ControlSend($wHandle, "", $cHandle, "^v");{CTRLDOWN}{CTRLUP} Until $x = 1 Sleep(128) Else ;flag = 1 Sends Raw, $flag = 0 Sends "normal" Do $x = ControlSend($wHandle, "", $cHandle, $sSend, $flag) Until $x = 1 Sleep(64) EndIf Do Sleep(8) Until MouseGetCursor() = 2 ;check for non-arrow mouse cursors (busy, etc.) Sleep(8) EndFunc ;==>OperaSend #include <Misc.au3> ; for MouseTrap Function Func _BlockWindow() $pos = WinGetPos($wHandle) If IsHWnd($wHandle) Then WinActivate($wHandle) WinSetState($wHandle, "", @SW_DISABLE) WinSetTrans($wHandle, "", 255 * 0.92) _MouseTrap($pos[0] + 10, $pos[1] + 32) _BlockKeys(1) ConsoleWrite("Blocked" & @CR) $blocked = 1 EndIf EndFunc ;==>_BlockWindow Func _unBlock() If IsHWnd($wHandle) Then _BlockKeys() _MouseTrap() WinSetTrans($wHandle, "", 255) WinSetState($wHandle, "", @SW_ENABLE) ConsoleWrite("Unblocked" & @CR) $blocked = 0 EndIf EndFunc ;==>_unBlock Func _BlockKeys($flag = 0);instead of _BlockInput(), because no Admin privileges If $flag = 1 Then HotKeySet("{TAB}", "_BlockKeysDummy") HotKeySet("^{TAB}", "_BlockKeysDummy") HotKeySet("!{TAB}", "_BlockKeysDummy");doesn´t really work HotKeySet("+{TAB}", "_BlockKeysDummy") HotKeySet("#{TAB}", "_BlockKeysDummy") HotKeySet("{ENTER}", "_BlockKeysDummy") ;...etc. Else HotKeySet("{TAB}") HotKeySet("^{TAB}") HotKeySet("!{TAB}");doesn´t really work HotKeySet("+{TAB}") HotKeySet("#{TAB}") HotKeySet("{ENTER}") ;...etc. EndIf EndFunc ;==>_BlockKeys Edited January 18, 2014 by guestscripter ImageSearch15.au3 featuring _ImageSearchStartup() and _ImageSearchShutdown()
guestscripter Posted January 18, 2014 Author Posted January 18, 2014 (edited) Is there any smarter way or more elegant to do this? Func CSTestInWin($wHandle, $aCheckSumTest, $iCheckSumStep = 1, $bCheckSumMode = 1) ;$aCheckSumTest is Array where [0] is a checksum, ;and [1]-[4] are left, top, right and bottom coordinates respectively. ;The coordinates shall be relative to the window specified by the $wHandle. ;Option "PixelCoordMode" needs to be set to 0=relative. If WinExists($wHandle) And UBound($aCheckSumTest) = 5 And Opt("PixelCoordMode") = 0 Then Local $aCheckSum = PixelChecksum($aCheckSumTest[1], $aCheckSumTest[2], _ $aCheckSumTest[3], $aCheckSumTest[4], $iCheckSumStep, $wHandle, $bCheckSumMode) If $aCheckSum = $aCheckSumTest[0] Then Return True Else Return False EndIf Else SetError(1) ConsoleWrite("_CheckSumTestInWindow found invalid parameter, or ""PixelCoordMode"" option not set to 0."&@CRLF) Return False EndIf EndFunc ;==>CSTestInWin ;Usage example: Opt("PixelCoordMode",0) ;... Local Const $JavaAppWinTitle = "xyz" Local Const $aProfileSearchNoResult[5] = [1798928452, 140, 155, 1002, 748] Local Const $aProfileSearchWithResults[5] = [4654546456, 140, 155, 1002, 748] ;... $wHandleJavaApp = WinWaitActive($sJavaAppWinTitle) ;... While NOT CSTestInWin($wHandleJavaApp,$aProfileSearchNoResult) _ OR CSTestInWin($wHandleJavaApp,$aProfileSearchWithResults) _ OR $bExitLoop = 0 ;tight loop WEnd If CSTestInWin($wHandleJavaApp,$aProfileSearchNoResult) Then CreateNewProfile() ElseIf CSTestInWin($wHandleJavaApp,$aProfileSearchWithResults) Then QueryUser_UseExistingProfileOrCreateNew() EndIf To clarify, this is part of a script I am building that automates software that uses Java, so not identifiable Controls to "see" or work with (although I found I can use ControlSend to send to the right window at least). Already tried Java UDF and it looks like I need a workaround because this "Java Access Bridge" that Java UDF works with only helps with identifying Controls in "accessibility friendly" Java Software. So that way works for example for jEdit, but not for the software I want to automate. (if you know another way please let me know!) I looked around at different things like "ImageSearch" and "FastFind".. ...UDFs that use .dll´s to offer functionality like searching for a button by a .bmp file, just then I found that the native PixelCheckSum function is already well powerfull. Edited January 18, 2014 by guestscripter ImageSearch15.au3 featuring _ImageSearchStartup() and _ImageSearchShutdown()
junkew Posted January 18, 2014 Posted January 18, 2014 you could try this '?do=embed' frameborder='0' data-embedContent>> guestscripter 1 FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets
guestscripter Posted January 18, 2014 Author Posted January 18, 2014 (edited) First of all thank you very much for the recommendation. I downloaded the include files and ran the "simple spy demo" from the first post in that thread. With my java interface window in focus, and the caret in an edit box selecting the text "asdasdasdasd" as a test, I got what I have pasted below. Maybe this can help you help me help myself? expandcollapse popupMouse position is retrieved 649-314 At least we have an element [MYWINDOWTITLEWASHERE ][SunAwtFrame] Having the following values for all properties: Title is: <MYWINDOWTITLEWASHERE > Class := <SunAwtFrame> controltype:= <UIA_WindowControlTypeId> ,<50032> , (0000C370) *** Parent Information *** Title is: <Desktop> Class := <#32769> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371) *** Detailed properties of the highlighted element *** UIA_AcceleratorKeyPropertyId := UIA_AccessKeyPropertyId :=Alt+ UIA_AriaPropertiesPropertyId := UIA_AriaRolePropertyId := UIA_AutomationIdPropertyId := UIA_BoundingRectanglePropertyId :=448;156;1007;694 UIA_ClassNamePropertyId :=SunAwtFrame UIA_ClickablePointPropertyId := UIA_ControllerForPropertyId := UIA_ControlTypePropertyId :=50032 UIA_CulturePropertyId :=0 UIA_DescribedByPropertyId := UIA_DockDockPositionPropertyId :=5 UIA_ExpandCollapseExpandCollapseStatePropertyId :=3 UIA_FlowsToPropertyId := UIA_FrameworkIdPropertyId :=Win32 UIA_GridColumnCountPropertyId :=0 UIA_GridItemColumnPropertyId :=0 UIA_GridItemColumnSpanPropertyId :=1 UIA_GridItemContainingGridPropertyId := UIA_GridItemRowPropertyId :=0 UIA_GridItemRowSpanPropertyId :=1 UIA_GridRowCountPropertyId :=0 UIA_HasKeyboardFocusPropertyId :=True UIA_HelpTextPropertyId := UIA_IsContentElementPropertyId :=True UIA_IsControlElementPropertyId :=True UIA_IsDataValidForFormPropertyId :=False UIA_IsDockPatternAvailablePropertyId :=False UIA_IsEnabledPropertyId :=True UIA_IsExpandCollapsePatternAvailablePropertyId :=False UIA_IsGridItemPatternAvailablePropertyId :=False UIA_IsGridPatternAvailablePropertyId :=False UIA_IsInvokePatternAvailablePropertyId :=False UIA_IsItemContainerPatternAvailablePropertyId :=False UIA_IsKeyboardFocusablePropertyId :=True UIA_IsLegacyIAccessiblePatternAvailablePropertyId :=True UIA_IsMultipleViewPatternAvailablePropertyId :=False UIA_IsOffscreenPropertyId :=False UIA_IsPasswordPropertyId :=False UIA_IsRangeValuePatternAvailablePropertyId :=False UIA_IsRequiredForFormPropertyId :=False UIA_IsScrollItemPatternAvailablePropertyId :=False UIA_IsScrollPatternAvailablePropertyId :=False UIA_IsSelectionItemPatternAvailablePropertyId :=False UIA_IsSelectionPatternAvailablePropertyId :=False UIA_IsSynchronizedInputPatternAvailablePropertyId :=False UIA_IsTableItemPatternAvailablePropertyId :=False UIA_IsTablePatternAvailablePropertyId :=False UIA_IsTextPatternAvailablePropertyId :=False UIA_IsTogglePatternAvailablePropertyId :=False UIA_IsTransformPatternAvailablePropertyId :=True UIA_IsValuePatternAvailablePropertyId :=False UIA_IsVirtualizedItemPatternAvailablePropertyId :=False UIA_IsWindowPatternAvailablePropertyId :=True UIA_ItemStatusPropertyId := UIA_ItemTypePropertyId := UIA_LabeledByPropertyId := UIA_LegacyIAccessibleChildIdPropertyId :=0 UIA_LegacyIAccessibleDefaultActionPropertyId := UIA_LegacyIAccessibleDescriptionPropertyId := UIA_LegacyIAccessibleHelpPropertyId := UIA_LegacyIAccessibleKeyboardShortcutPropertyId :=Alt+ UIA_LegacyIAccessibleNamePropertyId :=MYWINDOWTITLEWASHERE UIA_LegacyIAccessibleRolePropertyId :=10 UIA_LegacyIAccessibleSelectionPropertyId := UIA_LegacyIAccessibleStatePropertyId :=1048580 UIA_LegacyIAccessibleValuePropertyId := UIA_LocalizedControlTypePropertyId :=Fenster UIA_MultipleViewCurrentViewPropertyId :=0 UIA_MultipleViewSupportedViewsPropertyId := UIA_NamePropertyId :=MYWINDOWTITLEWASHERE UIA_NativeWindowHandlePropertyId :=1836932 UIA_OrientationPropertyId :=0 UIA_ProcessIdPropertyId :=2696 UIA_ProviderDescriptionPropertyId :=[pid:5780,hwnd:0x1C0784 Main:Nested [pid:2696,hwnd:0x1C0784 Annotation(parent link):Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main:Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Nonclient:Microsoft: Non-Client Proxy (unmanaged:uiautomationcore.dll); Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)] UIA_RangeValueIsReadOnlyPropertyId :=True UIA_RangeValueLargeChangePropertyId :=0 UIA_RangeValueMaximumPropertyId :=0 UIA_RangeValueMinimumPropertyId :=0 UIA_RangeValueSmallChangePropertyId :=0 UIA_RangeValueValuePropertyId :=0 UIA_RuntimeIdPropertyId :=42;1836932 UIA_ScrollHorizontallyScrollablePropertyId :=False UIA_ScrollHorizontalScrollPercentPropertyId :=0 UIA_ScrollHorizontalViewSizePropertyId :=100 UIA_ScrollVerticallyScrollablePropertyId :=False UIA_ScrollVerticalScrollPercentPropertyId :=0 UIA_ScrollVerticalViewSizePropertyId :=100 UIA_SelectionCanSelectMultiplePropertyId :=False UIA_SelectionIsSelectionRequiredPropertyId :=False UIA_SelectionselectionPropertyId := UIA_SelectionItemIsSelectedPropertyId :=False UIA_SelectionItemSelectionContainerPropertyId := UIA_TableColumnHeadersPropertyId := UIA_TableItemColumnHeaderItemsPropertyId := UIA_TableRowHeadersPropertyId := UIA_TableRowOrColumnMajorPropertyId :=2 UIA_TableItemRowHeaderItemsPropertyId := UIA_ToggleToggleStatePropertyId :=2 UIA_TransformCanMovePropertyId :=True UIA_TransformCanResizePropertyId :=True UIA_TransformCanRotatePropertyId :=False UIA_ValueIsReadOnlyPropertyId :=True UIA_ValueValuePropertyId := UIA_WindowCanMaximizePropertyId :=True UIA_WindowCanMinimizePropertyId :=True UIA_WindowIsModalPropertyId :=False UIA_WindowIsTopmostPropertyId :=False UIA_WindowWindowInteractionStatePropertyId :=2 UIA_WindowWindowVisualStatePropertyId :=0 I can see that it found a Window Handle and a Control Handle of some sort... ...just I suspect this control handle is the "one size fits all" that I have resorted to using so far like $wHandleOpera = WinWaitActive($sOperaWinTitle) Global $cHandleOpera = ControlGetHandle($wHandleOpera, "", "") ;..stuff here... ControlSend($wHandleOpera, "", $cHandleOpera, "{TAB}") ClipPut("asfljalsfjlasjflkasj") ControlSend($wHandleOpera, "", $cHandleOpera, "^v") which, in a shaky (script trips when user presses a key or window loses focus, so lots of mouse/key/etc. blocking) and limited (needing pixelchecksums to give the script any kind of "eyes") way, kind of works.... Or can you see something more usable in the results? Can these results indicated yes/no if this software is potentially automatable with the IUIAutomation Functions? Edited January 18, 2014 by guestscripter ImageSearch15.au3 featuring _ImageSearchStartup() and _ImageSearchShutdown()
Solution junkew Posted January 18, 2014 Solution Posted January 18, 2014 The UI Automation will not come far enough if it only exposes AWT Frame Attached some code which shows JAB (Java Accessibility Bridge) can be loaded from AutoIT. Not finished at all (put on my long term TODO list) Maybe its usefull for you or others as a starting point. expandcollapse popup;~ By inspecting the WindowsAccessBridge-32.dll it reveals some information about the hidden dialogs ;~ So it seems the hidden dialog is shown after you call windows_run() no clue if interaction is needed ;~ ;~ Somehow it sends a message unclear if this is to the JVM to respond to ;~ push SSZ6E73E320_AccessBridge_FromJava_Hello ;~ push SSZ6E73E300_AccessBridge_FromWindows_Hello ;~ db 'AccessBridge-FromWindows-Hello',0 ;~ db 'AccessBridge-FromJava-Hello',0 ;~ Copied from this NVDA reference and translated to AutoIT ;~ http://www.webbie.org.uk/nvda/api/JABHandler-pysrc.html ;~ ;~ def initialize(): ;~ global isRunning ;~ if not bridgeDll: ;~ raise NotImplementedError("dll not available") ;~ bridgeDll.Windows_run() ;~ #Accept wm_copydata and any wm_user messages from other processes even if running with higher privilages ;~*** ChangeWindowMessageFilter=getattr(windll.user32,'ChangeWindowMessageFilter',None) ;~*** if ChangeWindowMessageFilter: ;~*** if not ChangeWindowMessageFilter(winUser.WM_COPYDATA,1): ;~*** raise WinError() ;~*** for msg in xrange(winUser.WM_USER+1,65535): ;~*** if not ChangeWindowMessageFilter(msg,1): ;~*** raise WinError() ;~ #Register java events ;~ bridgeDll.setFocusGainedFP(internal_event_focusGained) ;~ bridgeDll.setPropertyActiveDescendentChangeFP(internal_event_activeDescendantChange) ;~ bridgeDll.setPropertyNameChangeFP(event_nameChange) ;~ bridgeDll.setPropertyDescriptionChangeFP(event_descriptionChange) ;~ bridgeDll.setPropertyValueChangeFP(event_valueChange) ;~ bridgeDll.setPropertyStateChangeFP(internal_event_stateChange) ;~ bridgeDll.setPropertyCaretChangeFP(internal_event_caretChange) ;~ isRunning=True ;create the structs ;~ ;~ http://docs.oracle.com/javase/accessbridge/2.0.2/api.htm ;~ ;~ #define MAX_STRING_SIZE 1024 ;~ #define SHORT_STRING_SIZE 256 ;~ struct AccessBridgeVersionInfo { ;~ wchar_t VMversion[SHORT_STRING_SIZE]; // version of the Java VM ;~ wchar_t bridgeJavaClassVersion[SHORT_STRING_SIZE]; // version of the AccessBridge.class ;~ wchar_t bridgeJavaDLLVersion[SHORT_STRING_SIZE]; // version of JavaAccessBridge.dll ;~ wchar_t bridgeWinDLLVersion[SHORT_STRING_SIZE]; // version of WindowsAccessBridge.dll ;~ }; ;~ Local $AccessBridgeVersionInfo=DllStructCreate("WCHAR VMversion[256];WCHAR bridgeJavaClassVersion[256];WCHAR bridgeJavaDLLVersion[256];WCHAR bridgeWinDLLVersion[256]") Local $AccessBridgeVersionInfo=DllStructCreate("WCHAR VMversion[256];WCHAR bridgeJavaClassVersion[256];WCHAR bridgeJavaDLLVersion[256];WCHAR bridgeWinDLLVersion[256]") ;~ struct AccessibleContextInfo { ;~ wchar_ name[MAX_STRING_SIZE]; // the AccessibleName of the object ;~ wchar_ description[MAX_STRING_SIZE]; // the AccessibleDescription of the object ;~ wchar_ role[SHORT_STRING_SIZE]; // localized AccesibleRole string ;~ wchar_ states[SHORT_STRING_SIZE]; // localized AccesibleStateSet string ;~ // (comma separated) ;~ jint indexInParent // index of object in parent ;~ jint childrenCount // # of children, if any ;~ jint x; // screen x-axis co-ordinate in pixels ;~ jint y; // screen y-axis co-ordinate in pixels ;~ jint width; // pixel width of object ;~ jint height; // pixel height of object ;~ BOOL accessibleComponent; // flags for various additional ;~ BOOL accessibleAction; // Java Accessibility interfaces ;~ BOOL accessibleSelection; // FALSE if this object doesn't ;~ BOOL accessibleText; // implement the additional interface ;~ BOOL accessibleInterfaces; // new bitfield containing additional ;~ // interface flags ;~ }; ;~ Local $AccessibleContextInfo=DllStructCreate("WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR states[256];int indexInParent;int childrenCount;int x;int y;int width;int height;BOOL accessibleComponent;BOOL accessibleAction;BOOL accessibleSelection;BOOL accessibleText;BOOL accessibleInterfaces") Local $AccessibleContextInfo=DllStructCreate("WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR states[256];int64 indexInParent;int64 childrenCount;int64 x;int64 y;int64 width;int64 height;byte accessibleComponent;byte accessibleAction;byte accessibleSelection;byte accessibleText;byte accessibleinterfaces") ;~ #AutoIt3Wrapper_UseX64=Y #include <EditConstants.au3> #include <GUIConstantsEx.au3> #include <WindowsConstants.au3> #include <constants.au3> #include <WinAPI.au3> #include <debug.au3> local $hwnd local $i local $result local $vmID local $ac ;~ Make messages elevated _ChangeWindowMessageFilter($WM_COPYDATA,1) for $i=$WM_USER to $WM_USER+65536 _ChangeWindowMessageFilter($i, 1) Next Func _ChangeWindowMessageFilter($iMsg, $iAction) Local $aCall = DllCall("user32.dll", "bool", "ChangeWindowMessageFilter", "dword", $iMsg, "dword", $iAction) If @error Or Not $aCall[0] Then Return SetError(1, 0, 0) Return 1 EndFunc local $bridgeDLL=DLLOPEN("WindowsAccessBridge-32.dll") ;~ sleep(500) If $bridgeDLL=true Then consolewrite($bridgeDLL & @CRLF) Else consolewrite("DLL not found try to change to WindowsAccessBridge-64.dll if you use 64 bits autoit and windows") endIf ;~ TODO: Handle messages received from initialize $result =dllcall($bridgeDLL,"int:cdecl", "Windows_run") consolewrite($result & " " & @error & " initializeAccessBridge is finished") sleep(250) consolewrite("Windows_run passed :" & $result & @CRLF) Local $var = WinList() consolewrite("Before loading all Windows:" & $var[0][0] & @CRLF) For $i = 1 To $var[0][0] ; Only display visble windows that have a title If IsVisible($var[$i][1]) Then local $handle=wingethandle($var[$i][0]) $result =dllcall($bridgeDLL,"BOOL:cdecl", "isJavaWindow", "hwnd", $handle) if @error=0 Then if $result[0]=1 Then consolewrite( $i & " Java Window Title=" & $var[$i][0] & " Handle=" & $var[$i][1] & @TAB & " res: " & $result[0] & @CRLF) local $ac=0 local $vmID=0 $result =dllcall($bridgeDLL,"BOOL:cdecl", "getAccessibleContextFromHWND", "hwnd", $handle, "long*", $vmID, "ptr*", $ac) if @error=0 Then consolewrite("We have a context " & @CRLF) $vmID=$result[2] $ac=$result[3] if @error > 0 then consolewrite("Struct error") ;~ $result =dllcall($bridgeDLL, "BOOL:cdecl", "getVersionInfo", "long", $vmId, "struct", $AccessBridgeVersionInfo) $result =dllcall($bridgeDLL, "BOOL:cdecl", "getVersionInfo", "long", $vmId, "ptr", DllStructGetPtr($AccessBridgeVersionInfo)) consolewrite( @error & " context found of " & $vmID & @CRLF) if @error=0 Then $s1=dllstructgetdata($AccessBridgeVersionInfo, "VMVersion") $s2=dllstructgetdata($AccessBridgeVersionInfo, "bridgeJavaClassVersion") $s3=dllstructgetdata($AccessBridgeVersionInfo, "bridgeJavaDLLVersion") $s4=dllstructgetdata($AccessBridgeVersionInfo, "bridgeWinDLLVersion") consolewrite("VMVersion: <" & $s1 & ">" & @CRLF) consolewrite("bridgeJavaClassVersion: <" & $s2 & ">" & @CRLF) consolewrite("bridgeJavaDLLVersion: <" & $s3 & ">" & @CRLF) consolewrite("bridgeWinDLLVersion: <" & $s4 & ">" & @CRLF) EndIf ;~ Retrieves an AccessibleContextInfo object of the AccessibleContext object ac. $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "ptr", $ac, "ptr", DllStructGetPtr($AccessibleContextInfo)) ;~ Local $AccessibleContextInfo=DllStructCreate("WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR states[256];int indexInParent;int childrenCount;int x;int y;int width;int height;BOOL accessibleComponent;BOOL accessibleAction;BOOL accessibleSelection;BOOL accessibleText;BOOL accessibleInterfaces") if @error<>0 Then consolewrite("We have an error " & @CRLF) EndIf $s1=dllstructgetdata($AccessibleContextInfo, "name") $s2=dllstructgetdata($AccessibleContextInfo, "description") consolewrite("name: <" & $s1 & ">" & @CRLF) consolewrite("description: <" & $s2 & ">" & @CRLF) ;~ AccessibleContext GetAccessibleChildFromContext(long vmID, AccessibleContext ac, jint index); ;~ Returns an AccessibleContext object that represents the nth child of the object ac, where n is specified by the value index. Else consolewrite( @error & " No context found" & @CRLF) endif EndIf Else EndIf endif Next ;~ http://www.autohotkey.com/board/topic/95343-how-to-send-to-unseen-controls-in-a-java-app/ local $result =dllcall($bridgeDLL,"BOOL", "shutdownAccessBridge") Func IsVisible($handle) If BitAND(WinGetState($handle), 2) Then Return 1 Else Return 0 EndIf EndFunc ;==>IsVisible guestscripter 1 FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets
guestscripter Posted January 19, 2014 Author Posted January 19, 2014 (edited) Thanks junkew. For my current skill level the code looks heavy, and besides I highly doubt that the App I want to automate supports accessibility... (I´ve done some research on JAB before when trying to use the Java UDF ('?do=embed' frameborder='0' data-embedContent>>) by seangriffin). Thanks a lot anyway! I´m marking the thread as "solved", as the question, thanks to you, has been answered as well as possible for now I reckon. Planning to instead continue cleaning up my "dirty" method with PixelChecksums and ControlSend (to the overall window) as much as possible... (I´ve made progress with that to a point that actually looks quite satisfying) Edited January 19, 2014 by guestscripter ImageSearch15.au3 featuring _ImageSearchStartup() and _ImageSearchShutdown()
guestscripter Posted January 19, 2014 Author Posted January 19, 2014 (edited) P.S. If anyone has interest (or can improve the direction I´m taking) here are snippets of how the "dirty" method is working. (any pointers in regards to best coding practice would be great) Global $wHandleOpera Global $cHandleOpera $wHandleOpera = WinActivate($sOperaWinTitle) $cHandleOpera = ControlGetHandle($wHandleOpera, "", "") Func OperaSend($sSend, $flag = 0) WaitForTheMouse() If $flag = 2 Then ClipPut("") ClipPut($sSend) Sleep(128);Must not be removed Do Local $x = ControlSend($wHandleOpera, "", $cHandleOpera, "^v") Until $x = 1 ;Sleep(128) Else Sleep(64) Do $x = ControlSend($wHandleOpera, "", $cHandleOpera, $sSend, $flag) Until $x = 1 EndIf EndFunc ;==>OperaSend Func WaitForTheMouse() While MouseGetCursor() <> 2 ;cr("WaitForTheMouse") Sleep(100) WEnd EndFunc ;==>WaitForTheMouse Func CSTestInOpera($aCheckSumTest, $iCheckSumStep = 1, $bCheckSumMode = 1) ;$aCheckSumTest is Array where [0] is a checksum and [1]-[4] are left, top, right and bottom coordinates respectively. ;Window Handle from Global Scope. Option "PixelCoordMode" needs to be set to 0=relative. If WinExists($wHandleOpera) And UBound($aCheckSumTest) = 5 Then Local $aCheckSum = PixelChecksum($aCheckSumTest[1], $aCheckSumTest[2], $aCheckSumTest[3], $aCheckSumTest[4], $iCheckSumStep, $wHandleOpera, $bCheckSumMode) If $aCheckSum = $aCheckSumTest[0] Then Return True Else Return False EndIf Else SetError(1) Return False EndIf EndFunc ;==>CSTestInOpera Func AdlibCheckWinStillActive() If Not WinActive($wHandleOpera) Then WinActivate($wHandleOpera) If @error Then _Exit() EndIf EndIf EndFunc ;==>AdlibCheckWinStillActive Func BlockInactiveWindowsAndMouse($flag = 0);if Flag = 1 then just reblock mouse ;Note: Check out http://www.autoitscript.com/forum/topic/87735-blockinputex-udf/ If $flag = 0 Then WinSetOnTop($wHandleOpera, "", 1) WinSetState($wHandleOpera, "", @SW_DISABLE) Global $aBlockerWinList = WinList() For $i = 1 To $aBlockerWinList[0][0] If BitAND(WinGetState($aBlockerWinList[$i][1]), 2) And BitAND(WinGetState($aBlockerWinList[$i][1]), 4) And $aBlockerWinList[$i][1] <> $wHandleOpera Then;Visible+Enabled+NotActive WinSetState($aBlockerWinList[$i][1], "", @SW_DISABLE) $aBlockerWinList[$i][0] = 1 Else $aBlockerWinList[$i][0] = 0 EndIf Next WinMove($wHandleOpera, "", Default, Default, 1024, 768) EndIf Local $aTempWinGetPos = WinGetPos($wHandleOpera) _MouseTrap($aTempWinGetPos[0] + Int($aTempWinGetPos[2] / 2), $aTempWinGetPos[1] + Int($aTempWinGetPos[3] / 2), $aTempWinGetPos[0] + Int($aTempWinGetPos[2] / 2), $aTempWinGetPos[1] + Int($aTempWinGetPos[3] / 2)) If Not WinActive($wHandleOpera) Then WinActivate($wHandleOpera) EndFunc ;==>BlockInactiveWindowsAndMouse Func UnDoBlockInactiveWindowsAndMouse() _MouseTrap() WinSetOnTop($wHandleOpera, "", 0) For $i = 1 To $aBlockerWinList[0][0] If $aBlockerWinList[$i][0] = 1 Then WinSetState($aBlockerWinList[$i][1], "", @SW_ENABLE) EndIf Next If Not WinActive($wHandleOpera) Then WinActivate($wHandleOpera) WinSetState($wHandleOpera, "", @SW_ENABLE) EndFunc ;==>UnDoBlockInactiveWindowsAndMouse Edited January 19, 2014 by guestscripter ImageSearch15.au3 featuring _ImageSearchStartup() and _ImageSearchShutdown()
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now