Jump to content

Recommended Posts

Good afternoon all!

I am in the process of revisiting some of my previous issues now that I have a better understanding of AutoIt and have made some decent strides in other areas of my automation. Originally I had(and still do) an issue with reading ListItems from a list box within my application. This is the only control I could not get a good read on. Even hidden elements, etc. were visible when given the right script.

Fast forward to today and I just happened to download UI Accessibility checker. Previously, nothing I've tried has been able to see more than just the list box. However, this application identifies each name in the list in the tree view. This has given me hope that there may be a solution to automate this.

You can see the control and the list tree below:

Capture.thumb.JPG.a0deaca9da5adfd45aa863

I've done a bit of research in the forums, but haven't seen anything with definitive evidence that this can be done with AutoIt. I am currently attempting to do everything in AutoItx4Java and using base AutoIt commands, which has worked well so far. Does anyone have any insight into how I might be able to interact with these elements in AutoIt or a java based solution that I might try. This would really perfect the project I've been working on and any help would be greatly appreciated.

I have looked into the UIA thread, but it doesn't seem to be as useful here as UIA inspection does not return any results.

Thank you for any and all replies!

Share this post


Link to post
Share on other sites

did you try simplespy and inspect as described in the uia thread?
whats the output of those tools, as you use the UI Accessibility checker I am 99.99% sure that inspect and simplespy give you nformation and that way you can automate.

This thread can also help but I did not find time to enhance / investigate further. (what you post seems to be a Java application)
this makes use of the JavaAccessibilityBridge and as such called it JABSimpleSpy.
https://www.autoitscript.com/forum/topic/166830-java-object-automation-and-simple-spy/

Share this post


Link to post
Share on other sites

As far as I know the UI Accessibility checker (AccCheckUI.exe) is the predecessor of Inspect.exe. Both programs are based on two methods to identify windows and controls: MSAA-techniques and UIA-techniques. Because AccCheckUI.exe is the old program I think it's primarily based on MSAA-techniques. And the new program Inspect.exe is primarily based on UIA-techniques.

If AccCheckUI.exe can identify the list items, Inspect.exe should also be able to identify the list items. I remember you didn't have much success with Inspect.exe in the old thread. But did you try to run Inspect.exe in pure MSAA mode (combo box in upper left corner)?

If you can verify that the list items can be identified with MSAA-techniques, there can be a chance to be able to automate the list box with MSAA-code (instead of UIA-code).

Share this post


Link to post
Share on other sites

Sorry for the late reply - I got started on something else this week and am just now coming back to all of this. Thanks for getting back with me on this - I was hoping both of you would respond!

@junkew Using Simple UIA Spy I get the following result:

Mouse position is retrieved 1006-544
At least we have an element [][]

Having the following values for all properties: 
Title is: <>    Class   := <>   controltype:= <UIA_ListItemControlTypeId>   ,<50007>    , (0000C357)    0;1;82524432;536870911
*** Parent Information top down ***
8: Title is: <Desktop>  Class   := <#32769> controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    0;0;1920;1080
"Title:=Desktop;controltype:=UIA_PaneControlTypeId;class:=#32769"   
7: Title is: <MEDHOST>  Class   := <LSPANEL>    controltype:= <UIA_WindowControlTypeId> ,<50032>    , (0000C370)    449;167;1022;746
"Title:=MEDHOST;controltype:=UIA_WindowControlTypeId;class:=LSPANEL"    
6: Title is: <Setlib>   Class   := <LSPANEL>    controltype:= <UIA_WindowControlTypeId> ,<50032>    , (0000C370)    664;362;591;356
"Title:=Setlib;controltype:=UIA_WindowControlTypeId;class:=LSPANEL" 
5: Title is: <Setlib>   Class   := <LSPANEL>    controltype:= <UIA_WindowControlTypeId> ,<50032>    , (0000C370)    664;389;591;302
"Title:=Setlib;controltype:=UIA_WindowControlTypeId;class:=LSPANEL" 
4: Title is: <Setlib>   Class   := <LSPANEL>    controltype:= <UIA_WindowControlTypeId> ,<50032>    , (0000C370)    677;366;591;352
"Title:=Setlib;controltype:=UIA_WindowControlTypeId;class:=LSPANEL" 
3: Title is: <> Class   := <#32770> controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    685;396;4096;4096
"Title:=;controltype:=UIA_PaneControlTypeId;class:=#32770"  
2: Title is: <Select Library List to Copy>  Class   := <LSLISTBOX>  controltype:= <UIA_ListControlTypeId>   ,<50008>    , (0000C358)    718;456;451;179
"Title:=Select Library List to Copy;controltype:=UIA_ListControlTypeId;class:=LSLISTBOX"    
1: Title is: <> Class   := <LSLISTSUB>  controltype:= <UIA_ListControlTypeId>   ,<50008>    , (0000C358)    719;457;436;177
"Title:=;controltype:=UIA_ListControlTypeId;class:=LSLISTSUB"   
0: Title is: <> Class   := <LSLISTLIST> controltype:= <UIA_ListControlTypeId>   ,<50008>    , (0000C358)    719;474;4096;160
"Title:=;controltype:=UIA_ListControlTypeId;class:=LSLISTLIST"  


;~ *** Standard code ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP7=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=MEDHOST;controltype:=UIA_WindowControlTypeId;class:=LSPANEL", $treescope_children)    
_UIA_Action($oP7,"setfocus")
Local $oP6=_UIA_getObjectByFindAll($oP7, "Title:=Setlib;controltype:=UIA_WindowControlTypeId;class:=LSPANEL", $treescope_children)  
_UIA_Action($oP6,"setfocus")
Local $oP5=_UIA_getObjectByFindAll($oP6, "Title:=Setlib;controltype:=UIA_WindowControlTypeId;class:=LSPANEL", $treescope_children)  
_UIA_Action($oP5,"setfocus")
Local $oP4=_UIA_getObjectByFindAll($oP5, "Title:=Setlib;controltype:=UIA_WindowControlTypeId;class:=LSPANEL", $treescope_children)  
_UIA_Action($oP4,"setfocus")
Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=#32770", $treescope_children)   
_UIA_Action($oP3,"setfocus")
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=Select Library List to Copy;controltype:=UIA_ListControlTypeId;class:=LSLISTBOX", $treescope_children) 
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_ListControlTypeId;class:=LSLISTSUB", $treescope_children)    
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_ListControlTypeId;class:=LSLISTLIST", $treescope_children)   
_UIA_Action($oP0,"setfocus")
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_ListItemControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=;ControlType:=UIA_ListItemControlTypeId", $treescope_subtree)
_UIA_action($oUIElement,"click")


*** Detailed properties of the highlighted element ***
UIA_iaccessiblechildId:= <1>
UIA_handle:= <0>
UIA_RuntimeId:= <42;3672536;2;-2147483647;3672536;-4;1>
UIA_BoundingRectangle:= <0;1;82524432;536870911>
; Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

I've attempted to use the Java Spy, but it doesn't see anything on that application. I confirmed it worked with an actual java application.

 

@larsj

Thanks for the tip on inspect.exe - I didn't realize that was an option. Here are the results, which seem to be promising.

Capture.thumb.JPG.4d6cd9f5e5a67ce2437c05Capture.thumb.JPG.a78569d5a51b5f46d52c0c

From this information and your understanding - is there any way to use AutoIt to accomplish this? Or will I most likely need to look into a Java solution for MSAA(Which so far has not been very fruitful).

I'm not really sure where to go from here if it is possible. None of my searching threads here has really pulled anything on how to use MSAA if you can. Anything to get me started would be greatly appreciated. As always, thank you so much for your help and if you need any more info please let me know!

Share this post


Link to post
Share on other sites

As inspect.exe gives you the result / value you are looking for you do not have to go to a java solution.

But I do not have an out of the box solution as I expected that the simplespy should give the same result as inspect.exe

  • option 1 fix simple spy or based on the standard code as given by simplespy get your information with the treewalker (see examples1-3)
  • option 2 more difficult but I assume 1 can work although if you see the data in inspect.exe with MSAA option turned on than this is the way to go
    If you read the thread from start to end you have all the pieces of IUIAutomation, IAccessible, IAccessible2, ISimpledomand in that there will be a name value with the right content.
    This is the area where LarsJ provided the IAccessible ... stuff (they are also in the zip of the first post)
    https://www.autoitscript.com/forum/topic/153520-iuiautomation-ms-framework-automate-chrome-ff-ie/?do=findComment&comment=1156373

 

Share this post


Link to post
Share on other sites

Thanks for getting back to me. I thought I would go with option 2 first as it seems like the most promising, even if it is more difficult. I have all of the files in place and have searched the thread, but I am getting issues with the code you linked me to.

"C:\Program Files (x86)\AutoIt3\Include\AutoItConstants.au3"(53,40) : error: $OBJID_WINDOW previously declared as a 'Const'.
Global Const $OBJID_WINDOW = 0x00000000
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
"C:\Program Files (x86)\AutoIt3\Include\AutoItConstants.au3"(54,42) : error: $OBJID_TITLEBAR previously declared as a 'Const'.
Global Const $OBJID_TITLEBAR = 0xFFFFFFFE
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
"C:\Program Files (x86)\AutoIt3\Include\AutoItConstants.au3"(55,42) : error: $OBJID_SIZEGRIP previously declared as a 'Const'.
Global Const $OBJID_SIZEGRIP = 0xFFFFFFF9
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
"C:\Program Files (x86)\AutoIt3\Include\AutoItConstants.au3"(56,39) : error: $OBJID_CARET previously declared as a 'Const'.
Global Const $OBJID_CARET = 0xFFFFFFF8
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
"C:\Program Files (x86)\AutoIt3\Include\AutoItConstants.au3"(57,40) : error: $OBJID_CURSOR previously declared as a 'Const'.
Global Const $OBJID_CURSOR = 0xFFFFFFF7
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
"C:\Program Files (x86)\AutoIt3\Include\AutoItConstants.au3"(58,39) : error: $OBJID_ALERT previously declared as a 'Const'.
Global Const $OBJID_ALERT = 0xFFFFFFF6
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
"C:\Program Files (x86)\AutoIt3\Include\AutoItConstants.au3"(59,39) : error: $OBJID_SOUND previously declared as a 'Const'.
Global Const $OBJID_SOUND = 0xFFFFFFF5
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
C:\Users\Keven.Reed\Desktop\iAccessible1.au3 - 7 error(s), 0 warning(s)

I don't see where it is referencing these aside from in the Constants file. I was trying to search around AutoIt and Google and couldn't find anything about these specific variables. Is this something you have faced or that you might know how to get around?

Thanks!

 

EDIT: Also - to go ahead and grab you while you're here - I am guessing from what I've read that this will return a way of identifying the elements in the list with MSAA. I am assuming that next I will be able to use the UIA commands you've written to interact with this list in AutoIt? Sorry if I am confused on something, but wanted to go ahead and check.

Edited by automaterk

Share this post


Link to post
Share on other sites

you should get a better text editor for searching text in au3 files in directories

  • msaccessibility.au3
  • msactiveaccessibility.au3

you can uncomment it there as it now seems to be in AutoItConstants nowadays.

I did not integrate (yet) anything from MSAA into the UIA framework commands.  so no UIA commands as in the UIAWrappers will not work but the plain CUIAutomation stuff will work also with MSAA, study the example from LarsJ and hopefully you grasp the concept.

 

Share this post


Link to post
Share on other sites

automaterk, This script based on MSAA-code prints information in SciTE console about all elements it can find in a window. You have to add a line in top of script to get window handle. Let's see whether the code can find the listbox elements.

#include "CUIAutomation2.au3"
#include "MSAccessibility.au3"

Opt( "MustDeclareVars", 1 )

Example()


Func Example()

  ; Get window handle
  ;Local $hWindow = WinGetHandle( "[CLASS:Chrome_WidgetWin_1]", "" ) ; Chrome            ; http://tweepi.com/auth/login
  ;Local $hWindow = WinGetHandle( "[CLASS:MozillaWindowClass]", "" ) ; Mozilla Firefox
  ;Local $hWindow = WinGetHandle( "[CLASS:IEFrame]" )                ; Internet Explorer
  ;Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]" )          ; Windows Explorer, Vista, 7, 8
  ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]" )          ; Windows Explorer, Windows XP
  Local $hWindow = WinGetHandle( "Calculator" )                     ; Calculator
  ;Local $hWindow = ControlGetHandle("[CLASS:Notepad]", "", "Edit1") ; Notepad
  ;Local $hWindow = 0x000403DA                                       ; Chrome            ; http://tweepi.com/auth/login
  If Not $hWindow Then Return ConsoleWrite( "Window handle ERR" & @CRLF )
  ConsoleWrite( "Window handle OK" & @CRLF )

  ; Get Window object
  Local $pWindow, $oWindow
  AccessibleObjectFromWindow( $hWindow, $OBJID_CLIENT, $tIID_IAccessible, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IAccessible, $dtagIAccessible )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Window object ERR" & @CRLF )
  ConsoleWrite( "Window object OK" & @CRLF )

  ; Window name
  Local $sName
  ConsoleWrite( @CRLF & "Window:" )
  ConsoleWrite( @CRLF & "-------" & @CRLF )
  PrintElementInfo( $oWindow, $CHILDID_SELF, "" )

  ; Get children
  ConsoleWrite( "Children:" & @CRLF )
  ConsoleWrite( "---------" & @CRLF )
  WalkTreeWithAccessibleChildren( $pWindow, 0 )

EndFunc


Func WalkTreeWithAccessibleChildren( $pAcc, $iLevel, $iLevels = 0 )

  If $iLevels And $iLevel = $iLevels Then Return

  ; Create object
  Local $oAcc = ObjCreateInterface( $pAcc, $sIID_IAccessible, $dtagIAccessible )
  If Not IsObj( $oAcc ) Then Return
  $oAcc.AddRef()

  Local $iChildCount, $iReturnCount, $tVarChildren

  ; Get children
  If $oAcc.get_accChildCount( $iChildCount ) Or Not $iChildCount Then Return
  If AccessibleChildren( $pAcc, 0, $iChildCount, $tVarChildren, $iReturnCount ) Then Return

  ; Indentation
  Local $sIndent = ""
  For $i = 0 To $iLevel - 1
    $sIndent &= "    "
  Next

  Local $hWnd
  WindowFromAccessibleObject( $pAcc, $hWnd )
  ConsoleWrite( $sIndent & "Window  = " & $hWnd & @CRLF & @CRLF )

  Local $vt, $pChildObj, $oChildObj, $iChildElem

  ; For each child
  For $i = 1 To $iReturnCount

    ; $tVarChildren is an array of VARIANTs with information about the children
    $vt = BitAND( DllStructGetData( $tVarChildren, $i, 1 ), 0xFFFF )

    If $vt = $VT_DISPATCH Then

      ; Child object
      $pChildObj = DllStructGetData( $tVarChildren, $i, 3 )
      $oChildObj = ObjCreateInterface( $pChildObj, $sIID_IAccessible, $dtagIAccessible )
      If IsObj( $oChildObj ) Then
        PrintElementInfo( $oChildObj, $CHILDID_SELF, $sIndent )
        WalkTreeWithAccessibleChildren( $pChildObj, $iLevel + 1, $iLevels )
      EndIf

    Else ; $vt = $VT_I4

      ; Child element
      $iChildElem = DllStructGetData( $tVarChildren, $i, 3 )
      PrintElementInfo( $oAcc, $iChildElem, $sIndent )

    EndIf

  Next

EndFunc

You need CUIAutomation2.au3 and MSAccessibility.au3.

If you get errors on double-defined constants, comment out the constants in MSAccessibility.au3.

Edited by LarsJ

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • By nooneclose
      Good morning, all. I am having a very hard time figuring out why my code "breaks" every second (and every time after) iteration in the loop. 
      I am trying to automate the process of filling out a form for work. I need to find and send data to seven fields and get data from one field. 
      Here are a couple of pictures of the fields I am interacting with. The fields are on the same forum/page. (the green boxed ones)
       


      Here is my code so far. 
       
      #AutoIt3Wrapper_Au3Check_Parameters=-q -d -w 1 -w 2 -w 3 -w- 4 -w 5 -w 6 -w- 7 ;Allows for the combining of UDFs by disabling error messages. (I get error messages for "problems" that dont actually exsist) #include <AD.au3> ;Allows Active Directory functions #include <Array.au3> ;Allows special Array functions #include <Constants.au3> ;Allows constants to be used #include <Date.au3> ;Allows Date and Time functions #include <DateTimeConstants.au3> ;Allows Date and Time constant functions #include "DTC.au3" ;Allows Custom Date and Time functions #include <IE.au3> ;Allows Internet Explorer functions #include <Misc.au3> ;Allows the "is pressed" function to be used #include <MsgBoxConstants.au3> ;Allows Message box function to be used #include <OutlookEX.au3> ;Allows the Outlook functions #include <String.au3> ;Allows special string functions #include <StringConstants.au3> ;Allows String constant functions #include <Timers.au3> ;Allows timer functions ;#RequireAdmin ;Requests admin privliges before starting program ; Special UDFs for UIA automation #include <EditConstants.au3> #include <GUIConstantsEx.au3> #include <WindowsConstants.au3> #include <debug.au3> #include <WinAPI.au3> #include "UIAWrappers.au3" #AutoIt3Wrapper_UseX64=N ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os ;******************************************************************************* ; Format and send the vaild E-mail's contents ;******************************************************************************* Func SendEmailData() ConsoleWrite("Action 16: Send E-mail Data to it's proper fields" & " at line: " & @ScriptLineNumber & @CRLF) ;=============================================================================== ; Store the Elements ;=============================================================================== ; Description _UIA_setVar("oDescription","Title:=Description:;controltype:=UIA_EditControlTypeId;class:=") ; Long Description OK Button _UIA_setVar("oLongButton","Title:=Long Description;controltype:=UIA_ImageControlTypeId;class:=") _UIA_setVar("oLongOk","Title:=OK;controltype:=UIA_ButtonControlTypeId;class:=") ; Work Order Number _UIA_setVar("oWorkOrderNum","Title:=Work Order:;controltype:=UIA_EditControlTypeId;class:=") ; Location _UIA_setVar("oLocation","Title:=Location:;controltype:=UIA_EditControlTypeId;class:=") ; Work Type _UIA_setVar("oWorkType","Title:=Work Type:;controltype:=UIA_EditControlTypeId;class:=") ; Reported By Date _UIA_setVar("oDate","Title:=Reported Date:;controltype:=UIA_EditControlTypeId;class:=") ; Reported By Person _UIA_setVar("oCustomer","Title:=Reported By:;controltype:=UIA_EditControlTypeId;class:=") ; Reported By Email _UIA_setVar("oEmail","Title:=Reported By Email:;controltype:=UIA_EditControlTypeId;class:=") ; Reported By Phone _UIA_setVar("oPhone","Title:=Reported By Phone:;controltype:=UIA_EditControlTypeId;class:=") Local $WorkType = "CM" ; Send the stored data to the proper field Local $stringSplitter = ", " Local $conCateStrings = $eDescription & $stringSplitter & $eComments Local $ComStringLen = StringLen($eComments) Local $TrimedDesc = StringTrimLeft($eDescription, 1) ;=============================================================================== ; Check to see if the Description and the Long Description can be concatenated If StringLen($conCateStrings) <= 100 Then Sleep(2000) _UIA_action("oDescription","highlight") Sleep(300) _UIA_Action("oDescription","setfocus") Sleep(300) _UIA_action("oDescription","click") Sleep(500) _UIA_action("oDescription", "setvalue", $conCateStrings) _UIA_action("oDescription","highlight") ; Wipes the long description so no duplication happens $eComments = "" ;=============================================================================== ; Send the Long Description in place of the Description ElseIf $TrimedDesc = "Other" AND $ComStringLen <= 100 Then Sleep(2000) _UIA_action("oDescription","highlight") Sleep(300) _UIA_Action("oDescription","setfocus") Sleep(300) _UIA_action("oDescription","click") Sleep(500) _UIA_action("oDescription", "setvalue", $eComments) _UIA_action("oDescription","highlight") Sleep(2000) ;=============================================================================== ; Send the Long Description in place of the Description and Send the rest of it to the Long Description field ElseIf $TrimedDesc = "Other" AND $ComStringLen >= 100 Then SplitLongDesc() ; Description gets the first 100 Characters $eDescription = $FirstLine ; Long Description gets the rest $eComments = StringMid($sNewString, 101) Sleep(2000) _UIA_action("oDescription","highlight") Sleep(500) _UIA_Action("oDescription","setfocus") Sleep(500) _UIA_action("oDescription","click") Sleep(500) _UIA_action("oDescription", "setvalue", $eDescription) _UIA_action("oDescription","highlight") Sleep(2000) ;=============================================================================== ; Send the Description and some of the Long Description ElseIf StringLen($eDescription) <= 80 AND $eComments <> "" Then SplitLongDesc() ; Description gets the first 100 Characters $eDescription = $FirstLine ; Long Description gets the rest $eComments = StringMid($sNewString, 101) Sleep(2000) _UIA_action("oDescription","highlight") Sleep(300) _UIA_Action("oDescription","setfocus") Sleep(300) _UIA_action("oDescription","click") Sleep(500) _UIA_action("oDescription", "setvalue", $eDescription) _UIA_action("oDescription","highlight") Sleep(2000) ;=============================================================================== ; Send the original Description Else Sleep(2000) _UIA_action("oDescription","highlight") Sleep(300) _UIA_Action("oDescription","setfocus") Sleep(300) _UIA_action("oDescription","click") Sleep(500) _UIA_action("oDescription", "setvalue", $eDescription) _UIA_action("oDescription","highlight") Sleep(2000) EndIf ;=============================================================================== ; Click the long description button Sleep(2000) _UIA_action("oLongButton","highlight") Sleep(300) _UIA_Action("oLongButton","setfocus") Sleep(300) _UIA_action("oLongButton", "click") _UIA_action("oLongButton", "highlight") _UIA_action("oLongButton", "click") Sleep(2000) ;=============================================================================== ; Send the long description Sleep(1000) if StringLen($eComments) >= 250 Then Sleep(500) Send($eComments) Sleep(10000) ElseIf StringLen($eComments) >= 150 Then Sleep(500) Send($eComments) Sleep(6000) ElseIf StringLen($eComments) <= 10 Then Sleep(500) Send($eComments) Sleep(1500) Else Sleep(500) Send($eComments) Sleep(4000) EndIf ;=============================================================================== ; Click the ok button Sleep(2000) _UIA_action("oLongOk","highlight") Sleep(300) _UIA_Action("oLongOk","setfocus") Sleep(300) _UIA_action("oLongOk", "click") _UIA_action("oLongOk", "highlight") Sleep(2000) ;=============================================================================== ; The work order number _UIA_action("oWorkOrderNum","highlight") Sleep(300) _UIA_Action("oWorkOrderNum","setfocus") Sleep(300) _UIA_action("oWorkOrderNum", "getvalue", $WoNum) _UIA_action("oWorkOrderNum","highlight") ;MsgBox("", "Work order number", $WoNum[2]) ;=============================================================================== ; The Location field Sleep(2000) _UIA_action("oLocation","highlight") Sleep(300) _UIA_Action("oLocation","setfocus") Sleep(300) _UIA_action("oLocation", "setvalue", $eLocation) _UIA_action("oLocation","highlight") ;=============================================================================== ; The Work Type field Sleep(2000) _UIA_action("oWorkType","highlight") Sleep(300) _UIA_Action("oWorkType","setfocus") Sleep(300) _UIA_action("oWorkType", "setvalue", $WorkType) _UIA_action("oWorkType","highlight") ;=============================================================================== ; The Reported By Date Sleep(2000) _UIA_action("oDate","highlight") Sleep(300) _UIA_Action("oDate","setfocus") Sleep(300) _UIA_action("oDate", "setvalue", $eSentOn) _UIA_action("oDate","highlight") ;=============================================================================== ; The Requested by field Sleep(2000) _UIA_action("oCustomer","highlight") Sleep(300) _UIA_Action("oCustomer","setfocus") Sleep(300) _UIA_action("oCustomer", "setvalue", $eReportedBy) _UIA_action("oCustomer","highlight") ;=============================================================================== ; The Requested by E-mail Sleep(2000) _UIA_action("oEmail","highlight") Sleep(300) _UIA_Action("oEmail","setfocus") Sleep(300) _UIA_action("oEmail", "setvalue", $eSenderAddress) _UIA_action("oEmail","highlight") ;=============================================================================== ; The Requested by Phone Sleep(2000) _UIA_action("oPhone","highlight") Sleep(300) _UIA_Action("oPhone","setfocus") Sleep(300) _UIA_action("oPhone", "setvalue", $eRoomPhone) _UIA_action("oPhone","highlight") MouseClick("left", 1843, 437, 2) Sleep(1500) Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Sleep(1000) MouseClick("left", 591, 279, 2) Sleep(3000) $lCount += 1 ConsoleWrite("Action 16: Finished" & " at line: " & @ScriptLineNumber & @CRLF) Sleep(2000) EndFunc  
      That is not all the code but that is the function that keeps "breaking" and by "breaking" I mean the program acts like it doesn't know where the description field is after the first iteration.
      My code will just click somewhere in the far top left corner and somehow shrink the browser in the process of finding the description field.  
       
      The code works perfectly the first iteration but after that, it "breaks" (doesn't know where to click/send data to). 
      As always any help is greatly appreciated. 
       
      NOTE: By the time 'Action 16' runs Chrome has already been opened and has navigated to the proper webpage.  I am trying to loop 'Action 16' multiple times. I do not leave that webpage while looping. 
    • By nooneclose
      Good morning, I am having a hard time figuring out how to read or get the valve (628850) from the red squared box in the picture below. 

      Here is what I have tried so far. (see code below. Also, I'm very sorry my code is sloppy.)
       
      #AutoIt3Wrapper_Au3Check_Parameters=-q -d -w 1 -w 2 -w 3 -w- 4 -w 5 -w 6 -w- 7 ;Allows for the combining of UDFs by disabling error messages. (I get error messages for "problems" that dont actually exsist) #include <AD.au3> ;Allows Active Directory functions #include <Array.au3> ;Allows special Array functions #include <Constants.au3> ;Allows constants to be used #include <Date.au3> ;Allows Date and Time functions #include <DateTimeConstants.au3> ;Allows Date and Time constant functions #include "DTC.au3" ;Allows Custom Date and Time functions #include <IE.au3> ;Allows Internet Explorer functions #include <Misc.au3> ;Allows the "is pressed" function to be used #include <MsgBoxConstants.au3> ;Allows Message box function to be used #include <OutlookEX.au3> ;Allows the Outlook functions #include <String.au3> ;Allows special string functions #include <StringConstants.au3> ;Allows String constant functions #include <Timers.au3> ;Allows timer functions ;#RequireAdmin ;Requests admin privliges before starting program #include <EditConstants.au3> #include <GUIConstantsEx.au3> #include <WindowsConstants.au3> #include <debug.au3> #include <WinAPI.au3> #include "UIAWrappers.au3" #AutoIt3Wrapper_UseX64=N ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os ; Sets the script to accept coordinates off the program window and not the screen AutoItSetOption('MouseCoordMode', 0) ; Defines the time it takes to send text to the software/program AutoItSetOption('SendKeyDelay', 10) ;******************************************************************************* ; Close the script if "End" is pressed ;******************************************************************************* Local $oMyError = ObjEvent("AutoIt.Error","MyErrFunc") HotKeySet("{END}", "Terminate") Func Terminate() MsgBox($MB_ICONINFORMATION, "DEATH", "AI Stoped!", 5) Exit EndFunc ;******************************************************************************* ; Script has started ;******************************************************************************* Sleep(2000) MsgBox($MB_ICONINFORMATION, "WO_AI", "AI Started!", 5) Sleep(1500) ;******************************************************************************* ; Main functions of the program ;******************************************************************************* test() ;=============================================================================== ; Function Definitions ;=============================================================================== Func test() MsgBox("", "TESTing", "Timer Started") Sleep(15000) MsgBox("", "TESTing", "timer Finished") ;~ *** Standard code maintainable *** AutoItSetOption("MustDeclareVars", 1) _UIA_setVar("oP1","Title:=Work Order Tracking - Google Chrome;controltype:=UIA_PaneControlTypeId;class:=Chrome_WidgetWin_1") ;Work Order Tracking - Google Chrome _UIA_setVar("oP2","Title:=Work Order Tracking;controltype:=UIA_DocumentControlTypeId;class:=Chrome_RenderWidgetHostHWND") ;Work Order Tracking _UIA_setVar("oP3","Title:=  Work Order Tracking;controltype:=UIA_CustomControlTypeId;class:=") ;  Work Order Tracking _UIA_setVar("oP4","Title:=main form;controltype:=UIA_GroupControlTypeId;class:=") ;main form _UIA_setVar("oP5","Title:=;controltype:=UIA_CustomControlTypeId;class:=") ; _UIA_setVar("oP6","Title:=View Work Details;controltype:=UIA_CustomControlTypeId;class:=") ;View Work Details _UIA_setVar("oP7","Title:=;controltype:=UIA_CustomControlTypeId;class:=") ; _UIA_setVar("oP8","Title:=Location: Location description Long Description Include Children? Include Children: unchecked Include Ancestors? Include Ancestors: unchecked Include History? Include History: unchecked Work Scope Minimize Work Scope Section Minimize Work Scope Section Work Scope Work Close Filter CTRL+Z Close Filter CTRL+Z Filter Filter Table Filter Table Next Row [CTRL+Down Arrow] Next Row [CTRL+Down Arrow] 1 - 4 of 4 Download Hide Table: Work Hide Table: Work Work Work Table Button Group;controltype:=UIA_CustomControlTypeId;class:=") ;Location: Location description Long Description Include Children? Include Children: unchecked Include Ancestors? Include Ancestors: unchecked Include History? Include History: unchecked Work Scope Minimize Work Scope Section Minimize Work Scope Section Work Scope Work Close Filter CTRL+Z Close Filter CTRL+Z Filter Filter Table Filter Table Next Row [CTRL+Down Arrow] Next Row [CTRL+Down Arrow] 1 - 4 of 4 Download Hide Table: Work Hide Table: Work Work Work Table Button Group _UIA_setVar("oP9","Title:=;controltype:=UIA_CustomControlTypeId;class:=") ; _UIA_setVar("oP10","Title:=Work;controltype:=UIA_TableControlTypeId;class:=") ;Work _UIA_setVar("oP11","Title:=;controltype:=UIA_CustomControlTypeId;class:=") ; _UIA_setVar("oP12","Title:=Record Detail Menu;controltype:=UIA_DataItemControlTypeId;class:=") ;Record Detail Menu ; Clicks on the main record field _UIA_setVar("oUIElement","Title:=Record ;controltype:=UIA_EditControlTypeId;class:=") ;ControlType:=UIA_EditControlTypeId;classname:=") ; Trying to click on / get value from the 3rd from top record field (this is me trying to use the id to access the record box) Local $otest =_UIA_getObjectByFindAll("Record.mainwindow", "id:=ma9038468_tdrow_[C\:1]_txt-tb[R\:3].;ControlType:=UIA_DataItemControlTypeId", $treescope_subtree) ;~ Actions split away from logical/technical definition above can come from configfiles ;~_UIA_Action("oP1","highlight") _UIA_Action("oP1","setfocus") ;~_UIA_Action("oP2","highlight") _UIA_Action("oP2","setfocus") ;~_UIA_Action("oP3","highlight") _UIA_Action("oP3","setfocus") ;~_UIA_Action("oP4","highlight") _UIA_Action("oP4","setfocus") ;~_UIA_Action("oP5","highlight") _UIA_Action("oP5","setfocus") ;~_UIA_Action("oP6","highlight") _UIA_Action("oP6","setfocus") ;~_UIA_Action("oP7","highlight") _UIA_Action("oP7","setfocus") ;~_UIA_Action("oP8","highlight") _UIA_Action("oP8","setfocus") ;~_UIA_Action("oP9","highlight") _UIA_Action("oP9","setfocus") ;~_UIA_Action("oP10","highlight") _UIA_Action("oP10","setfocus") ;~_UIA_Action("oP11","highlight") _UIA_Action("oP11","setfocus") ;~_UIA_Action("oP12","highlight") _UIA_Action("oP12","setfocus") _UIA_action("oUIElement","highlight") _UIA_Action("oUIElement","setfocus") _UIA_action("oUIElement","click") Sleep(2500) _UIA_action($otest,"highlight") _UIA_Action($otest,"setfocus") _UIA_action($otest,"click") MsgBox("", "End of program", "death") EndFunc I'm using $otest to try and click on the box, through the id that chrome gives it, with the UIA functions. I am kind of completely new to the UIA UDF but I do find it capable of doing a lot of cool things.   
      as always any help would be greatly appreciated. I know how hard you people hard. Thank you for taking the time to help us. 
    • By Valnurat
      I have this code and I want to control the size of the picture that is being showing on the GUI not matter how big the source of the picture is.
      To draw the picture you just add the x and y position, but I would like a end position too.
      #include <GDIPlus.au3> #include <GUIConstantsEx.au3> #include <AD.au3> Global $sUsername = 'shortname' Example() Func Example() _GDIPlus_Startup() ;initialize GDI+ Local Const $iWidth = 300, $iHeight = 300, $iBgColor = 0xFFE8FF ;$iBgColor format RRGGBB Local $hGUI = GUICreate("GDI+ Example (" & @ScriptName & ")", $iWidth, $iHeight) ;create a test GUI GUISetBkColor($iBgColor, $hGUI) ;set GUI background color GUISetState(@SW_SHOW) Local $hGraphics = _GDIPlus_GraphicsCreateFromHWND($hGUI) ;create a graphics object from a window handle Local $hBitmap = _GDIPlus_BitmapCreateFromMemory(_ADO_USERIMAGE($sUsername)) ;load binary saved GIF image and convert it to GDI+ bitmap format if Not @error Then Local $iW = _GDIPlus_ImageGetWidth($hBitmap), $iH = _GDIPlus_ImageGetHeight($hBitmap) MsgBox(0, $iW, $iH) _GDIPlus_GraphicsDrawImage($hGraphics, $hBitmap, 15, 20) ;display image in GUI EndIf Do Until GUIGetMsg() = $GUI_EVENT_CLOSE ;cleanup GDI+ resources _GDIPlus_BitmapDispose($hBitmap) _GDIPlus_GraphicsDispose($hGraphics) _GDIPlus_Shutdown() GUIDelete($hGUI) EndFunc ;==>Example Func _ADO_USERIMAGE($sUsename) _AD_Open() Local $oADO_USERIMAGE = _AD_GetObjectAttribute($sUseName, 'thumbnailPhoto') If @error Then Exit _AD_Close() Return $oADO_USERIMAGE EndFunc  
    • By junichironakashima
      Im creating a code that will work in this sequence:
      1. Copy the text (question) in one atea of the screen
      2. Catch the 2 strings (number)
      3. Multiply the 2 strings ( $1*$2)
      4. Click the next area to put the answer
      5. Paste the answer
       
      This is my code
       
      MouseClick($MOUSE_CLICK_LEFT, 479, 802, 3, 1) ;Clicking all of the text
      Send("^c") 
      $x = StringRegExpReplace(ClipGet(), 'What is (\d*) x (\d*) \?$', "$1*$2")
      MouseClick($MOUSE_CLICK_LEFT, 480, 844, 1, 1)
      ClipPut($x)
      Send("^v")
       
      However the output is this
      $1*$2
       
      How can I make it solve itself? Because I tried this code:
      MouseClick($MOUSE_CLICK_LEFT, 479, 802, 3, 1) ;Clicking all of the text
      Send("^c")
      MouseClick($MOUSE_CLICK_LEFT, 480, 844, 1, 1) $x = Execute(StringRegExpReplace(ClipGet(), 'What is (\d*) x (\d*) \?$', "$1*$2"))
      ClipPut($x)
      Send("^v")
      Output is just blank text

    • By junichironakashima
      I have a windows 10 pc and Im trying to create an automatic solving in calculator with a simple worded question (example: What is 45 x 53 ?). thou I cant make the select...Case...EndSelect statement work, here is my code
      #include <AutoItConstants.au3> HotKeySet("{F4}", "ExitProg") Func ExitProg() Exit 0 EndFunc MouseClick($MOUSE_CLICK_LEFT, 417, 659, 2, 1) Send("^c") Func valData() $Chek = "What " If ClipGet() == $Chek Then Check() Else Do MouseClick($MOUSE_CLICK_LEFT, 417, 659, 2, 3) Send("^c") Sleep(500) Until ClipGet() == $Chek EndIf EndFunc Func Check() $Chek2 = "?" c1() c2() c3() c4() c5() c6() c7() Select Case c1() = $Chek2 ;two MouseClick($MOUSE_CLICK_LEFT, 453, 645, 2, 1) Send("^c") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 1347, 197, 1, 1) Send("^v") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 474, 645, 2, 1) Send("^c") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 1347, 197, 1, 1) Send("*") Send("^v") Send("{NUMPADENTER}") Case c2() = $Chek2 ;three MouseClick($MOUSE_CLICK_LEFT, 453, 645, 2, 1) Send("^c") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 1347, 197, 1, 1) Send("^v") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 482, 645, 2, 1) Send("^c") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 1347, 197, 1, 1) Send("*") Send("^v") Send("{NUMPADENTER}") Case c3() = $Chek2 ;four MouseClick($MOUSE_CLICK_LEFT, 453, 645, 2, 1) Send("^c") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 1347, 197, 1, 1) Send("^v") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 487, 645, 2, 1) Send("^c") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 1347, 197, 1, 1) Send("*") Send("^v") Send("{NUMPADENTER}") Case c4() = $Chek2 ;five MouseClick($MOUSE_CLICK_LEFT, 453, 645, 2, 1) Send("^c") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 1347, 197, 1, 1) Send("^v") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 495, 645, 2, 1) Send("^c") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 1347, 197, 1, 1) Send("*") Send("^v") Send("{NUMPADENTER}") Case c5() = $Chek2 ;six MouseClick($MOUSE_CLICK_LEFT, 453, 645, 2, 1) Send("^c") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 1347, 197, 1, 1) Send("^v") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 495, 645, 2, 1) Send("^c") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 1347, 197, 1, 1) Send("*") Send("^v") Send("{NUMPADENTER}") Case c6() = $Chek2 ;seven MouseClick($MOUSE_CLICK_LEFT, 453, 645, 2, 1) Send("^c") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 1347, 197, 1, 1) Send("^v") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 509, 645, 2, 1) Send("^c") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 1347, 197, 1, 1) Send("*") Send("^v") Send("{NUMPADENTER}") Case c7() = $Chek2 ;eight MouseClick($MOUSE_CLICK_LEFT, 453, 645, 2, 1) Send("^c") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 1347, 197, 1, 1) Send("^v") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 509, 645, 2, 1) Send("^c") Sleep(150) MouseClick($MOUSE_CLICK_LEFT, 1347, 197, 1, 1) Send("*") Send("^v") Send("{NUMPADENTER}") Case Else Exit EndSelect EndFunc Func c1() MouseClick($MOUSE_CLICK_LEFT, 485, 643, 2, 1) Send("^c") EndFunc Func c2() MouseClick($MOUSE_CLICK_LEFT, 493, 644, 2, 1) Send("^c") EndFunc Func c3() MouseClick($MOUSE_CLICK_LEFT, 498, 645, 2, 1) Send("^c") EndFunc Func c4() MouseClick($MOUSE_CLICK_LEFT, 508, 647, 2, 1) Send("^c") EndFunc Func c5() MouseClick($MOUSE_CLICK_LEFT, 514, 645, 2, 1) Send("^c") EndFunc Func c6() MouseClick($MOUSE_CLICK_LEFT, 523, 645, 2, 1) Send("^c") EndFunc Func c7() MouseClick($MOUSE_CLICK_LEFT, 530, 645, 2, 1) Send("^c") EndFunc valData() MouseClick($MOUSE_CLICK_LEFT, 1349, 196, 1, 1) ;clicking the answer in calc Send("^c") Send("{DEL}") MouseClick($MOUSE_CLICK_LEFT, 499, 706, 1, 1) Send("^v") Exit  
×
×
  • Create New...