Jump to content
Sign in to follow this  
plastix

Chrome Automation of existing instance

Recommended Posts

Hi all

Can IUIAutomation automate existing instances of Chrome to find objects (by ID, for example) within iframes and set text of inputs within them ?  I need to basically poll existing Chrome instances regularly to see if a login form is loaded (which will itself be loaded in an iframe potentially within any number of tabs/instances). Chrome will have accessibility 'on', but there are restrictions that won't permit the installation of any non-mainstream extensions etc (othewise would have looked at the _Chrome_UDF). I started dabbling with Selenium (via Selenium Wrapper ) and it worked well spawning an instance of Chrome browser etc - but I later realised that it can't attach to an existing instance of Chrome... I've looked at Junkew's IUIAutomation framework and could see might be feasible to manipulate controls by click etc - but not necessarily at a DOM / DOM tree level...

Anyone got any experience / success with this specific issue ? TIA ;)

 

Share this post


Link to post
Share on other sites

If you can get the control of the chrome instance I don't see why you couldn't do this. You could also make the shortcut for chrome start a script that starts chrome.


Get Scite to add a popup when you use a 3rd party UDF -> http://www.autoitscript.com/autoit3/scite/docs/SciTE4AutoIt3/user-calltip-manager.html

Share this post


Link to post
Share on other sites

No problem you just can attach to running instance with iuiautomation. With tab accessibility of chrome you can see the default exposed tree. By entering javascript lines thru the addressbar the sky will be the limit on what you can see in chrome but also other browsers.

Edited by junkew

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Similar Content

    • By JLogan3o13
      There are a number of posts on the forum regarding use of Selenium in AutoIt. I recently had a go at using the PowerShell Selenium module, and was amazed at how easy it is. Thought I would post an example here; if anyone is interested this could probably be incorporated into AutoIt code pretty easily. 
      Pre-Req - The true star of this script is the ChroPath extension, available for Edge, Chrome and FireFox. With it installed, you just click on the element, select Inspect, and then ChroPath generates the XPath to the element for you. Here is an example based on a simple form I created on one of my sites.
      $myForm = Start-SeChrome -StartURL "http://logancomputerser.com/Appointment.html" -Maximized $firstName = Find-SeElement -Driver $myForm -Timeout 30 -XPath "//input[@id='formElement_First']" $lastName = Find-SeElement -Driver $myForm -Timeout 30 -XPath "//input[@id='formElement_Last']" $address = Find-SeElement -Driver $myForm -Timeout 30 -XPath "//input[@id='formElement_Street1']" $city = Find-SeElement -Driver $myForm -Timeout 30 -XPath "//input[@id='formElement_City']" $zip = Find-SeElement -Driver $myForm -Timeout 30 -XPath "//input[@id='formElement_Zip']" $state = Find-SeElement -Driver $myForm -Timeout 30 -XPath "//select[@id='formElement_State']" $phoneDay = Find-SeElement -Driver $myForm -Timeout 30 -XPath "//input[@id='formElement_DaytimePhone']" $phoneNight = Find-SeElement -Driver $myForm -Timeout 30 -XPath "//input[@id='formElement_EveningPhone']" $email = Find-SeElement -Driver $myForm -Timeout 30 -XPath "//input[@id='formElement_liamE']" $user = Find-SeElement -Driver $myForm -Timeout 30 -XPath "//input[@id='formElement_48564']" $pw = Find-SeElement -Driver $myForm -Timeout 30 -XPath "//input[@id='formElement_f403c']" $submit = Find-SeElement -Driver $myForm -Timeout 30 -XPath "//input[@id='wstForm_Contact_Submit']" $reset = Find-SeElement -Driver $myForm -Timeout 30 -XPath "//input[@id='wstForm_Contact_Reset']" Send-SeKeys -Element $firstName -Keys "Joe" Send-SeKeys -Element $lastName -Keys "Blow" Send-SeKeys -Element $address -Keys "111 S. Main St." Send-SeKeys -Element $city -Keys "AnyCity" Send-SeKeys -Element $zip -Keys "90210" Send-SeKeys -Element $state -Keys "CA" Send-SeKeys -Element $phoneDay -Keys "555.867.5309" Send-SeKeys -Element $phoneNight -Keys "555.888.1212" Send-SeKeys -Element $email -Keys "1Adam12@gmail.com" Send-SeKeys -Element $user -Keys "JBlow" Send-SeKeys -Element $pw -Keys "MyPassword" Start-Sleep 1 Invoke-SeClick -Element $submit Stop-SeDriver -Driver $myForm  
      As mentioned, this is just another way to skin the cat, but I found it a pretty fast way to initiate some easy testing in Selenium, and have used it a couple of times in projects now, both straight through PowerShell and wrapped in AutoIt.
    • By nooneclose
      Good morning, all. I am having a very hard time figuring out why my code "breaks" every second (and every time after) iteration in the loop. 
      I am trying to automate the process of filling out a form for work. I need to find and send data to seven fields and get data from one field. 
      Here are a couple of pictures of the fields I am interacting with. The fields are on the same forum/page. (the green boxed ones)
       


      Here is my code so far. 
       
      #AutoIt3Wrapper_Au3Check_Parameters=-q -d -w 1 -w 2 -w 3 -w- 4 -w 5 -w 6 -w- 7 ;Allows for the combining of UDFs by disabling error messages. (I get error messages for "problems" that dont actually exsist) #include <AD.au3> ;Allows Active Directory functions #include <Array.au3> ;Allows special Array functions #include <Constants.au3> ;Allows constants to be used #include <Date.au3> ;Allows Date and Time functions #include <DateTimeConstants.au3> ;Allows Date and Time constant functions #include "DTC.au3" ;Allows Custom Date and Time functions #include <IE.au3> ;Allows Internet Explorer functions #include <Misc.au3> ;Allows the "is pressed" function to be used #include <MsgBoxConstants.au3> ;Allows Message box function to be used #include <OutlookEX.au3> ;Allows the Outlook functions #include <String.au3> ;Allows special string functions #include <StringConstants.au3> ;Allows String constant functions #include <Timers.au3> ;Allows timer functions ;#RequireAdmin ;Requests admin privliges before starting program ; Special UDFs for UIA automation #include <EditConstants.au3> #include <GUIConstantsEx.au3> #include <WindowsConstants.au3> #include <debug.au3> #include <WinAPI.au3> #include "UIAWrappers.au3" #AutoIt3Wrapper_UseX64=N ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os ;******************************************************************************* ; Format and send the vaild E-mail's contents ;******************************************************************************* Func SendEmailData() ConsoleWrite("Action 16: Send E-mail Data to it's proper fields" & " at line: " & @ScriptLineNumber & @CRLF) ;=============================================================================== ; Store the Elements ;=============================================================================== ; Description _UIA_setVar("oDescription","Title:=Description:;controltype:=UIA_EditControlTypeId;class:=") ; Long Description OK Button _UIA_setVar("oLongButton","Title:=Long Description;controltype:=UIA_ImageControlTypeId;class:=") _UIA_setVar("oLongOk","Title:=OK;controltype:=UIA_ButtonControlTypeId;class:=") ; Work Order Number _UIA_setVar("oWorkOrderNum","Title:=Work Order:;controltype:=UIA_EditControlTypeId;class:=") ; Location _UIA_setVar("oLocation","Title:=Location:;controltype:=UIA_EditControlTypeId;class:=") ; Work Type _UIA_setVar("oWorkType","Title:=Work Type:;controltype:=UIA_EditControlTypeId;class:=") ; Reported By Date _UIA_setVar("oDate","Title:=Reported Date:;controltype:=UIA_EditControlTypeId;class:=") ; Reported By Person _UIA_setVar("oCustomer","Title:=Reported By:;controltype:=UIA_EditControlTypeId;class:=") ; Reported By Email _UIA_setVar("oEmail","Title:=Reported By Email:;controltype:=UIA_EditControlTypeId;class:=") ; Reported By Phone _UIA_setVar("oPhone","Title:=Reported By Phone:;controltype:=UIA_EditControlTypeId;class:=") Local $WorkType = "CM" ; Send the stored data to the proper field Local $stringSplitter = ", " Local $conCateStrings = $eDescription & $stringSplitter & $eComments Local $ComStringLen = StringLen($eComments) Local $TrimedDesc = StringTrimLeft($eDescription, 1) ;=============================================================================== ; Check to see if the Description and the Long Description can be concatenated If StringLen($conCateStrings) <= 100 Then Sleep(2000) _UIA_action("oDescription","highlight") Sleep(300) _UIA_Action("oDescription","setfocus") Sleep(300) _UIA_action("oDescription","click") Sleep(500) _UIA_action("oDescription", "setvalue", $conCateStrings) _UIA_action("oDescription","highlight") ; Wipes the long description so no duplication happens $eComments = "" ;=============================================================================== ; Send the Long Description in place of the Description ElseIf $TrimedDesc = "Other" AND $ComStringLen <= 100 Then Sleep(2000) _UIA_action("oDescription","highlight") Sleep(300) _UIA_Action("oDescription","setfocus") Sleep(300) _UIA_action("oDescription","click") Sleep(500) _UIA_action("oDescription", "setvalue", $eComments) _UIA_action("oDescription","highlight") Sleep(2000) ;=============================================================================== ; Send the Long Description in place of the Description and Send the rest of it to the Long Description field ElseIf $TrimedDesc = "Other" AND $ComStringLen >= 100 Then SplitLongDesc() ; Description gets the first 100 Characters $eDescription = $FirstLine ; Long Description gets the rest $eComments = StringMid($sNewString, 101) Sleep(2000) _UIA_action("oDescription","highlight") Sleep(500) _UIA_Action("oDescription","setfocus") Sleep(500) _UIA_action("oDescription","click") Sleep(500) _UIA_action("oDescription", "setvalue", $eDescription) _UIA_action("oDescription","highlight") Sleep(2000) ;=============================================================================== ; Send the Description and some of the Long Description ElseIf StringLen($eDescription) <= 80 AND $eComments <> "" Then SplitLongDesc() ; Description gets the first 100 Characters $eDescription = $FirstLine ; Long Description gets the rest $eComments = StringMid($sNewString, 101) Sleep(2000) _UIA_action("oDescription","highlight") Sleep(300) _UIA_Action("oDescription","setfocus") Sleep(300) _UIA_action("oDescription","click") Sleep(500) _UIA_action("oDescription", "setvalue", $eDescription) _UIA_action("oDescription","highlight") Sleep(2000) ;=============================================================================== ; Send the original Description Else Sleep(2000) _UIA_action("oDescription","highlight") Sleep(300) _UIA_Action("oDescription","setfocus") Sleep(300) _UIA_action("oDescription","click") Sleep(500) _UIA_action("oDescription", "setvalue", $eDescription) _UIA_action("oDescription","highlight") Sleep(2000) EndIf ;=============================================================================== ; Click the long description button Sleep(2000) _UIA_action("oLongButton","highlight") Sleep(300) _UIA_Action("oLongButton","setfocus") Sleep(300) _UIA_action("oLongButton", "click") _UIA_action("oLongButton", "highlight") _UIA_action("oLongButton", "click") Sleep(2000) ;=============================================================================== ; Send the long description Sleep(1000) if StringLen($eComments) >= 250 Then Sleep(500) Send($eComments) Sleep(10000) ElseIf StringLen($eComments) >= 150 Then Sleep(500) Send($eComments) Sleep(6000) ElseIf StringLen($eComments) <= 10 Then Sleep(500) Send($eComments) Sleep(1500) Else Sleep(500) Send($eComments) Sleep(4000) EndIf ;=============================================================================== ; Click the ok button Sleep(2000) _UIA_action("oLongOk","highlight") Sleep(300) _UIA_Action("oLongOk","setfocus") Sleep(300) _UIA_action("oLongOk", "click") _UIA_action("oLongOk", "highlight") Sleep(2000) ;=============================================================================== ; The work order number _UIA_action("oWorkOrderNum","highlight") Sleep(300) _UIA_Action("oWorkOrderNum","setfocus") Sleep(300) _UIA_action("oWorkOrderNum", "getvalue", $WoNum) _UIA_action("oWorkOrderNum","highlight") ;MsgBox("", "Work order number", $WoNum[2]) ;=============================================================================== ; The Location field Sleep(2000) _UIA_action("oLocation","highlight") Sleep(300) _UIA_Action("oLocation","setfocus") Sleep(300) _UIA_action("oLocation", "setvalue", $eLocation) _UIA_action("oLocation","highlight") ;=============================================================================== ; The Work Type field Sleep(2000) _UIA_action("oWorkType","highlight") Sleep(300) _UIA_Action("oWorkType","setfocus") Sleep(300) _UIA_action("oWorkType", "setvalue", $WorkType) _UIA_action("oWorkType","highlight") ;=============================================================================== ; The Reported By Date Sleep(2000) _UIA_action("oDate","highlight") Sleep(300) _UIA_Action("oDate","setfocus") Sleep(300) _UIA_action("oDate", "setvalue", $eSentOn) _UIA_action("oDate","highlight") ;=============================================================================== ; The Requested by field Sleep(2000) _UIA_action("oCustomer","highlight") Sleep(300) _UIA_Action("oCustomer","setfocus") Sleep(300) _UIA_action("oCustomer", "setvalue", $eReportedBy) _UIA_action("oCustomer","highlight") ;=============================================================================== ; The Requested by E-mail Sleep(2000) _UIA_action("oEmail","highlight") Sleep(300) _UIA_Action("oEmail","setfocus") Sleep(300) _UIA_action("oEmail", "setvalue", $eSenderAddress) _UIA_action("oEmail","highlight") ;=============================================================================== ; The Requested by Phone Sleep(2000) _UIA_action("oPhone","highlight") Sleep(300) _UIA_Action("oPhone","setfocus") Sleep(300) _UIA_action("oPhone", "setvalue", $eRoomPhone) _UIA_action("oPhone","highlight") MouseClick("left", 1843, 437, 2) Sleep(1500) Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Send("{UP}") Sleep(1000) MouseClick("left", 591, 279, 2) Sleep(3000) $lCount += 1 ConsoleWrite("Action 16: Finished" & " at line: " & @ScriptLineNumber & @CRLF) Sleep(2000) EndFunc  
      That is not all the code but that is the function that keeps "breaking" and by "breaking" I mean the program acts like it doesn't know where the description field is after the first iteration.
      My code will just click somewhere in the far top left corner and somehow shrink the browser in the process of finding the description field.  
       
      The code works perfectly the first iteration but after that, it "breaks" (doesn't know where to click/send data to). 
      As always any help is greatly appreciated. 
       
      NOTE: By the time 'Action 16' runs Chrome has already been opened and has navigated to the proper webpage.  I am trying to loop 'Action 16' multiple times. I do not leave that webpage while looping. 
    • By nooneclose
      Good morning, I am having a hard time figuring out how to read or get the valve (628850) from the red squared box in the picture below. 

      Here is what I have tried so far. (see code below. Also, I'm very sorry my code is sloppy.)
       
      #AutoIt3Wrapper_Au3Check_Parameters=-q -d -w 1 -w 2 -w 3 -w- 4 -w 5 -w 6 -w- 7 ;Allows for the combining of UDFs by disabling error messages. (I get error messages for "problems" that dont actually exsist) #include <AD.au3> ;Allows Active Directory functions #include <Array.au3> ;Allows special Array functions #include <Constants.au3> ;Allows constants to be used #include <Date.au3> ;Allows Date and Time functions #include <DateTimeConstants.au3> ;Allows Date and Time constant functions #include "DTC.au3" ;Allows Custom Date and Time functions #include <IE.au3> ;Allows Internet Explorer functions #include <Misc.au3> ;Allows the "is pressed" function to be used #include <MsgBoxConstants.au3> ;Allows Message box function to be used #include <OutlookEX.au3> ;Allows the Outlook functions #include <String.au3> ;Allows special string functions #include <StringConstants.au3> ;Allows String constant functions #include <Timers.au3> ;Allows timer functions ;#RequireAdmin ;Requests admin privliges before starting program #include <EditConstants.au3> #include <GUIConstantsEx.au3> #include <WindowsConstants.au3> #include <debug.au3> #include <WinAPI.au3> #include "UIAWrappers.au3" #AutoIt3Wrapper_UseX64=N ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os ; Sets the script to accept coordinates off the program window and not the screen AutoItSetOption('MouseCoordMode', 0) ; Defines the time it takes to send text to the software/program AutoItSetOption('SendKeyDelay', 10) ;******************************************************************************* ; Close the script if "End" is pressed ;******************************************************************************* Local $oMyError = ObjEvent("AutoIt.Error","MyErrFunc") HotKeySet("{END}", "Terminate") Func Terminate() MsgBox($MB_ICONINFORMATION, "DEATH", "AI Stoped!", 5) Exit EndFunc ;******************************************************************************* ; Script has started ;******************************************************************************* Sleep(2000) MsgBox($MB_ICONINFORMATION, "WO_AI", "AI Started!", 5) Sleep(1500) ;******************************************************************************* ; Main functions of the program ;******************************************************************************* test() ;=============================================================================== ; Function Definitions ;=============================================================================== Func test() MsgBox("", "TESTing", "Timer Started") Sleep(15000) MsgBox("", "TESTing", "timer Finished") ;~ *** Standard code maintainable *** AutoItSetOption("MustDeclareVars", 1) _UIA_setVar("oP1","Title:=Work Order Tracking - Google Chrome;controltype:=UIA_PaneControlTypeId;class:=Chrome_WidgetWin_1") ;Work Order Tracking - Google Chrome _UIA_setVar("oP2","Title:=Work Order Tracking;controltype:=UIA_DocumentControlTypeId;class:=Chrome_RenderWidgetHostHWND") ;Work Order Tracking _UIA_setVar("oP3","Title:=  Work Order Tracking;controltype:=UIA_CustomControlTypeId;class:=") ;  Work Order Tracking _UIA_setVar("oP4","Title:=main form;controltype:=UIA_GroupControlTypeId;class:=") ;main form _UIA_setVar("oP5","Title:=;controltype:=UIA_CustomControlTypeId;class:=") ; _UIA_setVar("oP6","Title:=View Work Details;controltype:=UIA_CustomControlTypeId;class:=") ;View Work Details _UIA_setVar("oP7","Title:=;controltype:=UIA_CustomControlTypeId;class:=") ; _UIA_setVar("oP8","Title:=Location: Location description Long Description Include Children? Include Children: unchecked Include Ancestors? Include Ancestors: unchecked Include History? Include History: unchecked Work Scope Minimize Work Scope Section Minimize Work Scope Section Work Scope Work Close Filter CTRL+Z Close Filter CTRL+Z Filter Filter Table Filter Table Next Row [CTRL+Down Arrow] Next Row [CTRL+Down Arrow] 1 - 4 of 4 Download Hide Table: Work Hide Table: Work Work Work Table Button Group;controltype:=UIA_CustomControlTypeId;class:=") ;Location: Location description Long Description Include Children? Include Children: unchecked Include Ancestors? Include Ancestors: unchecked Include History? Include History: unchecked Work Scope Minimize Work Scope Section Minimize Work Scope Section Work Scope Work Close Filter CTRL+Z Close Filter CTRL+Z Filter Filter Table Filter Table Next Row [CTRL+Down Arrow] Next Row [CTRL+Down Arrow] 1 - 4 of 4 Download Hide Table: Work Hide Table: Work Work Work Table Button Group _UIA_setVar("oP9","Title:=;controltype:=UIA_CustomControlTypeId;class:=") ; _UIA_setVar("oP10","Title:=Work;controltype:=UIA_TableControlTypeId;class:=") ;Work _UIA_setVar("oP11","Title:=;controltype:=UIA_CustomControlTypeId;class:=") ; _UIA_setVar("oP12","Title:=Record Detail Menu;controltype:=UIA_DataItemControlTypeId;class:=") ;Record Detail Menu ; Clicks on the main record field _UIA_setVar("oUIElement","Title:=Record ;controltype:=UIA_EditControlTypeId;class:=") ;ControlType:=UIA_EditControlTypeId;classname:=") ; Trying to click on / get value from the 3rd from top record field (this is me trying to use the id to access the record box) Local $otest =_UIA_getObjectByFindAll("Record.mainwindow", "id:=ma9038468_tdrow_[C\:1]_txt-tb[R\:3].;ControlType:=UIA_DataItemControlTypeId", $treescope_subtree) ;~ Actions split away from logical/technical definition above can come from configfiles ;~_UIA_Action("oP1","highlight") _UIA_Action("oP1","setfocus") ;~_UIA_Action("oP2","highlight") _UIA_Action("oP2","setfocus") ;~_UIA_Action("oP3","highlight") _UIA_Action("oP3","setfocus") ;~_UIA_Action("oP4","highlight") _UIA_Action("oP4","setfocus") ;~_UIA_Action("oP5","highlight") _UIA_Action("oP5","setfocus") ;~_UIA_Action("oP6","highlight") _UIA_Action("oP6","setfocus") ;~_UIA_Action("oP7","highlight") _UIA_Action("oP7","setfocus") ;~_UIA_Action("oP8","highlight") _UIA_Action("oP8","setfocus") ;~_UIA_Action("oP9","highlight") _UIA_Action("oP9","setfocus") ;~_UIA_Action("oP10","highlight") _UIA_Action("oP10","setfocus") ;~_UIA_Action("oP11","highlight") _UIA_Action("oP11","setfocus") ;~_UIA_Action("oP12","highlight") _UIA_Action("oP12","setfocus") _UIA_action("oUIElement","highlight") _UIA_Action("oUIElement","setfocus") _UIA_action("oUIElement","click") Sleep(2500) _UIA_action($otest,"highlight") _UIA_Action($otest,"setfocus") _UIA_action($otest,"click") MsgBox("", "End of program", "death") EndFunc I'm using $otest to try and click on the box, through the id that chrome gives it, with the UIA functions. I am kind of completely new to the UIA UDF but I do find it capable of doing a lot of cool things.   
      as always any help would be greatly appreciated. I know how hard you people hard. Thank you for taking the time to help us. 
    • By UGH
      Autoit team are assbags.    They want to hold out that they are soooo much smarter than everyone else and they really don't want to help anyone with their questions.   In that case then why have a website.   Oh i know, they get their kicks out of laughing at everyone. 
       
       
       
       
       
       
       
    • By nooneclose
      I want to check some Excel data against data on a website in Chrome. I use Chrome because the site I use does not function properly in Internet Explorer or Firefox. I know how to do the Excel stuff I just can not figure out how to send to Chrome, let alone check to see if the data matches or not. I am also having trouble finding any help online while searching for Chrome functions for Autoit. I have a Chrome UDF installed but I still can not figure out how to get my code to properly function. (I am not posting code because I am  sure my code isn't right, to begin with)
      As usual, any and all help would be greatly appreciated. 
×
×
  • Create New...