WebDriver UDF - Help & Support (II)

[Danp2]

@svenjatzu Suggest that you join their site because we won't be assisting you with bypassing any website security.

[svenjatzu]

Yes sorry my fault didnt recognise that this can be seen ass an way to automate captchas

but thanks so far : )

[water]

Is there a way to maximize the Firefox window opened by  geckodriver? When I open www.enforcementtracker.com and retrieve the data table I get all 9 columns when the screen is large enough to display all 9 columns in the browser window.
If the screen is too small I only get the rightmost plus the first 7 columns from the left (so column 9 is missing in the result).

To retrieve the data table I use the following script:

expandcollapse popup

#include "wd_core.au3"
#include "wd_helper.au3"

Global $sDesiredCapabilities, $sSession
Global $aDebugLevel[][2] = [["None", $_WD_DEBUG_None], ["Error", $_WD_DEBUG_Error], ["Full", $_WD_DEBUG_Info]]

; Execute browser setup routine for user's browser selection
SetupGecko()

_WD_Startup()
If @error <> $_WD_ERROR_Success Then Exit -1
$sSession = _WD_CreateSession($sDesiredCapabilities)
If @error = $_WD_ERROR_Success Then DemoNavigation()
MsgBox($MB_ICONINFORMATION, "Demo complete!", "Click ok to shutdown the browser and console")
_WD_DeleteSession($sSession)
_WD_Shutdown()
Exit

Func DemoNavigation()
    _WD_Navigate($sSession, "http://www.enforcementtracker.com/")
    Local $aResult = _WD_GetTable($sSession, "//table[@id='penalties']")
    $aElements = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//a[@id='penalties_next'][not(@disabled)]")
    If Not @error then _WD_ElementAction($sSession, $aElements, "click")
    _ArrayDisplay($aResult)
EndFunc   ;==>DemoNavigation

Func SetupGecko()
    _WD_Option('Driver', 'geckodriver.exe')
    _WD_Option('DriverParams', '--log trace')
    _WD_Option('Port', 4444)
    $sDesiredCapabilities = '{"desiredCapabilities":{"javascriptEnabled":true,"nativeEvents":true,"acceptInsecureCerts":true}}'
EndFunc   ;==>SetupGecko

; #FUNCTION# ====================================================================================================================
; Name ..........: _WD_GetTable
; Description ...: Return all elements of a table
; Syntax ........: _WD_GetTable($sSession, $sBaseElement)
; Parameters ....: $sSession     - Session ID from _WDCreateSession
;                  $sBaseElement - XPath of the table to return
; Return values .: Success      - Element ID(s) returned by web driver
;                  Failure      - ""
;                  @ERROR       - $_WD_ERROR_Success
;                               - $_WD_ERROR_Exception
;                               - $_WD_ERROR_NoMatch
;                  @EXTENDED    - WinHTTP status code
; Author ........: danylarson
; Modified ......: water
; Remarks .......:
; Related .......:
; Link ..........: https://www.autoitscript.com/forum/topic/191990-webdriver-udf-w3c-compliant-version-01182020/page/18/?tab=comments#comment-1415164
; Example .......: No
; ===============================================================================================================================
Func _WD_GetTable($sSession, $aElements)
    Local Const $sFuncName = "_WD_GetTable"
    Local $sBaseElement, $iLines, $iColumns, $iRow, $iColumn
    $sBaseElement = $aElements
    $aElements = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, $sBaseElement & "/tbody/tr", "", True)
    $iLines = UBound($aElements)
    $aElements = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, $sBaseElement & "/tbody/tr[1]/td", "", True)
    $iColumns = UBound($aElements)
    Local $aTable[$iLines][$iColumns]
    $aElements = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, $sBaseElement & "/tbody/tr/td", "", True)
    For $i = 0 To UBound($aElements) - 1
        $iRow = Int($i / $iColumns)
        $iColumn = Mod($i, $iColumns)
        $aTable[$iRow][$iColumn] = _WD_ElementAction($sSession, $aElements[$i], 'Text')
    Next
    ReDim $aTable[$iRow+1][$iColumns]
    Return $aTable
EndFunc   ;==>_WD_GetTableContentWater

 

Edited January 25, 2020 by water

[Danp2]

@water Have you looked at using _WD_Window?

[water]

That sounds great! Will have a look. Thanks!

[Maxxx23]

Hello, Is it possible to find out the XY coordinates of an HTML element with WebDriver UDF. I didn't find anything about it, just outdated information from IE UDF.

I would be very grateful for a code snippet. Since I'm a beginner.

[Danp2]

@Maxxx23 There's an example of this in the DemoElements function in wd_demo.au3

[Maxxx23]

Thank you i'm very sorry i didn't see that.

; Get element's coordinates
    $oERect = _WD_ElementAction($sSession, $sElement, 'rect')
    ConsoleWrite("Element Coords = " & $oERect.Item('x') & " / " & $oERect.Item('y') & " / " & $oERect.Item('width') & " / " & $oERect.Item('height') & @CRLF)

[NeuroToxic]

Hello,

Is it possible to launch a customized version of Firefox and a customized profile?

And does the UDF allow to launch several instances of firefox at the same time and to control them?

Thank you

[Danp2]

3 hours ago, NeuroToxic said:

Is it possible to launch a customized version of Firefox and a customized profile?

Yes. The wiki already shows how to utilize a custom profile. You can specify the Firefox binary in moz:firefoxOptions

Quote

does the UDF allow to launch several instances of firefox at the same time and to control them?

Not something that I've tested, but I imagine it should be possible. Give it a shot and let us know how it goes.

[ngocthang26]

@Danp2

Hi Danp2, you have problem set proxy for chromedriver.

If proxy no user:pass. Code is working. But proxy has user:pass have error alert.

Proxy JSON Object

A JSON object describing a Proxy configuration.

Key	Type	Description
proxyType	string	(Required) The type of proxy being used. Possible values are: direct - A direct connection - no proxy in use, manual - Manual proxy settings configured, e.g. setting a proxy for HTTP, a proxy for FTP, etc, pac - Proxy autoconfiguration from a URL, autodetect - Proxy autodetection, probably with WPAD, system - Use system settings
proxyAutoconfigUrl	string	(Required if proxyType == pac, Ignored otherwise) Specifies the URL to be used for proxy autoconfiguration. Expected format example: http://hostname.com:1234/pacfile
ftpProxy, httpProxy, sslProxy, socksProxy	string	(Optional, Ignored if proxyType != manual) Specifies the proxies to be used for FTP, HTTP, HTTPS and SOCKS requests respectively. Behaviour is undefined if a request is made, where the proxy for the particular protocol is undefined, if proxyType is manual. Expected format example: hostname.com:1234
socksUsername	string	(Optional, Ignored if proxyType != manual and socksProxy is not set) Specifies SOCKS proxy username.
socksPassword	string	(Optional, Ignored if proxyType != manual and socksProxy is not set) Specifies SOCKS proxy password.
noProxy	string	(Optional, Ignored if proxyType != manual) Specifies proxy bypass addresses. Format is driver specific.

I find doc from this website but try my code until don't working. i try _WD_Alert but return False or don't send text.

You can suggest help me how use proxy have user:pass.

Thanks so much

#include "wd_core.au3"
#include "wd_helper.au3"

Local $sSession
Local $sDesiredCapabilities

SetupChrome()
_WD_Startup()
$sSession = _WD_CreateSession($sDesiredCapabilities)
;~ _WD_Navigate($sSession, "https://whatismyipaddress.com/")
;~ Sleep(2000)

Sleep(3000)


Global $Handle = _WD_Alert($sSession, 'status')
MsgBox("","",$Handle,100)

Func SetupChrome()
_WD_Option('Driver', 'chromedriver.exe')
_WD_Option('Port', 9515)
_WD_Option('DriverParams', '--log-path="' & @ScriptDir & '\chrome.log"')
$sDesiredCapabilities = '{"capabilities":{"alwaysMatch":{"unhandledPromptBehavior":"ignore","proxy":{"proxyType":"manual","httpProxy":"123.31.45.40:1808","ftpProxy":"123.31.45.40:1808","sslProxy":"123.31.45.40:1808","socksUsername":"ngocthang456789","socksPassword":"ngocthang456789","autodetect":"true"},"goog:chromeOptions":{"w3c":true,"excludeSwitches":["enable-automation"],"useAutomationExtension":false,"prefs":{"credentials_enable_service":false},"args":["start-maximized"]}}}}'
$sDesiredCapabilities
EndFunc

 

[Danp2]

22 minutes ago, ngocthang26 said:

you have problem set proxy for chromedriver.

If proxy no user:pass. Code is working. But proxy has user:pass have error alert.

Please post this "error alert" along with any results from the Scite output panel (you should always include this if you are serious about getting assistance).

Quote

i try _WD_Alert but return False or don't send text

Not sure what you mean by "don't send text", but _WD_Alert is supposed to return either True or False when you call it with the "status" command. A return value of False indicated that there isn't an active alert.

Perhaps you should explain what you expected this command to do.

[ngocthang26]

_WDStartup: OS: WIN_7 WIN32_NT 7601 Service Pack 1
_WDStartup: AutoIt: 3.3.14.5
_WDStartup: WD.au3: 0.1.0.15
_WDStartup: Driver: chromedriver.exe
_WDStartup: Params: --log-path="C:\Users\Administrator\Desktop\Project test\View youtube proxy\chrome.log"
_WDStartup: Port:   9515
__WD_Post: URL=HTTP://127.0.0.1:9515/session; $sData={"capabilities":{"alwaysMatch":{"unhandledPromptBehavior":"ignore","proxy":{"proxyType":"manual","httpProxy":"123.31.45.40:1808","ftpProxy":"123.31.45.40:1808","sslProxy":"123.31.45.40:1808","socksUsername":"ngocthang456789","socksPassword":"ngocthang456789","autodetect":"true"},"goog:chromeOptions":{"w3c":true,"excludeSwitches":["enable-automation"],"useAutomationExtension":false,"prefs":{"credentials_enable_service":false},"args":["start-maximized"]}}}}
__WD_Post: StatusCode=200; ResponseText={"value":{"capabilities":{"acceptInsecureCerts":false,"acceptSslCerts":false,"applicationCacheEnabled":false,"browserConnectionEnabled":false,"browserName":"chrome","chrome":{"chromedriverVersion":"2.43.600210 (68dcf5eebde37173d4027fa8635e332711d2874a)","userDataDir":"C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\scoped_dir4776_5026"},"cssSelectorsEnabled":true,"databaseEnabled":false,"goog:chromeOptions":{"debuggerAddress":"localhost:60015"},"handlesAlerts":true,"hasTouchScreen":false,"javascriptEnabled":true,"locationContextEnabled":true,"mobileEmulationEnabled":false,"nativeEvents":true,"networkConnectionEnabled":false,"pageLoadStrategy":"normal","platform":"Windows NT","rotatable":false,"setWindowRect":true,"takesHeapSnapshot":true,"takesScreenshot":true,"unhandledPromptBehavior":"ignore","version":"79.0.3945.117","webStorageEnabled":true},"sessionId":"58156972e776e22985d9a88fb134c51e"}}
_WD_CreateSession: {"value":{"capabilities":{"acceptInsecureCerts":false,"acceptSslCerts":false,"applicationCacheEnabled":false,"browserConnectionEnabled":false,"browserName":"chrome","chrome":{"chromedriverVersion":"2.43.600210 (68dcf5eebde37173d4027fa8635e332711d2874a)","userDataDir":"C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\scoped_dir4776_5026"},"cssSelectorsEnabled":true,"databaseEnabled":false,"goog:chromeOptions":{"debuggerAddress":"localhost:60015"},"handlesAlerts":true,"hasTouchScreen":false,"javascriptEnabled":true,"locationContextEnabled":true,"mobileEmulationEnabled":false,"nativeEvents":true,"networkConnectionEnabled":false,"pageLoadStrategy":"normal","platform":"Windows NT","rotatable":false,"setWindowRect":true,"takesHeapSnapshot":true,"takesScreenshot":true,"unhandledPromptBehavior":"ignore","version":"79.0.3945.117","webStorageEnabled":true},"sessionId":"58156972e776e22985d9a88fb134c51e"}}
__WD_Post: URL=HTTP://127.0.0.1:9515/session/58156972e776e22985d9a88fb134c51e/url; $sData={"url":"https://whatismyipaddress.com/"}
__WD_Post: StatusCode=200; ResponseText={"value":null}
_WD_Navigate: {"value":null}

It my debug log. but problem Sign In Alerts require type User & Pass. I have set SocketUser and SocketPassword in $sDesiredCapabilities but don't effective.

Thanks

 

Edited February 1, 2020 by ngocthang26

[Danp2]

@ngocthang26 From what I can tell, the socks username and password is only valid if you're using a SOCKS proxy. Since your current code isn't working, you might try something like this instead --

$sDesiredCapabilities = '{"capabilities":{"alwaysMatch":{"unhandledPromptBehavior":"ignore","proxy":{"proxyType":"manual","httpProxy":"username:password@123.31.45.40:1808","ftpProxy":"123.31.45.40:1808","sslProxy":"username:password@123.31.45.40:1808","autodetect":"true"},"goog:chromeOptions":{"w3c":true,"excludeSwitches":["enable-automation"],"useAutomationExtension":false,"prefs":{"credentials_enable_service":false},"args":["start-maximized"]}}}}'

It should be obvious, but be sure to replace "username" and "password" with the correct values. 😄

[IanN1990]

Hi,

#NoTrayIcon
#AutoIt3Wrapper_Run_AU3Check=n
#include ".\Includes\wd_core.au3"
#include ".\Includes\wd_helper.au3"

$_WD_DEBUG = 0
_WD_Option('Driver', @ScriptDir & '\Includes\geckodriver-v0.26.0.exe')
_WD_Option('Port', 4444)

$sDesiredCapabilities = '{"desiredCapabilities":{"javascriptEnabled":true,"nativeEvents":true,"acceptInsecureCerts":true, "pageLoadStrategy":"none", "args":["-kiosk"], "args":["-p Automation"] }}'

_WD_Startup()
If @error <> $_WD_ERROR_Success Then Exit MsgBox(0, "Error", "Firefox Driver failed to load")

$sSession = _WD_CreateSession($sDesiredCapabilities)
If @error <> $_WD_ERROR_Success Then Exit MsgBox(0, "Error", "Firefox Driver failed to create a session")

_WD_Navigate($sSession, "https://www.google.com")

Are you about to spot where i have gone wrong?
Firefox loads and navigates to google but my two args  ("-kiosk") and ("-p Automation") are not having an effect. 
I have tested them outside of Webdriver and confirmed they work. 

Edited February 9, 2020 by IanN1990

[Danp2]

@IanN1990 I believe that your DesiredCapabilities string is incorrect as follows --

• There should be a single "args" entry, which can contain multiple arguments
• It should be inside a "moz:firefoxOptions" entry

[IanN1990]

@Danp2

Following your advice;

#NoTrayIcon

#AutoIt3Wrapper_Run_AU3Check=n
#include ".\Includes\wd_core.au3"
#include ".\Includes\wd_helper.au3"

ProcessClose("geckodriver-v0.26.0.exe")

$_WD_DEBUG = 0
_WD_Option('Driver', @ScriptDir & '\Includes\geckodriver-v0.26.0.exe')
_WD_Option('Port', 4444)

;$sDesiredCapabilities = '{"desiredCapabilities":{"javascriptEnabled":true,"nativeEvents":true,"acceptInsecureCerts":true, "pageLoadStrategy":"none"}}'
$sDesiredCapabilities = '{"capabilities":{"alwaysMatch": {"moz:firefoxOptions": {"args": ["-kiosk", "-private", "-profile", "C:/Users/Ian/Desktop/Firefox"]}}}}'

_WD_Startup()
If @error <> $_WD_ERROR_Success Then Exit MsgBox(0, "Error", "Firefox Driver failed to load")

$sSession = _WD_CreateSession($sDesiredCapabilities)
If @error <> $_WD_ERROR_Success Then Exit MsgBox(0, "Error", "Firefox Driver failed to create a session")

_WD_Navigate($sSession, "https://www.google.com")

Firefox now starts in Kiosk mode, with my profile and in private window but it now hangs at _WD_CreateSession($sDesiredCapabilities)
I am also unsure how to merge the desiredCapabilities  with the capabilities

Edited February 9, 2020 by IanN1990

[Danp2]

A few observations --

ProcessClose("geckodriver-v0.26.0.exe")

This line shouldn't be needed because _WD_Startup will automatically close the prior instance for you.

$_WD_DEBUG = 0

Suggest that you use the Enum constants, which should eliminate an issues when / if their underlying values change in the future

Quote

the _WD_Navigation command no longer works.

Please set $_WD_DEBUG = $_WD_DEBUG_Info, rerun the code, and then post the results from the Scite output panel

Quote

I am also unsure how to merge the desiredCapabilities  with the capabilities

They would still come after the alwayMatch, but not inside "moz:firefoxOptions" structure, like this --

$sDesiredCapabilities = '{"capabilities":{"alwaysMatch": {{"desiredCapabilities":{"javascriptEnabled":true,"nativeEvents":true,"acceptInsecureCerts":true, "pageLoadStrategy":"none", "moz:firefoxOptions": {"args": ["-kiosk", "-private", "-profile", "C:/Users/Ian/Desktop/Firefox"]}}}}'

 

[IanN1990]

@Danp2 Implementing your recommendations produces the following output. 

#NoTrayIcon
#AutoIt3Wrapper_Run_AU3Check=n
#include ".\Includes\wd_core.au3"
#include ".\Includes\wd_helper.au3"

$_WD_DEBUG = $_WD_DEBUG_Info
_WD_Option('Driver', @ScriptDir & '\Includes\geckodriver-v0.26.0.exe')
_WD_Option('Port', 4444)

$sDesiredCapabilities = '{"capabilities":{"alwaysMatch": {{"desiredCapabilities":{"javascriptEnabled":true,"nativeEvents":true,"acceptInsecureCerts":true, "pageLoadStrategy":"none", "moz:firefoxOptions": {"args": ["-kiosk", "-private", "-profile", "C:/Users/Ian/Desktop/Firefox"]}}}}'

_WD_Startup()
If @error <> $_WD_ERROR_Success Then Exit MsgBox(0, "Error", "Firefox Driver failed to load")

$sSession = _WD_CreateSession($sDesiredCapabilities)
If @error <> $_WD_ERROR_Success Then Exit MsgBox(0, "Error", "Firefox Driver failed to create a session")

_WD_Navigate($sSession, "https://www.bbc.com")



_WDStartup: OS: WIN_10 WIN32_NT 18363 
_WDStartup: AutoIt: 3.3.14.5
_WDStartup: WD.au3: 0.2.0.5
_WDStartup: WinHTTP:    1.6.4.1
_WDStartup: Driver: C:\Users\Vocalink\Desktop\09-02-2020\Includes\geckodriver-v0.26.0.exe
_WDStartup: Params: 
_WDStartup: Port:   4444
__WD_Post: URL=HTTP://127.0.0.1:4444/session; $sData={"capabilities":{"alwaysMatch": {{"desiredCapabilities":{"javascriptEnabled":true,"nativeEvents":true,"acceptInsecureCerts":true, "pageLoadStrategy":"none", "moz:firefoxOptions": {"args": ["-kiosk", "-private", "-profile", "C:/Users/Ian/Desktop/Firefox"]}}}}
__WD_Post: StatusCode=400; ResponseText={"value":{"error":"invalid argument","message":"Failed to decode request as JSON: {\"capabilities\":{\"alwaysMatch\": {{\"desiredCapabilities\":{\"javascriptEnabled\":true,\"nativeEvents\":true,\"acceptInsecureCerts\":true, \"pageLoadStrategy\":\"none\", \"moz:firefoxOptions\": {\"args\": [\"-kiosk\", \"-private\", \"-profile\", \"C:/Users/Ian/Desktop/Firefox\"]}}}}","stacktrace":"Syntax error at :1:34"}}
_WD_CreateSession: {"value":{"error":"invalid argument","message":"Failed to decode request as JSON: {\"capabilities\":{\"alwaysMatch\": {{\"desiredCapabilities\":{\"javascriptEnabled\":true,\"nativeEvents\":true,\"acceptInsecureCerts\":true, \"pageLoadStrategy\":\"none\", \"moz:firefoxOptions\": {\"args\": [\"-kiosk\", \"-private\", \"-profile\", \"C:/Users/Ian/Desktop/Firefox\"]}}}}","stacktrace":"Syntax error at :1:34"}}
_WD_CreateSession ==> Webdriver Exception: Failed to decode request as JSON: {"capabilities":{"alwaysMatch": {{"desiredCapabilities":{"javascriptEnabled":true,"nativeEvents":true,"acceptInsecureCerts":true, "pageLoadStrategy":"none", "moz:firefoxOptions": {"args": ["-kiosk", "-private", "-profile", "C:/Users/Ian/Desktop/Firefox"]}}}}

**A small note, i noticed the forum is adding hidden characters. I had to copy it to notepad first to remove them. 

Edited February 9, 2020 by IanN1990

[Danp2]

I suggest that you try to troubleshoot one thing at a time. Please rerun after removing the latest edits to $sDesiredCapabilities.