WebDriver UDF (W3C compliant version) - 2023/02/15

[Danp2 1,382]

@BigDaddyO Because they aren't listed in the W3C specs.  https://w3c.github.io/webdriver/#locator-strategies

Were you using them somehow?

[BigDaddyO 81]

Yeah, i'm using $_WD_LOCATOR_ByID and $_WD_LOCATOR_ByClassName in a few places in the older part of my script as I hadn't learned xpath yet.

I suppose it wouldn't be to hard to update those to XPath.

 

edit, the $_WD_LOCATOR_ByID="id" which is used in 2 functions.  Is the "id" ok to use in _WD_FrameEnter(), _WD_ElementAction()?

Edited August 7, 2018 by BigDaddyO

[Danp2 1,382]

Please show me some working examples where it you were using $_WD_LOCATOR_ByID and $_WD_LOCATOR_ByClassName.

2 hours ago, BigDaddyO said:

Is the "id" ok to use in _WD_FrameEnter(), _WD_ElementAction()?

Don't confuse a locator type with a parameter being passed to the webdriver. The Switch to Frame specs call for a json key named "id", where the associated value is either null, an integer index, or a web element. I'm sure the specs are similar for interacting with elements.

 

[BigDaddyO 81]

I've already finished converting those over to xpath as it was very simple.  I'm testing a softphone and there are a few elements that have nothing but an id or class and they are unique so they used to work fine but the conversion to xpath was super simple.  I had never used xpath before using your UDF's.  Perhaps some xpath help links could be added to the first post as it seems to be VERY important to be able to really use this properly.

For me, I got a lot of help from these 3 site.  Also, it took a while to realize that xpath is case sensitive

https://www.guru99.com/xpath-selenium.html

https://www.guru99.com/using-contains-sbiling-ancestor-to-find-element-in-selenium.html

https://gist.github.com/LeCoupa/8c305ec8c713aad07b14

 

 

As for some examples of what I was using vs how I converted over to xpath

ByClassName:  if _WD_WaitElement($sSession, $_WD_LOCATOR_ByClassName, "ctiSoftphone", 2000, 120000) = 0 Then

Switched to:  if _WD_WaitElement($sSession, $_WD_LOCATOR_ByXPath, "//div[@class='ctiSoftphone']", 2000, 120000) = 0 Then

 

ByID:  $oExtension = _WD_FindElement($sSession, $_WD_LOCATOR_ByID, "ab120f6a04114d5bab4cfc96abc75da1_PRIMARY_LINE_NUMBER")

Switched to:  $oExtension = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//input[contains(@id, 'PRIMARY_LINE_NUMBER')]")

 

[Danp2 1,382]

15 minutes ago, BigDaddyO said:

ByID:  $oExtension = _WD_FindElement($sSession, $_WD_LOCATOR_ByID, "ab120f6a04114d5bab4cfc96abc75da1_PRIMARY_LINE_NUMBER")

Switched to:  $oExtension = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//input[contains(@id, 'PRIMARY_LINE_NUMBER')]")

I know you are using Chrome, which isn't completely up to snuff with the W3C specs at this time. Did you try this with a different browser? I suspect that it won't work, but would need to test to be sure.

[CYCho 109]

@Danp2 Same as BigDaddyO, I didn't know anything about xpath until recently when I had to change my codes. I changed all my ByID strategies to ByXPath and they all work fine in my chromedriver. I always appreciate your efforts to make the webdriver easier to use.

By the way, I find that the $sSelector part of the _WD_FindElement function must be enclosed in double quotes: "//input[@id='login-id']" works, but '//input[@id="login-id"]' doesn't. Is this by design?

Edited August 9, 2018 by CYCho

[Danp2 1,382]

2 hours ago, CYCho said:

By the way, I find that the $sSelector part of the _WD_FindElement function must be enclosed in double quotes: "//input[@id='login-id']" works, but '//input[@id="login-id"]' doesn't. Is this by design?

Thanks for pointing that out. I take a look to see if there's an easy fix.

[CYCho 109]

<a href="javascript:MemberCheck('2');">Search</a>

<a href="javascript:MemberCheck('3');">Search</a>

These are the links I want to click using chromedriver.

Since the text is same for both links, I tried _WD_LinkClickByText($sSession, "javascript:MemberCheck('2');"), which failed.

Maybe I should try _WD_FindElement, but I don't know how to handle 3 sets of quotation marks in $sSelector. HTML character codes do not seem to work here.

Please give me a hint to solve this problem. 

[Danp2 1,382]

@CYCho I would think _WD_FindElement would work here. Either --

• Use proper xpath to locate the 2nd instance of a link with the text of 'Search'
• Use the $lMultiple parameter to retrieve all instances of a link with the text of 'Search' and then process the results to locate the one you want

[CYCho 109]

Quote

• Use the $lMultiple parameter to retrieve all instances of a link with the text of 'Search' and then process the results to locate the one you want

Yes, it works now. Thank you as always.

[Danp2 1,382]

On 8/9/2018 at 3:26 AM, CYCho said:

By the way, I find that the $sSelector part of the _WD_FindElement function must be enclosed in double quotes: "//input[@id='login-id']" works, but '//input[@id="login-id"]' doesn't. Is this by design?

This appears to be a JSON limitation. You can make it work by escaping the double quotes, like this --

$sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, '//input[@id=\"lst-ib\"]')

 

[nulee 0]

hi 

I used _WD_Window($sSession, 'frame', '{"id":1}') to solve my problem. Now I need to enable flash support. What do I need to do in _WD_Option()?

Following code :

"profile.default_content_setting_values.plugins": 1,
"profile.content_settings.plugin_whitelist.adobe-flash-player": 1,
"profile.content_settings.exceptions.plugins.*,*.per_resource.adobe-flash-player": 1,
"PluginsAllowedForUrls": "http://www.xxx.cc"

my code:

$EnableFlash = '{"prefs":{"profile":{"default_content_setting_values":{"plugins": 1}},{"content_settings":{"plugin_whitelist":{"adobe-flash-player":1},{"exceptions":{"plugins":{"*":{"*":{"per_resource":{"adobe-flash-player":1}}}}}}}}},{"PluginsAllowedForUrls": "http://www.xxx.cc"}}'

$sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"chromeOptions": {"w3c": true},'&$EnableFlash&'}}}'

error:

__WD_Post: URL=HTTP://127.0.0.1:9515/session; $sData={"capabilities": {"alwaysMatch": {"chromeOptions": {"w3c": true},{"prefs":{"profile":{"default_content_setting_values":{"plugins": 1}},{"content_settings":{"plugin_whitelist":{"adobe-flash-player":1},{"exceptions":{"plugins":{"*":{"*":{"per_resource":{"adobe-flash-player":1}}}}}}}}},{"PluginsAllowedForUrls": "http://www.xxx.cc"}}}}}
__WD_Post: StatusCode=400; ResponseText=missing command parameters
_WD_CreateSession: missing command parameters
_WD_CreateSession ==> Webdriver Exception

 

Edited August 21, 2018 by nulee

[timmalos 3]

Hey all. Not sure I should post this there, but since ages when automatizing tests with AutoIt I ran into the simple issue that I always lack the possibility to retrieve the HTTP code returned by the request in my navigator. When you do heavy automations, it's painfull to need to retrive from JS (or other ways) an URI and then do another HTTP request manually in your script to check the HTTP statuscode. (Making another request, and possibly not the same answer than what you had in your navigator).

Looking into this library instead of the old deprecated FF.au3 (with Mozrepl), I saw sadly it's still true and WebDriver specification dosn't allow us to retrieve this HTTP statuscode.

However I was able this time to find another way to retrieve it, by parsing FF logs, so I share it if someone else need that. This works with FF only, but there is a way to do the same thing with Chrome if needed ( All you need to do is tell chrome driver to do "Network.enable". This can be done by enabling Performance logging).

 

I modified a little bit _WD_Option (see attached) to allow set another option :

_WD_Option('FFLogFile', _TempFile(@ScriptDir&"\fflogs\"))

If this value is set, I simply set two environment variables before the start of Firefox, thus I'm able to read FF logs, parse them and retrieve from them my HTTP Statuscode.

 

expandcollapse popup

Func _WD_GetHTTPStatusCode_FFOnly($URI)
    Local Const $sFuncName = "_WD_GetLastHTTPStatusCode"
    Local $uriID = ""

    $sFilePath = $_WD_Firefox_LogFile
    If $sFilePath = "" Then Return SetError(__WD_Error($sFuncName, $_WD_ERROR_InvalidValue, "You need to set option of LogFile in order to be able to retrieve HTTP statuscode"))
    Local $hFileOpen = FileOpen($sFilePath, 0)
    If $hFileOpen = -1 Then
        Return SetError(__WD_Error($sFuncName, $_WD_ERROR_InvalidValue, "Unable to open FF log file"))
    EndIf
    Local $sFileRead = FileRead($hFileOpen)

    ; this is how the log looks like when the request starts
    ; I have to get the id of the request using a regular expression
    ; and use that id later to get the response
    ;
    ;    2017-11-02 14:14:01.170000 UTC - [Main Thread]: D/nsHttp nsHttpChannel::BeginConnect [this=000000BFF27A5000]
    ;    2017-11-02 14:14:01.170000 UTC - [Main Thread]: D/nsHttp host=api.ipify.org port=-1
    ;    2017-11-02 14:14:01.170000 UTC - [Main Thread]: D/nsHttp uri=https://api.ipify.org/?format=text

    Local $pattern = "(?is)BeginConnect \[this=(.*?)\](?:.*?)uri=([^\]]+?)\R"
    $aRet = StringRegExp($sFileRead,$pattern,3)
    For $i = 1 To UBound($aRet) - 1 Step 2
        If $aRet[$i] = $URI Then
            $uriID = $aRet[$i-1]
            ContinueLoop
        EndIf
    Next

    If $uriID = "" THen Return SetError(__WD_Error($sFuncName, $_WD_ERROR_InvalidValue, "Unable to find related URI in FF logs to retrieve HTTP statuscode"))


    ; this is how the response looks like in the log file
    ; ProcessResponse [this=000000CED8094000 httpStatus=200]
    ; I will use another regular espression to get the httpStatus
    ;
    ;    2017-11-02 14:45:39.296000 UTC - [Main Thread]: D/nsHttp nsHttpChannel::OnStartRequest [this=000000CED8094000 request=000000CED8014BB0 status=0]
    ;    2017-11-02 14:45:39.296000 UTC - [Main Thread]: D/nsHttp nsHttpChannel::ProcessResponse [this=000000CED8094000 httpStatus=200]

    Local $pattern = "(?is)ProcessResponse \[this="&$uriID&" httpStatus=(.*?)\]";
    $aRet = StringRegExp($sFileRead,$pattern,1)
    FileClose($hFileOpen)
    Return $aRet[0]
EndFunc

 

And simple example :

$URI = "http://www.autoitscript.com/404/"
_WD_Navigate($sSession, $URI)
MsgBox(0,"",_WD_GetHTTPStatusCode_FFOnly($URI))

$URI = "https://www.autoitscript.com/forum/topic/191990-webdriver-udf-w3c-compliant-version-08042018/"
_WD_Navigate($sSession, $URI)
MsgBox(0,"",_WD_GetHTTPStatusCode_FFOnly($URI))

 

wd_core.au3

[Shogi 0]

Help plz!!

[Danp2 1,382]

@Shogi You should take a look at the demo (wd_demo.au3) if you haven't done that yet. Also, there is a separate, general support thread here --

 

[Shogi 0]

@danp thank you... also how can i press that button?

[Danp2 1,382]

Again, I would refer you back to the demo, specifically the DemoElements function.

Hint: You'll need to use _WD_FindElement to find the element (try using an xpath of "\\div[@id='play_button']") and then _WD_ElementAction to perform the click.

[omarjr16 0]

This Function helped me alot so i decided to share it with u

expandcollapse popup

;Add this to wd-core.au3
#Region Global Variables
Global $_WD_ResolveTimeout = Default, _
       $_WD_ConnectTimeout = Default, _
       $_WD_SendTimeout = Default, _
       $_WD_ReceiveTimeout = Default, _
#EndRegion Global Variables

;Add this ligne to wd-core.au3 under "_WinHttpOpen()" in (__WD_Get/__WD_Post/__WD_Delete) Functions
_WinHttpSetTimeouts($hOpen, $_WD_ResolveTimeout, $_WD_ConnectTimeout, $_WD_SendTimeout, $_WD_ReceiveTimeout)

;Add this function to wd-core.au3 or wd-helper.au3 or in ur own script
; #FUNCTION# ;===============================================================================
; Name...........: _WD_WinHttpTimeouts
; Description ...:
; Syntax.........: _WD_WinHttpTimeouts([, $sResolveTimeout = '' [, $sConnectTimeout = '' [, $iSendTimeout = '' [, $iReceiveTimeout = '' ]]]])
; Parameters ....: $sResolveTimeout - [optional] Time-out value, in milliseconds, to use for name resolution.
;                  $sConnectTimeout - [optional] Time-out value, in milliseconds, to use for server connection requests.
;                  $sSendTimeout - [optional] Time-out value, in milliseconds, to use for sending requests.
;                  $sReceiveTimeout - [optional] Time-out value, in milliseconds, to receive a response to a request.
; Return values .: Success      - Timeouts
; Author ........: OmarJr16
; Modified ......:
; Remarks .......: Initial values are:
;                  |- $sResolveTimeout = 0
;                  |- $sConnectTimeout = 60000
;                  |- $sSendTimeout = 30000
;                  |- $sReceiveTimeout = 30000
; Related .......:
; Link ..........: http://msdn.microsoft.com/en-us/library/aa384116.aspx
; Example .......: No
;============================================================================================
Func _WD_WinHttpTimeouts($sResolveTimeout = '', $sConnectTimeout = '', $sSendTimeout = '', $sReceiveTimeout = '')
    If $sResolveTimeout <> '' Then _
        $_WD_ResolveTimeout = $sResolveTimeout
    If $sConnectTimeout <> '' Then _
        $_WD_ConnectTimeout = $sConnectTimeout
    If $sSendTimeout <> '' Then _
        $_WD_SendTimeout = $sSendTimeout
    If $sReceiveTimeout <> '' Then _
        $_WD_ReceiveTimeout = $sReceiveTimeout
    $Timeouts =('{"timeout":{"resolve":' & $_WD_ResolveTimeout & ',"connect":' & $_WD_ConnectTimeout & ',"send":' & $_WD_SendTimeout & ',"receive":' & $_WD_ReceiveTimeout)
    If $_WD_DEBUG = $_WD_DEBUG_Info Then _
        ConsoleWrite($Timeouts)
    Return $Timeouts
EndFunc

Edited September 14, 2018 by omarjr16

[mLipok 1,389]

Please edit your example as follow:

//Add this ligne to wd-core.au3 under "_WinHttpOpen()" in (__WD_Get/__WD_Post/__WD_Delete) Functions

should be:

; Add this ligne to wd-core.au3 under "_WinHttpOpen()" in (__WD_Get/__WD_Post/__WD_Delete) Functions

as this 

//

is not a comment sign/character in AutoIt

 

[BigDaddyO 81]

I use an old function from SmOke_N to close chrome since chromedriver.exe spawns child processes that were not closing with the normal _WD_Shutdown()

 

expandcollapse popup

;Testing Chrome browser as Headless to run on a server after disconnecting from RDP
#AutoIt3Wrapper_UseX64=n
#include "wd_core.au3"
#include "wd_helper.au3"

$_WD_DEBUG = $_WD_DEBUG_None        ;set to $_WD_DEBUG_None before compiling.

    $sWebSite = "https://www.autoitscript.com/site/autoit/downloads/"
;~  $sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"chromeOptions": {"w3c": true, "useAutomationExtension": false, "args":["start-maximized", "disable-infobars"] }}}}' ;This will display Chrome Full screen and inteact with it
    $sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"chromeOptions": {"w3c": true, "useAutomationExtension": false, "args":["headless", "disable-gpu", "disable-infobars"] }}}}' ;This will launch Chrome in a Headless mode, Use _WD_Startup(True) to completely hide

MsgBox(0, "Wait for close", "This script will now wait 30 seconds before automatically continuing" & @CRLF & @CRLF & "Please disconnect from RDP to test without an Active Desktop", 30)


    _SetupChrome()

    $iPID = _WD_Startup()                                                               ;Return the PID of the ChromeDriver.exe as we will need it to close when done

    $sSession = _WD_CreateSession($sDesiredCapabilities)
    _WD_Timeouts($sSession, '{"type":"page load","ms":120000}')     ;Increase the default timeout.

    _WD_Navigate($sSession, $sWebSite)

    If _WD_WaitElement($sSession, $_WD_LOCATOR_ByXPath, "//div/table[1]/tbody/tr/td[2]", 1000, 20000, True) <> 1 Then
        ConsoleWrite("Error, Item not found" & @CRLF)
        Exit
    EndIf

    $eVer = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//div/table[1]/tbody/tr/td[1]")
    $sVer = _WD_ElementAction($sSession, $eVer, "text")

    $eDate = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//div/table[1]/tbody/tr/td[2]")
    $sDate = _WD_ElementAction($sSession, $eDate, "text")

ConsoleWrite(@CRLF & "*** Latest AutoIT version = ( " & $sVer & " )" & @CRLF & "*** Release Date = ( " & $sDate & " )" & @CRLF & @CRLF)


;New section since Chromedriver and chrome windows werent closing
$aSpawnedChildren = _ProcessGetChildren($iPID)          ;Get all processes spawned directly from the Chromedriver.exe
ProcessClose($iPID)                                                         ;Close the Chromedriver.exe
For $x = 0 to UBound($aSpawnedChildren) - 1                 ;Move through every child process
    ProcessClose($aSpawnedChildren[$x][0])                      ;Close the child process
Next


;From SmOke_N
;https://www.autoitscript.com/forum/topic/78445-solved-parent-process-child-process/
Func _ProcessGetChildren($i_pid) ; First level children processes only
    Local Const $TH32CS_SNAPPROCESS = 0x00000002

    Local $a_tool_help = DllCall("Kernel32.dll", "long", "CreateToolhelp32Snapshot", "int", $TH32CS_SNAPPROCESS, "int", 0)
    If IsArray($a_tool_help) = 0 Or $a_tool_help[0] = -1 Then Return SetError(1, 0, $i_pid)

    Local $tagPROCESSENTRY32 = _
    DllStructCreate _
    ( _
    "dword dwsize;" & _
    "dword cntUsage;" & _
    "dword th32ProcessID;" & _
    "uint th32DefaultHeapID;" & _
    "dword th32ModuleID;" & _
    "dword cntThreads;" & _
    "dword th32ParentProcessID;" & _
    "long pcPriClassBase;" & _
    "dword dwFlags;" & _
    "char szExeFile[260]" _
    )
    DllStructSetData($tagPROCESSENTRY32, 1, DllStructGetSize($tagPROCESSENTRY32))

    Local $p_PROCESSENTRY32 = DllStructGetPtr($tagPROCESSENTRY32)

    Local $a_pfirst = DllCall("Kernel32.dll", "int", "Process32First", "long", $a_tool_help[0], "ptr", $p_PROCESSENTRY32)
    If IsArray($a_pfirst) = 0 Then Return SetError(2, 0, $i_pid)

    Local $a_pnext, $a_children[11][2] = [[10]], $i_child_pid, $i_parent_pid, $i_add = 0
    $i_child_pid = DllStructGetData($tagPROCESSENTRY32, "th32ProcessID")
    If $i_child_pid <> $i_pid Then
    $i_parent_pid = DllStructGetData($tagPROCESSENTRY32, "th32ParentProcessID")
    If $i_parent_pid = $i_pid Then
    $i_add += 1
    $a_children[$i_add][0] = $i_child_pid
    $a_children[$i_add][1] = DllStructGetData($tagPROCESSENTRY32, "szExeFile")
    EndIf
    EndIf

    While 1
    $a_pnext = DLLCall("Kernel32.dll", "int", "Process32Next", "long", $a_tool_help[0], "ptr", $p_PROCESSENTRY32)
    If IsArray($a_pnext) And $a_pnext[0] = 0 Then ExitLoop
    $i_child_pid = DllStructGetData($tagPROCESSENTRY32, "th32ProcessID")
    If $i_child_pid <> $i_pid Then
    $i_parent_pid = DllStructGetData($tagPROCESSENTRY32, "th32ParentProcessID")
    If $i_parent_pid = $i_pid Then
    If $i_add = $a_children[0][0] Then
    ReDim $a_children[$a_children[0][0] + 11][2]
    $a_children[0][0] = $a_children[0][0] + 10
    EndIf
    $i_add += 1
    $a_children[$i_add][0] = $i_child_pid
    $a_children[$i_add][1] = DllStructGetData($tagPROCESSENTRY32, "szExeFile")
    EndIf
    EndIf
    WEnd

    If $i_add <> 0 Then
    ReDim $a_children[$i_add + 1][2]
    $a_children[0][0] = $i_add
    EndIf

    DllCall("Kernel32.dll", "int", "CloseHandle", "long", $a_tool_help[0])
    If $i_add Then Return $a_children
    Return SetError(3, 0, 0)
EndFunc



Func _SetupChrome()

    If _WD_Option('Driver', 'chromedriver.exe') = 0 Then Return SetError(1)

    If _WD_Option('Port', 9515) = 0 Then Return SetError(1)

    If _WD_Option('DriverParams', '--log-path="' & @ScriptDir & '\chrome.log"') = 0 Then Return SetError(1)

EndFunc   ;==>_SetupChrome

 

edit:  updated for the current version of the UDF, this just ensures all spawned chrome windows are closed.

 

Edited September 17, 2018 by BigDaddyO