WebDriver UDF (W3C compliant version) - 2024/02/19

[Danp2]

51 minutes ago, Purity8 said:

I noticed error code 13 has now become error code 4. This typically happens when a gallery ends & there's a 404 message saying file not found; which is normal.

No idea what you mean here. Please post a code sample that demonstrates the issue if you want me to look further into it

FWIW, I only added three lines of code to deal with the earlier issue you reported, which sets an appropriate error code when the InetRead fails.

P.S. You shouldn't be relying on values such as 4 or 13. Use the appropriate Enum constant instead.

[Purity8]

13 hours ago, Danp2 said:

Please post a code sample that demonstrates the issue if you want me to look further into it

In this case, the code isn't the issue. No major changes were done with the coding.
I simply copied the new wd_core & helper & used F5 to run the same script.

13 hours ago, Danp2 said:

P.S. You shouldn't be relying on values such as 4 or 13. Use the appropriate Enum constant instead.

Are these values automatically assigned? Based on 0.3.0.8's wd_core; the Global Enum's were:

General Error = 1
Invalid Value = 4
Send / Recv Error = 6
Not Found = 13

In 0.3.0.9, Not Found has become Invalid Value.

For example:
v.0.3.0.8 - When downloading images, gallery has 10 images.
10th image will be 0 =  successfully downloaded.
11th image is Not Found (13).

v0.3.0.9 - When downloading images, gallery has 10 images.
10th image will be 0 =  successfully downloaded.
11th image is Invalid Value (4).

 

The minor adjustments I made were simply commenting out this new change to reflect the previous "Invalid Value" that I've set to "Exit" script.

If @error = 0 Then
        ConsoleWrite(@CRLF & "Download Complete." & @CRLF & @CRLF)
    ElseIf @error = 1 Then
        MsgBox($MB_SYSTEMMODAL, "Caution:", "1 = General Error - Directory doesn't exist or incomplete file path.")
        ConsoleWrite(@CRLF & "1 = General Error - Directory doesn't exist or incomplete file path." & @CRLF & @CRLF)
        Exit
    ElseIf @error = 4 Then
        ;MsgBox($MB_SYSTEMMODAL, "Caution:", "4 = Invalid value - URL variable has an empty field and not assigned a base URL. Exitting script." & @CRLF)
        ;ConsoleWrite(@CRLF & "URL variable is empty. Exitting script." & @CRLF & @CRLF)
        ;Exit
    ElseIf @error = 13 Then
        ConsoleWrite(@CRLF & "Download link not found or end of gallery." & @CRLF)
    Else
        MsgBox($MB_SYSTEMMODAL, "Notice:", @CRLF & "Something else went wrong..." & @CRLF)
        ConsoleWrite(@CRLF & "Something else went wrong..." & @CRLF & @CRLF & $Link & @CRLF & $Dest & @CRLF)
        Exit
    EndIf

 

Edited September 14, 2020 by Purity8

[Danp2]

43 minutes ago, Purity8 said:

Are these values automatically assigned?

Yes... Look near the top of wd_core.au3 and you will see the following --

Global Enum _
        $_WD_ERROR_Success = 0, _ ; No error
        $_WD_ERROR_GeneralError, _ ; General error
        $_WD_ERROR_SocketError, _ ; No socket
        $_WD_ERROR_InvalidDataType, _ ; Invalid data type (IP, URL, Port ...)
        $_WD_ERROR_InvalidValue, _ ; Invalid value in function-call
        $_WD_ERROR_InvalidArgue, _ ; Invalid argument in function-call
        $_WD_ERROR_SendRecv, _ ; Send / Recv Error
        $_WD_ERROR_Timeout, _ ; Connection / Send / Recv timeout
        $_WD_ERROR_NoMatch, _ ; No match for _WDAction-find/search _WDGetElement...
        $_WD_ERROR_RetValue, _ ; Error echo from Repl e.g. _WDAction("fullscreen","true") <> "true"
        $_WD_ERROR_Exception, _ ; Exception from web driver
        $_WD_ERROR_InvalidExpression, _ ; Invalid expression in XPath query or RegEx
        $_WD_ERROR_NoAlert, _ ; No alert present when calling _WD_Alert
        $_WD_ERROR_NotFound, _ ;
        $_WD_ERROR_ElementIssue, _ ;
        $_WD_ERROR_SessionInvalid, _ ;
        $_WD_ERROR_UnknownCommand, _ ;
        $_WD_ERROR_COUNTER ;

The last time this was modified was back in v0.3.0.1. Use the enum name and not it's value when performing a comparison. That way, you don't run into issues if I change their order, insert a new entry in the middle, etc.

49 minutes ago, Purity8 said:

In 0.3.0.9, Not Found has become Invalid Value.

That isn't correct AFAICS. While I did add code to properly handle the InetRead failure, which I happened to use $_WD_ERROR_InvalidValue for the error response, that shouldn't have had any effect on _WD_DownloadFile returning $_WD_ERROR_NotFound when appropriate.

52 minutes ago, Purity8 said:

In this case, the code isn't the issue. No major changes were done with the coding.
I simply copied the new wd_core & helper & used F5 to run the same script.

Beg to differ, but the code is exactly the issue from my POV. I can't fix an issue, assuming there is one, unless I can reproduce it. At this point, I still don't understand why you believe that the return value from _WD_DownloadFile has changed beyond properly handling the InetRead issue.

Again, if you want me to look into this further, share a short "reproducer" script that I can run to see that the return value is different from the earlier release.

[Purity8]

22 hours ago, Danp2 said:

Again, if you want me to look into this further, share a short "reproducer" script that I can run to see that the return value is different from the earlier release.

expandcollapse popup

#include <../../Required/wd_helper.au3>
#include <../../Required/wd_core.au3>

Global Const $DownloadsDir = @UserProfileDir & "\Downloads\"
Global Const $Url_01 = "http://q1.xiongmaoya.com/2018/05/24/8883/"
Global $Download = _WD_DownloadFile, $Dest, $Link, $Img, $n1, $n2, $i = 1, $Format = ".jpg"


; Ask
Func Ask()
    ConsoleWrite(@CRLF & "Enter a number where image gallery begins." & @CRLF)
    $n1 = InputBox("Notice", "Which image number do you wish to START from?", "", " M3")
    If @error = 1 Then
        ConsoleWrite(@CRLF & "Cancelled!" & @CRLF)
        Exit
    EndIf

    ConsoleWrite($n1 & @CRLF)

    ConsoleWrite(@CRLF & "Enter a number where image gallery ends." & @CRLF)
    $n2 = InputBox("Notice", "Which image number do you wish to END?", "", " M3")
    If @error = 1 Then
        ConsoleWrite(@CRLF & "Cancelled!" & @CRLF)
        Exit
    EndIf
    ConsoleWrite($n2 & @CRLF)
EndFunc

; Download
Func DL()
    ConsoleWrite(@CRLF & "Silently downloading..." & @CRLF)

    ConsoleWrite("From: " & $Link & @CRLF & "To: " & $Dest & @CRLF)
    $Download($Link, $Dest & $Img)

    If @error = 0 Then
        ConsoleWrite(@CRLF & "Download Complete." & @CRLF & @CRLF)
    ElseIf @error = 1 Then
        MsgBox($MB_SYSTEMMODAL, "Caution:", "1 = General Error - Directory doesn't exist or incomplete file path.")
        ConsoleWrite(@CRLF & "1 = General Error - Directory doesn't exist or incomplete file path." & @CRLF & @CRLF)
        Exit
    ElseIf @error = 4 Then
        ;MsgBox($MB_SYSTEMMODAL, "Caution:", "4 = Invalid value - URL variable has an empty field and not assigned a base URL. Exitting script." & @CRLF)
        ;ConsoleWrite(@CRLF & "URL variable is empty. Exitting script." & @CRLF & @CRLF)
        ;Exit
    ElseIf @error = 13 Then
        ConsoleWrite(@CRLF & "Download link not found or end of gallery." & @CRLF)
    Else
        MsgBox($MB_SYSTEMMODAL, "Notice:", @CRLF & "Something else went wrong..." & @CRLF)
        ConsoleWrite(@CRLF & "Something else went wrong..." & @CRLF & @CRLF & $Link & @CRLF & $Dest & @CRLF)
        Exit
    EndIf
EndFunc

; For Loop
Func F_Loop()
For $i = $n1 To $n2 ; For Loop started..n1 & n2 = user defined
    ConsoleWrite(@CRLF & "For Looping..." & @CRLF)

    $Img = $i & $Format  ; Allows $Img to loop along $i


$Dest = $DownloadsDir
$Link = $Url_01
DL()

    ConsoleWrite(@CRLF & "Looping..." & @CRLF)
Next    ; End of 'For' Loop
    ConsoleWrite(@CRLF & "End of For Loop" & @CRLF)

    MsgBox($MB_OK, "Notice:", "All Downloads complete!")
EndFunc


Ask()
F_Loop()

As requested, this is a very basic sample of my script. It's actually much longer as I've made my own custom au3's but I've simplified it as much as possible.

I know it's probably not the best way to do it, but I don't quite understand how to do this using Arrays (as Tutorials did mention not to use loops to count) despite reading & doing testings for a long time.

49 is the last image of this gallery, 50 does not exist; hence should be Not Found.
However, it'll repeat & download 49 as 50 anyway in the latest version. This was not the case in previous version.

Edited September 15, 2020 by Purity8

[Danp2]

First off, that's an awful example IMO. You could have demonstrated the issue with just a few lines, like this --

#include "webdriver-0.3.0.9\wd_helper.au3"

Global Const $DownloadsDir = @UserProfileDir & "\Downloads\"
Global Const $Url_01 = "http://q1.xiongmaoya.com/2018/05/24/8883/49.jpg"
Global Const $Url_02 = "http://q1.xiongmaoya.com/2018/05/24/8883/50.jpg"

_WD_DownloadFile($Url_01, $DownloadsDir & "Url_01.jpg", 0)
_WD_DownloadFile($Url_02, $DownloadsDir & "Url_02.jpg", 0)

I now understand the issue, but I also see that the earlier "Not Found" error was by chance since that was the return value from InetRead, which we can't depend on being 13 as you previously experienced.

The inner workings of _WD_DownloadFile has changed over time, and I can see that there are still some remnants of using __WD_Get that need to be removed. At this point we just need to settle on the correct return value for when the download doesn't succeed.

[Purity8]

I was going to make one more revision, but I guess you figured it out.

The revision would correct to..

Global Const $Url_01 = "http://q1.xiongmaoya.com/2018/05/24/8883/"

And in the loop function:

$Link = $Url_01 & $Img

In this matter, it'll not only auto assign the associated image number corresponding to the gallery number, but allow to choose any image because of the Ask function (some galleries have 2 parts, thus the URL link changing & going 50+ images)

 

When I first started, I did exactly as mentioned:

_WD_DownloadFile($Url_01, $DownloadsDir & "Url_01.jpg", 0)
_WD_DownloadFile($Url_02, $DownloadsDir & "Url_02.jpg", 0)

However, some galleries have over 100 images. That meant I'd have to manually add lines like...

_WD_DownloadFile($Url_01, $DownloadsDir & "1.jpg", 0)
_WD_DownloadFile($Url_01, $DownloadsDir & "2.jpg", 0)
..
..
..
_WD_DownloadFile($Url_01, $DownloadsDir & "102.jpg", 0)
..
..
_WD_DownloadFile($Url_01, $DownloadsDir & "150.jpg", 0)
.etc

You can imagine how frustrated I'd be editting all those lines of codes. And that's just one gallery. People can have over 20 galleries randomly ending in any number.

Additionally, I can add their other galleries by simply using more Url's.

Edited September 15, 2020 by Purity8

[duzers]

Is it possible to attach with an existing website?

[Danp2]

2 minutes ago, duzers said:

Is it possible to attach with an existing website?

Yes... with some restrictions. There's already some info in the Wiki FAQ, but I believe that pertains mostly to Firefox. I've done some testing with Chrome, and it's possible. Can't recall if the details have been posted here or not. The same method should work for Edge since it operates on the same code base as Chrome.

P.S. Please stick to one thread and don't cross post your questions 😉

[duzers]

23 hours ago, Danp2 said:

Yes... with some restrictions. There's already some info in the Wiki FAQ, but I believe that pertains mostly to Firefox. I've done some testing with Chrome, and it's possible. Can't recall if the details have been posted here or not. The same method should work for Edge since it operates on the same code base as Chrome.

P.S. Please stick to one thread and don't cross post your questions 😉

Thx. I use ie.au3 for stock exchange autotrading and I have to search new fast possibilities (end of support Internet Explorer). Is the webdrive the same fast like IE.au3?

Which browser (-s) are you recommend to automation with webdrive?

 

Edited September 18, 2020 by duzers

[Danp2]

I believe all the major browsers support the webdriver standard. You should be fine using Chrome, Firefox, Edge, etc

[duzers]

Hello, I succesfull run WebDriver with Edge but how to attach to started session from other .au3?

Something like this. First .au3 navigate to site, logging and exit. Second .au3 send action from time to time.

Is it possible to do?

[Danp2]

@duzers Have you considered saving the SessionID in script one so that script two can use it? I previously posted an option for Chrome, which I'm guessing should also work for Edge --

 

[JimmyBeam]

Hey,

this is my first time using the webdriver udf - so far its brilliant. I am currently stuck at a problem where I cannot find an answer in this forum.

I want to automate some things on a website which heavily relies on ajax. For this I need to execute some scripts which I tested via the chrome developer console.

$scriptTest = _WD_ExecuteScript($sSession, FileRead(@ScriptDir & '\javascript\scriptTest.js'))
MsgBox(0, "Info", $scriptTest)

I want to get the response from my script as I do a post request. The only response I get is {"value":null} which is not the response I was waiting for because I need the token from the response to move on.

I attached an example of what I mean with response. It is basically a XHR Request where I need to receive the response. 

I hope this is possible because I could not find a solution. Thanks in advance!

[Danp2]

@JimmyBeam _WD_ExecuteScript supports a few optional parameters, one of which is $lAsync. You probably need to use this to retrieve the ajax response.

Your post is pretty generic without a lot of specifics, so you will need to provide more details if you need additional assistance.

[JimmyBeam]

3 hours ago, Danp2 said:

@JimmyBeam _WD_ExecuteScript supports a few optional parameters, one of which is $lAsync. You probably need to use this to retrieve the ajax response.

Your post is pretty generic without a lot of specifics, so you will need to provide more details if you need additional assistance.

Thanks for your fast response. I read about the async requests after your post to understand it.

I do not have a callback function in my javascript so I do not know if async is working. Sadly I do not have that much javascript know-how. When I perform the action manually I can see that sometimes it takes up to a second to get the response.

Without editing my code the console output with $IAsync looks like this

$scriptTest = _WD_ExecuteScript($sSession, FileRead(@ScriptDir & '\javascript\scriptTest.js'), '', True)

__WD_Post: URL=HTTP://127.0.0.1:9515/session/f14c4e6a5686c9eeb294dd65948065e9/execute/async; $sData={"script":"fetch(\"https://url-from-website.com/index.php?page=xyz&ajax=1\", {
  \"headers\": {
    \"content-type\": \"application/x-www-form-urlencoded; charset=UTF-8\",
    \"x-requested-with\": \"XMLHttpRequest\"
  },
  \"body\": \"token=\" + Token,
  \"method\": \"POST\"
});", "args":[]}
__WD_Post: StatusCode=500; ResponseText={"value":{"error":"script timeout","message":"script timeout\n  (Session info: chrome=85.0.4183.102)...
__WD_Post ==> Webdriver Exception: {"value":{"error":"script timeout","message":"script timeout\n  (Session info: chrome=85.0.4183.102)","stacktrace":"Backtrace:\n\tOrdinal0 [0x009AD383+3134339]\n\tOrdinal0 [0x0089A171+2007409]\n\tOrdinal0 [0x0073AD90+568720]\n\tOrdinal0 [0x006E9266+234086]\n\tOrdinal0 [0x006DEC5D+191581]\n\tOrdinal0 [0x006E8773+231283]\n\tOrdinal0 [0x006DEB0B+191243]\n\tOrdinal0 [0x006C2E77+77431]\n\tOrdinal0 [0x006C3E3E+81470]\n\tOrdinal0 [0x006C3DC9+81353]\n\tOrdinal0 [0x008B0CD9+2100441]\n\tGetHandleVerifier [0x00B1B75A+1396954]\n\tGetHandleVerifier [0x00B1B3D9+1396057]\n\tGetHandleVerifier [0x00B27126+1444518]\n\tGetHandleVerifier [0x00B1BCE8+1398376]\n\tOrdinal0 [0x008A7F51+2064209]\n\tOrdinal0 [0x008B22EB+2106091]\n\tOrdinal0 [0x008B2411+2106385]\n\tOrdinal0 [0x008C49C4+2181572]\n\tBaseThreadInitThunk [0x756C6359+25]\n\tRtlGetAppContainerNamedObjectPath [0x77167C24+228]\n\tRtlGetAppContainerNamedObjectPath [0x77167BF4+180]\n"}}
_WD_ExecuteScript: {"value":{"error":"script timeout","message":"script timeout\n  (Session info: chrome=85.0.4183.102)...
_WD_ExecuteScript ==> Webdriver Exception: HTTP status = 500

and the console output of a sync request looks like this

__WD_Post: URL=HTTP://127.0.0.1:9515/session/9428d5b22b0e77db2d2ae66274fdfae7/execute/sync; $sData={"script":"fetch(\"https://url-to-website.com/index.php?page=xyz&ajax=1\", {
  \"headers\": {
    \"content-type\": \"application/x-www-form-urlencoded; charset=UTF-8\",
    \"x-requested-with\": \"XMLHttpRequest\"
  },
  \"body\": \"token=\" + Token,
  \"method\": \"POST\"
});", "args":[]}
__WD_Post: StatusCode=200; ResponseText={"value":null}...
_WD_ExecuteScript: {"value":null}...

This is my simple javascript I want to execute

fetch("https://url-to-website.com/index.php?page=xyz&ajax=1", {
  "headers": {
    "content-type": "application/x-www-form-urlencoded; charset=UTF-8",
    "x-requested-with": "XMLHttpRequest"
  },
  "body": "token=" + Token,
  "method": "POST"
});

I use a variable to get the token which is embedded in the html source. For further requests I always need the response with the new token.

The script is getting executed correctly no matter if sync or async. But both methods dont give me the response I need.

I am doing researches and testing for atleast 3 hours now but cant get a result.

[Danp2]

@JimmyBeam  You are still being intentionally vague. Provide the actual URL if you want help from the forum. Otherwise, you are just wasting our time.

[JimmyBeam]

9 hours ago, Danp2 said:

@JimmyBeam  You are still being intentionally vague. Provide the actual URL if you want help from the forum. Otherwise, you are just wasting our time.

I am sorry for not being vague enough. I think I delivered enough information. Anyway I solve the problem because I was to stupid to declare a return value in my javascript. 😆

Thank you anyway and please keep up the good work on this awesome UDF! 🙂

[NassauSky]

Hi all,

Using Webdriver to click on a link opens a new tab. 

It was mentioned earlier in this thread to use:

_WD_Window($sSession, 'Switch', '{"handle":"' & $YOUR_TAB_HANDLE_GOES_HERE & '"}')

to change tabs but how do I know the handle of the new tab that was created after I use:

_WD_ElementAction($sSession, $sElement, 'click')

Where can I find the new handle :-)

Thanks.

[Danp2]

@NassauSky You can use _WD_Attach to "connect" with the new tab. If that doesn't work for your situation, then take a look at the coding for _WD_NewTab and how it retrieves the new tab's handle.

[finAC]

Hi all, is there an easy way to retrieve the Frame-URL or identify which frame I'm currently in?

_WD_GetSource($sSession) always gets the main source!

The only thing i can think off is to loop through each frame and look for an expected element. Somehow that doesn't feel right!

Why on earth are there still websites using frames 🙂

 

expandcollapse popup

#include <UDF/wd_core.au3>
#include <UDF/wd_helper.au3>
$_WD_DEBUG = $_WD_DEBUG_Error

_WD_Option('Driver', 'UDF/geckodriver.exe')
_WD_Option('DriverParams', '--log trace')
_WD_Option('Port', 4444)

Local $sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"browserName": "firefox", "acceptInsecureCerts":true}}}'
Local $sTargetURL = "https://myerponline.de/demosystem/security/Menu.html"
Local $pWD
Local $sSession, $sElement, $sFrames, $sContent

;Start webdriver
$pWD = ProcessExists("Geckodriver.exe")
If $pWD <> 0 Then ProcessClose($pWD)

$pWD = _WD_Startup()
If @error <> $_WD_ERROR_Success Then Exit -1

;Open Session
$sSession = _WD_CreateSession($sDesiredCapabilities)
_WD_Navigate($sSession, $sTargetURL)
;_WD_WaitElement($sSession, $_WD_LOCATOR_ByXPath, "//td[@id='folderCell2_800237']", 50, 5000) ;Probably times out due to wrong frame

MsgBox(0, "How many frames", _WD_GetFrameCount($sSession))
For $i = 0 To _WD_GetFrameCount($sSession) - 1
    _WD_FrameEnter($sSession, $i)
    $sContent &= "FrameURL " &$i &": " &_WD_Action($sSession, "url") &@CRLF
    ;MsgBox(0, "", _WD_GetSource($sSession))
    _WD_FrameLeave($sSession)
Next
MsgBox(0, "", $sContent)

Local $sFrames = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//frame", Default, True)
If VarGetType($sFrames) = "Array" Then
    For $i = 0 To UBound($sFrames) - 1
        _WD_FrameEnter($sSession, $sFrames[$i])
        MsgBox(0, "", _WD_GetSource($sFrames[$i]))
        MsgBox(0, "", _WD_Action($sFrames[$i], "url"))
        _WD_FrameLeave($sSession)
    Next
EndIf

_WD_DeleteSession($sSession)
_WD_Shutdown($pWD)