Jump to content

WebDriver UDF - Help & Support (II)


Danp2
 Share

Recommended Posts

Yes, I discovered this, too. It seems I can't get the page in autoit faster like in plain JS... the script is left free faster after 7ms because the ajax call is made and it is async, but if I wait the text coming to autoit I get longer timing, similar to loading pages in tabs...

Calculating timing into JS with ajax, using chrome console, I get 2 pages loaded in 134ms ( using on success: t0 = performance.now(), t1= performance.now() and t1-t0 )

But in Autoit with

$answer = _WD_ExecuteScript($sSession, "return myPage;")

as you, I get more than 1.5s to get the text of a single page! Let's say I could load 60 pages sometimes in a single cycle...

Any idea why ajax on JS page is  SO faster (and if we can fast it a bit optimizing autoit)?

EDIT:

I mean, 70ms vs 1500ms is a very BIG difference!! This is plain after JS success func called.

Instead, in JS, timing after putting text into a JS var, becomes 1300ms for 2 pages, that is 650ms per page.

 

EDIT2

Anyway, making a regExp in JS after the ajax calls, I get 867ms for 2 pages, that is 433ms per page having also some data I need (without assigning all the text page to another global var, directly using the var of the success func).

Edited by frank10
Link to comment
Share on other sites

I imagine the time different can be chalked up to the overhead of the additional layer when using webdriver to communicate with the browser.

Here's a simpler implementation that performs a synchronous call --

$jqueryCommand = "return jQuery.get('www.amazon.com/dp/B07H71XXGY', function(data){ return data.responseText; });"
$answer = _WD_ExecuteScript($sSession, $jqueryCommand)

 

Link to comment
Share on other sites

Ok, but we'll remain about 2.1s per page... too much.

But it remains strange that making inet or winhttp, that is direct receiving data from the net, instead of communicating with the browser, you get slower results than an ajax call from the browser, that is always a http request.

Maybe I must make all my call and parse inside JS and at the end pass one short var to autoit.

Link to comment
Share on other sites

This does not work:

local $jqueryCommand = " jQuery.parseHTML( '<b>aaa</b>' );"
    _WD_ExecuteScript($sSession, $jqueryCommand )

__WD_Post: URL=HTTP://127.0.0.1:9515/session/1a0dc618d5228451c569f4b78860e3bd/execute/sync; $sData={"script":" jQuery.parseHTML( '<b>a</b>' );", "args":[[]]}
__WD_Post: StatusCode=500; ResponseText={"value":{"error":"javascript error","message":"javascript error: jQuery.parseHTML is not a function\n  (Session info:....

Instead in Chrome console it works (after manually inserting jQuery script... why? Shouldn't the page already have the <script> jquery injected from autoit _WD_jQuerify ?).

Edited by frank10
Link to comment
Share on other sites

No, it's simple script like before where you load a page, add _WD_jQuerify and then that line with .parseHTML and does not work.

EDIT

Like this:

#include "wd_helper.au3"
#include "wd_core.au3"

OnAutoItExitRegister("_onExit")

Local $sDesiredCapabilities

$_WD_DEBUG =  True ; You could also use $_WD_DEBUG_Error
;~ You can also control the visibility of the console with the function _WD_ConsoleVisible.

_WD_Option('Driver', 'chromedriver.exe')
_WD_Option('Port', 9515)
_WD_Option('DriverParams', '--log-path=' & @ScriptDir & '\chrome.log')

$sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"goog:chromeOptions": {"w3c": true, "args":["--user-data-dir=C:\\Users\\' & @UserName & '\\AppData\\Local\\Google\\Chrome\\User Data\\", "--profile-directory=Default"]}}}}'

_WD_Startup()

$sSession = _WD_CreateSession($sDesiredCapabilities)

If @error = $_WD_ERROR_Success Then
    _WD_Navigate($sSession, "https://www.amazon.com/dp/B07H71XXGY/")
    _WD_LoadWait($sSession)
    _WD_jQuerify($sSession)

    local $jqueryCommand = " jQuery.parseHTML( '<b>aaa</b>' );"
    _WD_ExecuteScript($sSession, $jqueryCommand )
    
    MsgBox(0,'',"")
EndIf

Func _onExit()
    _WD_DeleteSession($sSession)
    _WD_Shutdown()
EndFunc

 

Edited by frank10
Link to comment
Share on other sites

If you examine that page in a browser, you'll find that it already has an older version of jQuery loaded. You can determine the exact version by issuing the command "jQuery.fn.jquery" in the browser console, which for me returns 1.6.4. Since the ParseHTML method wasn't added until v1.8, this explains why it isn't working in this case.

You can force the newer version of JQuery to load by running the following command before calling _WD_jQuerify --

_WD_ExecuteScript($sSession, "jQuery = undefined;")

 

Link to comment
Share on other sites

On 4/30/2020 at 7:40 PM, Danp2 said:

I'm glad that you found something that works for you. However, it's still unclear which commands that you tried before that resulted in a long pause. To be honest, I don't see why _WD_Timeouts would be faster that other commands, such as _WD_Status, _WD_Action, etc.

Also, you would be better off checking the value of @error since the value returned during an error condition is subject to change in the future.

Good idea to take @error!
I tested WD_Timeouts and WD_Action both working fine and no difference in speed, WD_Status however does not function at all. So my function to test if a session no more exists and I have to open a new session (because I don't want and I am not able to start a second session with FF and same profile). 

Func _WD_SessionNotExists($sSession)  ; Check if session still exists
    _WD_Action($sSession,"refresh")    ; or _WD_Timeouts($sSession)  
    Return @error
EndFunc   ; ==>_WD_SessionExists

 If you have an idea to improve, please...

Link to comment
Share on other sites

Is it possible to reconnect to the old session after closing a script, if you remember the $sSession? For example, prog1.au3 opens a Web page, saves $sSession to a file, and exits. Prog2.au3 starts, reads $sSession from the file, and connects to the web page opened by prog1.au3.

Link to comment
Share on other sites

11 hours ago, HJL said:

WD_Status however does not function at all.

Can you elaborate on this?

12 hours ago, HJL said:

Is it possible to reconnect to the old session after closing a script, if you remember the $sSession?

Yes. If the chromedriver console is still active, you should be able to continue interaction with it from an additional script. You would perform your normal steps, but skip the calls to _WD_Startup and _WD_CreateSession.

Link to comment
Share on other sites

3 hours ago, Danp2 said:

Can you elaborate on this?

 

image.png.42eac979eebff52fb05d6ed34d158b8f.png

WD_Status returns @extended = 200 even if the $sSession ist changed or the session is closed. But there must be an error in my test script to measure the time because @error is always reported as 0 and in reality all except wd_status report @error <> 0.

 

#include "wd_core.au3"
#include "wd_helper.au3"
#include <MsgBoxConstants.au3>
;MsgBox($MB_TOPMOST,"Errors", "@error:" & @error & @CRLF & "@extended:" & @extended & @CRLF)

;$_WD_DEBUG = False

Local $sDesiredCapabilities, $sSession, $sURL
Local $sE1 = "WD_Timeout:         @error/                  Time" & @CRLF & @tab & @TAB & "@extended" & @CRLF
Local $sE2 = "WD_Status:"   & @CRLF
Local $sE3 = "WD_Action:"   & @CRLF
Local $sE4 = "WD_Navigate:" & @CRLF

Test()  ; Exit with "Esc"

Func Test()
    SetUpGecko()
    _WD_Startup()
    $sSession = _WD_CreateSession($sDesiredCapabilities)
    _WD_LoadWait($sSession, 3000)
    _WD_Navigate($sSession, "http://wdr.de")
    _WD_LoadWait($sSession, 3000)
    TestTime("Normal:      ")
    _WD_LoadWait($sSession, 3000)
    Local $sSessionSave = $sSession
    $sSession = " / Dummy"
    TestTime("Session=Dummy: ")
    _WD_DeleteSession($sSessionSave)
    TestTime("SessionDeleted: ")
    MsgBox($MB_SYSTEMMODAL, "Time Differences", $sE1 & @CRLF & $sE2 & @CRLF  & $sE3 & @CRLF & $sE4 & @CRLF)
    _WD_ShutDown()
EndFunc  ;==> ReadLoop

Func TestTime($sText)
    Local $hTimer = TimerInit()
    _WD_Timeouts($sSession)
    $sE1 = $sE1 & $sText & @TAB & @error&"/"&@extended & @TAB & @TAB & TimerDiff($hTimer) & @CRLF

    Local $hTimer = TimerInit()
    _WD_Status()
    $sE2 = $sE2 & $sText & @TAB & @error&"/"&@extended & @TAB & @TAB & TimerDiff($hTimer) & @CRLF

    Local $hTimer = TimerInit()
    _WD_Action($sSession, "refresh")
    $sE3 = $sE3 & $sText & @TAB & @error&"/"&@extended & @TAB & @TAB & TimerDiff($hTimer) & @CRLF

    Local $hTimer = TimerInit()
    _WD_Navigate($sSession, "http:sdr.de")
    $sE4 = $sE4 & $sText & @TAB & @error&"/"&@extended & @TAB & @TAB & TimerDiff($hTimer) & @CRLF

EndFunc

Func SetupGecko()
    _WD_Option('Driver', 'geckodriver.exe')
    _WD_Option('DriverParams', '--log trace --marionette-port 2828')
    _WD_Option('Port', 4444)
    $sDesiredCapabilities = '{"capabilities":{"alwaysMatch": {"moz:firefoxOptions": {"args": ["-profile", "C:\\webdriver\\FF_WD_Profile"]}}}}'
EndFunc

 

Edited by HJL
Link to comment
Share on other sites

There are some issues with your code, as can be seen in the console output --

__WD_Get: URL=HTTP://127.0.0.1:4444/session/ / Dummy/timeouts
__WD_Get: StatusCode=405; $iResult = 0; $sResponseText=HTTP method not allowed...

I'll review the functionality of _WD_Status, but you can see that it does return useful info --

__WD_Get: URL=HTTP://127.0.0.1:4444/status
__WD_Get: StatusCode=200; $iResult = 0; $sResponseText={"value":{"message":"Session already started","ready":false}}...

__WD_Get: URL=HTTP://127.0.0.1:4444/status
__WD_Get: StatusCode=200; $iResult = 0; $sResponseText={"value":{"message":"","ready":true}}...

Here's a modified version of your code --

#include "wd_core.au3"
#include "wd_helper.au3"
#include <MsgBoxConstants.au3>

Local $sDesiredCapabilities, $sSession, $sURL
Local $sE1 = "WD_Timeout:         @error/                  Time" & @CRLF & @tab & @TAB & "@extended" & @CRLF
Local $sE2 = "WD_Status:"   & @CRLF
Local $sE3 = "WD_Action:"   & @CRLF
Local $sE4 = "WD_Navigate:" & @CRLF

Test()  ; Exit with "Esc"

Func Test()
    SetUpGecko()
    _WD_Startup()
    $sSession = _WD_CreateSession($sDesiredCapabilities)
    _WD_LoadWait($sSession, 3000)
    _WD_Navigate($sSession, "http://wdr.de")
    _WD_LoadWait($sSession, 3000)
    TestTime("Normal:      ")
    _WD_LoadWait($sSession, 3000)
    Local $sSessionSave = $sSession
    $sSession = "Dummy"
    TestTime("Session=Dummy: ")
    _WD_DeleteSession($sSessionSave)
    $sSession = $sSessionSave
    TestTime("SessionDeleted: ")
    MsgBox($MB_SYSTEMMODAL, "Time Differences", $sE1 & @CRLF & $sE2 & @CRLF  & $sE3 & @CRLF & $sE4 & @CRLF)
    _WD_ShutDown()
EndFunc  ;==> ReadLoop

Func TestTime($sText)
    Local $hTimer = TimerInit()
    _WD_Timeouts($sSession)
    $sE1 = $sE1 & $sText & @TAB & @error&"/"&@extended & @TAB & @TAB & TimerDiff($hTimer) & @CRLF

    Local $hTimer = TimerInit()
    _WD_Status()
    $sE2 = $sE2 & $sText & @TAB & @error&"/"&@extended & @TAB & @TAB & TimerDiff($hTimer) & @CRLF

    Local $hTimer = TimerInit()
    _WD_Action($sSession, "refresh")
    $sE3 = $sE3 & $sText & @TAB & @error&"/"&@extended & @TAB & @TAB & TimerDiff($hTimer) & @CRLF

    Local $hTimer = TimerInit()
    _WD_Navigate($sSession, "http:sdr.de")
    $sE4 = $sE4 & $sText & @TAB & @error&"/"&@extended & @TAB & @TAB & TimerDiff($hTimer) & @CRLF

EndFunc

Func SetupGecko()
    _WD_Option('Driver', 'geckodriver.exe')
    _WD_Option('DriverParams', '--log trace --marionette-port 2828')
    _WD_Option('Port', 4444)
    $sDesiredCapabilities = '{"capabilities":{"alwaysMatch": {"moz:firefoxOptions": {"args": ["-profile", "C:\\webdriver\\FF_WD_Profile"]}}}}'
;   $sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"browserName": "firefox", "acceptInsecureCerts":true}}}'
EndFunc

 

Link to comment
Share on other sites

I'm attempting to force PDFs to download in Chrome instead of viewing them. There are already lots of questions about this, where the answer is usually to add something to the capabilities. I've attempted to do this, but it seems it isn't supported anymore. This is the format of my capabilities string:

Spoiler
; I test with one option un-commented and the other removed
    Local $sDesiredCapabilities = _
        '{"capabilities":{' & _
            '"alwaysMatch": {' & _
                '"goog:chromeOptions": {' & _
;~                  '"plugins.plugins_disabled": ["Chrome PDF Viewer"],' & _
;~                  '"plugins.always_open_pdf_externally": true,' & _
                    '"w3c": true, ' & _
                    '"args":[' & _
;~                      '"--disable-extensions",' & _
                        '"--profile-directory=Default"' & _
                    ']' & _
                '}' & _
            '}' & _
        '}' & _
    '}'

Un-commenting either of the first two lines makes the session not be created (_WD_SessionCreate returns an exception). The command line argument does nothing and PDFs are displayed in the browser

Instead, I'm trying to load chrome://settings/content/pdfDocuments and change the setting every session with a click on the toggle. Of course, this isn't simple either, as it deals with ShadowDOM elements. It was a bit tricky at first, but I thought I got it down. After hitting the 3rd ShadowRoot object, however, it seems that JavaScript isn't loaded anymore as _WD_GetShadowRoot times out with this error in the console:

Spoiler
__WD_Post: URL=HTTP://127.0.0.1:9515/session/373186151c5b32925a77dfc38463c9d3/element/1cd884b1-c260-4496-97cc-e25fd0c60f13/element; $sData={"using":"tag name","value":"settings-basic-page"}
__WD_Post: StatusCode=500; ResponseText={"value":{"error":"javascript error","message":"javascript error: b.getElementsByTagName is not a function\n  (Session info: chrome=80.0.3987.100)","stacktrace":"Backtrace:\n\tOrdinal0 [0x00B10C83+1707139]\n\tOrdinal0 [0x00A768F1+1075441]\n\tOrdinal0 [0x009EDFC9+516041]\n\tOrdinal0 [0x009EFC75+523381]\n\tOrdinal0 [0x009EFB64+523108]\n\tOrdinal0 [0x009EFD20+523552]\n\tOrdinal0 [0x00988A82+100994]\n\tOrdinal0 [0x00983991+80273]\n\tOrdinal0 [0x00999ED0+171728]\n\tOrdinal0 [0x00983916+80150]\n\tOrdinal0 [0x0099A0E1+172257]\n\tOrdinal0 [0x009A30F4+209140]\n\tOrdinal0 [0x00999D4B+171339]\n\tOrdinal0 [0x00981D4A+73034]\n\tOrdinal0 [0x00982DC0+77248]\n\tOrdinal0 [0x00982D59+77145]\n\tOrdinal0 [0x00A8BB67+1162087]\n\tGetHandleVerifier [0x00BAA966+508998]\n\tGetHandleVerifier [0x00BAA6A4+508292]\n\tGetHandleVerifier [0x00BBF7B7+594583]\n\tGetHandleVerifier [0x00BAB1D6+511158]\n\tOrdinal0 [0x00A8402C+1130540]\n\tOrdinal0 [0x00A8D4CB+1168587]\n\tOrdinal0 [0x00A8D633+1168947]\n\tOrdinal0 [0x00AA5B35+1268533]\n\tBaseThreadInitThunk [0x76830419+25]\n\tRtlGetAppContainerNamedObjectPath [0x771C66DD+237]\n\tRtlGetAppContainerNamedObjectPath [0x771C66AD+189]\n"}}
__WD_Post ==> Timeout: {"value":{"error":"javascript error","message":"javascript error: b.getElementsByTagName is not a function\n  (Session info: chrome=80.0.3987.100)","stacktrace":"Backtrace:\n\tOrdinal0 [0x00B10C83+1707139]\n\tOrdinal0 [0x00A768F1+1075441]\n\tOrdinal0 [0x009EDFC9+516041]\n\tOrdinal0 [0x009EFC75+523381]\n\tOrdinal0 [0x009EFB64+523108]\n\tOrdinal0 [0x009EFD20+523552]\n\tOrdinal0 [0x00988A82+100994]\n\tOrdinal0 [0x00983991+80273]\n\tOrdinal0 [0x00999ED0+171728]\n\tOrdinal0 [0x00983916+80150]\n\tOrdinal0 [0x0099A0E1+172257]\n\tOrdinal0 [0x009A30F4+209140]\n\tOrdinal0 [0x00999D4B+171339]\n\tOrdinal0 [0x00981D4A+73034]\n\tOrdinal0 [0x00982DC0+77248]\n\tOrdinal0 [0x00982D59+77145]\n\tOrdinal0 [0x00A8BB67+1162087]\n\tGetHandleVerifier [0x00BAA966+508998]\n\tGetHandleVerifier [0x00BAA6A4+508292]\n\tGetHandleVerifier [0x00BBF7B7+594583]\n\tGetHandleVerifier [0x00BAB1D6+511158]\n\tOrdinal0 [0x00A8402C+1130540]\n\tOrdinal0 [0x00A8D4CB+1168587]\n\tOrdinal0 [0x00A8D633+1168947]\n\tOrdinal0 [0x00AA5B35+1268533]\n\tBaseThreadInitThunk [0x76830419+25]\n\tRtlGetAppContainerNamedObjectPath [0x771C66DD+237]\n\tRtlGetAppContainerNamedObjectPath [0x771C66AD+189]\n"}}
_WD_FindElement: {"value":{"error":"javascript error","message":"javascript error: b.getElementsByTagName is not a function\n  (Session info: chrome=80.0.3987.100)","stacktrace":"Backtrace:\n\tOrdinal0 [0x00B10C83+1707139]\n\tOrdinal0 [0x00A768F1+1075441]\n\tOrdinal0 [0x009EDFC9+516041]\n\tOrdinal0 [0x009EFC75+523381]\n\tOrdinal0 [0x009EFB64+523108]\n\tOrdinal0 [0x009EFD20+523552]\n\tOrdinal0 [0x00988A82+100994]\n\tOrdinal0 [0x00983991+80273]\n\tOrdinal0 [0x00999ED0+171728]\n\tOrdinal0 [0x00983916+80150]\n\tOrdinal0 [0x0099A0E1+172257]\n\tOrdinal0 [0x009A30F4+209140]\n\tOrdinal0 [0x00999D4B+171339]\n\tOrdinal0 [0x00981D4A+73034]\n\tOrdinal0 [0x00982DC0+77248]\n\tOrdinal0 [0x00982D59+77145]\n\tOrdinal0 [0x00A8BB67+1162087]\n\tGetHandleVerifier [0x00BAA966+508998]\n\tGetHandleVerifier [0x00BAA6A4+508292]\n\tGetHandleVerifier [0x00BBF7B7+594583]\n\tGetHandleVerifier [0x00BAB1D6+511158]\n\tOrdinal0 [0x00A8402C+1130540]\n\tOrdinal0 [0x00A8D4CB+1168587]\n\tOrdinal0 [0x00A8D633+1168947]\n\tOrdinal0 [0x00AA5B35+1268533]\n\tBaseThreadInitThunk [0x76830419+25]\n\tRtlGetAppContainerNamedObjectPath [0x771C66DD+237]\n\tRtlGetAppContainerNamedObjectPath [0x771C66AD+189]\n"}}
_WD_FindElement ==> Timeout: HTTP status = 500
_WD_GetShadowRoot: 
_WD_GetShadowRoot ==> Timeout

 

All my code provided is Public Domain... but it may not work. ;) Use it, change it, break it, whatever you want.

Spoiler

My Humble Contributions:
Personal Function Documentation - A personal HelpFile for your functions
Acro.au3 UDF - Automating Acrobat Pro
ToDo Finder - Find #ToDo: lines in your scripts
UI-SimpleWrappers UDF - Use UI Automation more Simply-er
KeePass UDF - Automate KeePass, a password manager
InputBoxes - Simple Input boxes for various variable types

Link to comment
Share on other sites

  • Jos locked this topic
Guest
This topic is now closed to further replies.
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...