WebDriver UDF (W3C compliant version) - 2025/09/01

[Parsix]

On 10/16/2025 at 8:55 PM, Parsix said:

problem 3:

 

3 - After calling the example function, the command after the example function will not be executed until the opened browser window is closed.

 

i checked all functions and find problem is _WinHttpSendRequest func in WinHttp.au3 and this line :

Local $aCall = DllCall($hWINHTTPDLL__WINHTTP, "bool", "WinHttpSendRequest", _
            "handle", $hRequest, _
            "wstr", $sHeaders, _
            "dword", 0, _
            "ptr", $pOptional, _
            "dword", $iOptionalLength, _
            "dword", $iTotalLength, _
            "dword_ptr", $iContext)

it is very slow 

Anybody have solution?

 

problem

 

[NMS]

I'm unsure if take make a thread or not so for now I'll post here. I'm having an issue with a website blocking chromedriver. By default I'm hoping to run it in headless mode so it looks for data in the background. However one of the websites is throwing me a "ERR_TOO_MANY_REDIRECTS".

So far I tried:
Opening main page first as no issues there and then proceeded to the desired page (https://www.mandarake.co.jp/).
Adding cookies.
Opening with user profile.

What I'm trying to achieve is getting information on whether the item is in stock from various websites. The database containing products and their direct sale page links are in the main script but the chromedriver is ran through a child process via IPC where that process will sit idle, and throughout the hour or so that it will need, check every product that I have. Beyond asking AI, I don't know what else to try.

#include 'wd_capabilities.au3'
#include 'wd_core.au3'
#include 'wd_helper.au3'

$_WD_DEBUG = $_WD_DEBUG_None
Local $TempWindowMode = '"--no-sandbox", ' & _
                        '"--disable-gpu", ' & _
                        '"--window-size=1920x1080", ' & _
                        '"--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36", ' & _
                        '"--allow-running-insecure-content"'
;~ $TempWindowMode &= ', "--headless"'
_WD_Option('Driver', @ScriptDir & '\Data\chromedriver.exe')
_WD_Option('Port', 9515)
_WD_Option('DriverParams', '--port=9515 --verbose --log-path="' & @ScriptDir & '\Tmp\ChromeDriverLog.log"')
Local $TempCapabilities = '{"capabilities": {"alwaysMatch": {"goog:chromeOptions": {"w3c": true, "args": [' & $TempWindowMode & ']}}}}'

_WD_Startup()
Local $TempSession = _WD_CreateSession($TempCapabilities)
If @error Then
    _WD_Shutdown()
    Exit
ElseIf @error = $_WD_ERROR_Success Then
    _WD_Navigate($TempSession, 'https://order.mandarake.co.jp/order/detailPage/item?itemCode=1254832487')
    If @error Then
        _WD_DeleteSession($TempSession)
        _WD_Shutdown()
        Exit
    EndIf
    _WD_LoadWait($TempSession, 5000)
EndIf

_WD_DeleteSession($TempSession)
_WD_Shutdown()
Exit

Hopefully this isn't against rules since the use-case isn't nefarious.

[mLipok]

  Arguments not works this way

1 hour ago, NMS said:

....
....
Local $TempWindowMode = '"--no-sandbox", ' & _
                        '"--disable-gpu", ' & _
                        '"--window-size=1920x1080", ' & _
                        '"--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36", ' & _
                        '"--allow-running-insecure-content"'
;~ $TempWindowMode &= ', "--headless"'
....
....
....
Local $TempCapabilities = '{"capabilities": {"alwaysMatch": {"goog:chromeOptions": {"w3c": true, "args": [' & $TempWindowMode & ']}}}}'
....
....

 

 

Take a look here:

https://github.com/mlipok/Au3WebDriver-testing/blob/main/wd_testing_helper.au3#L160-L209

 

 

[NMS]

@mLipok, from what I understand those arguments in the end are just a json string. Regardless of how they are written, as long as parsed correctly (I assume I wrote them well). But I did add every single option just to test from the link and it returned same result - too many redirects.

[mLipok]

Your JSON capabillities looks like:

{
    "capabilities":{
        "alwaysMatch": {
            "goog:chromeOptions": {
                "w3c": true,
                "args": [
                    "--no-sandbox",
                    "--disable-gpu",
                    "--window-size=1920x1080",
                    "--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36",
                    "--allow-running-insecure-content"
                ]
            }
        }
    }
}

I think you must to add:

Quote

"browserName":"chrome",

for example take a look for https://github.com/mlipok/Au3WebDriver-testing/blob/main/wd_testing_Example.au3

In this linked example you should see in SciTE console how capabilities look like .
Compare them with yours.

 

Edited December 29, 2025 by mLipok

[mLipok]

2 hours ago, NMS said:

'"--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36", ' & _

btw.

did you try:

user-agent=
instead
--user-agent=
 

[NMS]

Very well. I placed the desired URL into the test file and ran it with the exact same outcome - too many redirects.

Look, I understand that my arguments for webdriver might be flawed, but the script works on every other website I have, except for this one because it has built-in protection in a form of a redirection loop that causes webdrivers to not work (according to info I gathered around on google). I'm just trying to see if there's a parameter or something that would stop that loop or load the main page first then redirect to the desired product page. But I know nothing about web development so this is a dead-end for me.

[mLipok]

 

3 hours ago, NMS said:

Beyond asking AI, I don't know what else to try.

Just modified your code a little to be runable (Au3Check support and driver update) and debug in full mode:

expandcollapse popup

;~ https://www.autoitscript.com/forum/topic/208633-webdriver-udf-w3c-compliant-version-20250901/page/56/#findComment-1548510
#include "wd_capabilities.au3"
#include "wd_core.au3"
#include "wd_helper.au3"

$_WD_DEBUG = $_WD_DEBUG_Full
_Example()
Func _Example()
    Local $TempWindowMode = '"--no-sandbox", ' & _
            '"--disable-gpu", ' & _
            '"--window-size=1920x1080", ' & _
            '"--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36", ' & _
            '"--allow-running-insecure-content"'
;~ $TempWindowMode &= ', "--headless"'
    _WD_Option('Driver', @ScriptDir & '\chromedriver.exe')
    _WD_Option('Port', 9515)
    DirCreate(@ScriptDir & '\Tmp')
    _WD_Option('DriverParams', '--port=9515 --verbose --log-path="' & @ScriptDir & '\Tmp\ChromeDriverLog.log"')
    Local $TempCapabilities = '{"capabilities": {"alwaysMatch": {"goog:chromeOptions": {"w3c": true, "args": [' & $TempWindowMode & ']}}}}'

    _WD_UpdateDriver('chrome', Default, False, True)
    _WD_Startup()
    Local $TempSession = _WD_CreateSession($TempCapabilities)
    If @error Then
        _WD_Shutdown()
        Exit
    ElseIf @error = $_WD_ERROR_Success Then
        _WD_Navigate($TempSession, 'https://order.mandarake.co.jp/order/detailPage/item?itemCode=1254832487')
;~      _WD_Navigate($TempSession, 'https://google.com')
        If @error Then
            _WD_DeleteSession($TempSession)
            _WD_Shutdown()
            Exit
        EndIf
        _WD_LoadWait($TempSession, 5000)
    EndIf

    MsgBox($MB_TOPMOST, "TEST #" & @ScriptLineNumber, 'END')
    _WD_DeleteSession($TempSession)
    _WD_Shutdown()
    Exit
EndFunc   ;==>_Example

it works as it should works as using the given url:   https://order.mandarake.co.jp/order/detailPage/item?itemCode=1254832487 in any other browser always end in the same way with the same message.

3 hours ago, NMS said:

websites is throwing me a "ERR_TOO_MANY_REDIRECTS".

I suppose that it is related to the fact that it requrires to login to the page at first and possibly do other related stuff specific to the page due to the security reason.
As for me it is normal behavior of this website.

[NMS]

9 hours ago, mLipok said:

I suppose that it is related to the fact that it requrires to login to the page at first and possibly do other related stuff specific to the page due to the security reason.
As for me it is normal behavior of this website.

And that is exactly my point. There was nothing wrong with my script to begin with, the issue is the website itself.

As I said, I tried adding cookies, opening with user profile and opening main page (which works) and then going to product-specific one. However beyond that I haven't found a solution yet.

[mLipok]

1 hour ago, NMS said:

However beyond that I haven't found a solution yet.

Login to the page before you try to :

      _WD_Navigate($TempSession, 'https://order.mandarake.co.jp/order/detailPage/item?itemCode=1254832487')

 

[NMS]

There's a cloudflare protection on login. Every single time it even throws me a cloudflare check on first login. Only can log in after 2 tries. Not sure if a bug or by design to combat bots but it's irritating and genius at the same time. And I'm talking here about normal login.

 

Edit:
If anyone ever finds this useful, what solved my issue was a bit of element clicking to "manually" navigate the website.
 

_WD_Navigate($TempSession, 'https://mandarake.co.jp/') ;Load main page to get language selection
_WD_LoadWait($TempSession, 1000)
Local $TempLink = _WD_FindElement($TempSession, $_WD_LOCATOR_ByCSSSelector, "a[href='./index2.html']") ;Find element and click to go to JP website
_WD_ElementAction($TempSession, $TempLink, 'click')
_WD_LoadWait($TempSession, 1000)
_WD_Navigate($TempSession, 'https://order.mandarake.co.jp/order/detailPage/item?itemCode=1254832487') ;And go to desired product page

 

Edited December 31, 2025 by NMS