Validate Internet Domains

[water]

My company has registered about 400 internet domains.
Now I need to code a script to validate them.
This means:

• Grab a returned HTTP error code (e.g. 400 Bad Request (Invalid Hostname))
• Grab the target URL if the request was redirected

What's the best approach (existing UDF etc.) for this task?

Thanks a lot in advance.

[Danp2]

My first thought would be to avoid a browser and use the WinHTTP UDF.

[water]

Thanks for the quick reply.
That's what I try to do right now. Unfortunately I know near to nothing about HTTP.
I can successfully connect to the domain but don't know how to retrieve the reply (HTTP statusr 400 or 307/308 for a redirect).

[Danp2]

Which of the _WinHttp* functions are you using? Show us some code.

I'm sure there are some examples in the forum. FWIW, I'm using _WinHttpSimpleRequest, which returns the HTTP status in @extended, in the WebDriver UDF.

[water]

I use this script to connect to the AutoIt home page:

#include "WinHttp.au3"

Opt("MustDeclareVars", 1)

; Initialize and get session handle
Global $hOpen = _WinHttpOpen()
ConsoleWrite("HTTPOpen: @error = " & @error & ", @extended = " & @extended & @CRLF)

; Get connection handle
Global $hConnect = _WinHttpConnect($hOpen, "https://www.autoitscript.com")
ConsoleWrite("HTTPConn: @error = " & @error & ", @extended = " & @extended & @CRLF)

; Simple-request it...
Global $sResult = _WinHttpSimpleRequest($hConnect)
ConsoleWrite("HTTPRequ: @error = " & @error & ", @extended = " & @extended & @CRLF)
MsgBox(64, "Returned (first 1400 characters)", StringLeft($sResult, 1400) & " ...")

; Close handles
_WinHttpCloseHandle($hConnect)
_WinHttpCloseHandle($hOpen)

But all I get is:

Quote

HTTPOpen: @error = 0, @extended = 0
HTTPConn: @error = 0, @extended = 0
HTTPRequ: @error = 2, @extended = 0

Could this be caused by my company environment (Proxy etc.)?

[Danp2]

I got this --

HTTPOpen: @error = 0, @extended = 0
HTTPConn: @error = 0, @extended = 0
HTTPRequ: @error = 0, @extended = 400

Removing the "https://" gave me this --

HTTPOpen: @error = 0, @extended = 0
HTTPConn: @error = 0, @extended = 0
HTTPRequ: @error = 0, @extended = 200

 

[Simpel]

@water I get the same errors like you. I am at the company behind a proxy. But if I change my script to:

expandcollapse popup

#include "WinHttp.au3"

Opt("MustDeclareVars", 1)

; Proxy server
Global $sProxy = 
; Proxy user
Global $sUserName = 
; Proxy password
Global $sPassword = 

; Initialize and get session handle
Global $hOpen = _WinHttpOpen("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.10240", $WINHTTP_ACCESS_TYPE_NAMED_PROXY, $sProxy)
;~ Global $hOpen = _WinHttpOpen()
ConsoleWrite("HTTPOpen: @error = " & @error & ", @extended = " & @extended & @CRLF)

; Get connection handle
Global $hConnect = _WinHttpConnect($hOpen, "https://www.autoitscript.com")
ConsoleWrite("HTTPConn: @error = " & @error & ", @extended = " & @extended & @CRLF)

; Simple-request it...
Global $sResult = _WinHttpSimpleSendRequest_ProxyAuth($hConnect)
;~ Global $sResult = _WinHttpSimpleRequest($hConnect)
ConsoleWrite("HTTPRequ: @error = " & @error & ", @extended = " & @extended & @CRLF)
MsgBox(64, "Returned (first 1400 characters)", StringLeft($sResult, 1400) & " ...")

; Close handles
_WinHttpCloseHandle($hConnect)
_WinHttpCloseHandle($hOpen)


Func _WinHttpSimpleSendRequest_ProxyAuth($hConnect, $sType = Default, $sPath = Default, $sReferrer = Default, $sDta = Default, $sHeader = Default) ; by trancexx
    Local $hRequest = _WinHttpSimpleSendSSLRequest($hConnect, $sType, $sPath, $sReferrer, $sDta, $sHeader) ; Specify the reguest
    If $hRequest Then
        Local $iStatusCode = _WinHttpQueryHeaders($hRequest, $WINHTTP_QUERY_STATUS_CODE) ; Query status code
        If $iStatusCode = $HTTP_STATUS_PROXY_AUTH_REQ Then ; Check status code
            Local $iSupportedSchemes, $iFirstScheme, $iAuthTarget ; Query Authorization scheme
            If _WinHttpQueryAuthSchemes($hRequest, $iSupportedSchemes, $iFirstScheme, $iAuthTarget) Then
                _WinHttpSetCredentials($hRequest, $iAuthTarget, $iFirstScheme, $sUserName, $sPassword)  ; Set passed credentials
                _WinHttpSendRequest($hRequest) ; Send request again now
                _WinHttpReceiveResponse($hRequest) ; And wait for the response again
            EndIf
        EndIf
    EndIf
    Return $hRequest
EndFunc

I get all errors and extended 0. Maybe the proxy catches the returned http status.

Conrad

[water]

Thanks a lot for your support, guys.
This gets just too time consuming - so unfortunately no quick win

[ripdad]

If the domains don't use ssl, then perhaps you could use something like this...

Local $sAddress, $nError, $nSocket, $sRecv, $sData = ''
;
$sData &= 'HEAD / HTTP/1.1' & @CRLF
$sData &= 'HOST: www.irongeek.com' & @CRLF & @CRLF
;
TCPStartup()
;
$sAddress = TCPNameToIP('www.irongeek.com')
$nSocket = TCPConnect($sAddress, 80)
;
TCPSend($nSocket, $sData)
$nError = @error
;
If Not $nError Then
    For $i = 1 To 200
        $sRecv = TCPRecv($nSocket, 256)
        $nError = @error
        ;
        If $nError Or $sRecv <> '' Then
            ExitLoop
        EndIf
        Sleep(10)
    Next
EndIf
;
TCPCloseSocket($nSocket)
TCPShutdown()
;
MsgBox(0, 'Error: ' & $nError, $sRecv)
;

 

Otherwise, would Ping() be sufficient?

 

[water]

Thanks for your reply!
I wanted to open an URL.

• If an error gets returned because there is no server replying or some other kind of error happened then I wanted to write this information to a file
• If a page of the requested URL gets opened I wanted to write this to a file
• if the URL gets redirected to another site I wanted to log this redirected site

But I think we will check the URLs by hand as writing a script takes much longer