Sign in to follow this  
Followers 0
Seminko

_INetGetSource not working

30 posts in this topic

Hey fellas,

I have a script that grabs data from hearthpwn.com.

It was working fine for half a year or so but now _INetGetSource started giving me an error. So I googled and came across WinHttp.au3 and this script:

#include <Constants.au3>
#include <WinHttp.au3>

; Designed by guinness 2013.
Global Const $HTTP_GOOD_STATUSCODE = '|' & $HTTP_STATUS_OK & '|' & $HTTP_STATUS_MOVED & '|' & $HTTP_STATUS_REDIRECT & '|' & _
        $HTTP_STATUS_REDIRECT_METHOD & '|' & $HTTP_STATUS_NOT_MODIFIED & '|' & $HTTP_STATUS_REDIRECT_KEEP_VERB & '|' & _
        $HTTP_STATUS_BAD_REQUEST & '|' & $HTTP_STATUS_DENIED & '|' & $HTTP_STATUS_FORBIDDEN & '|' & _
        $HTTP_STATUS_BAD_METHOD & '|'

Example()

Func Example()
    Local $iPort = 80, _
            $sDomain = 'www.hearthpwn.com'

    ; Open the handles.
    Local Const $hOpen = _WinHttpOpen()
    Local Const $hConnect = _WinHttpConnect($hOpen, $sDomain, $iPort)
    Local Const $hRequest = _WinHttpOpenRequest($hConnect, 'HEAD')


    ; Send a request.
    _WinHttpSendRequest($hRequest)

    ; Wait for a response.
    _WinHttpReceiveResponse($hRequest)

    ; Retrieve the http status and check if matches the OK status code.
    Local $sRequestCode = _WinHttpQueryHeaders($hRequest, $WINHTTP_QUERY_STATUS_CODE)

    ; Check if the status code is in the $HTTP_GOOD_STATUSCODE string.
    Local $fIsUp = StringInStr($HTTP_GOOD_STATUSCODE, '|' & $sRequestCode & '|') > 0

    ; Display value of $fIsUp.
    MsgBox($MB_SYSTEMMODAL, '', 'The return was: ' & $fIsUp & @CRLF)

    ; Close the handles.
    _WinHttpCloseHandle($hRequest)
    _WinHttpCloseHandle($hConnect)
    _WinHttpCloseHandle($hOpen)
EndFunc   ;==>Example

When I run this I get 'The return was: False' so I guess something is with the site itself? Is there any way around it?

Here is an example script of the _INetGetSource:

#include <INet.au3>

$Source = _INetGetSource("http://www.hearthpwn.com")
If @error Then
    MsgBox(1, "Error", "Error")
Else
    MsgBox(1, "", $Source)
EndIf

Thanks,

Jan

Share this post


Link to post
Share on other sites



#include <Constants.au3>
#include <WinHttp.au3>

; Designed by guinness 2013.
Global Const $HTTP_GOOD_STATUSCODE = '|' & $HTTP_STATUS_OK & '|' & $HTTP_STATUS_MOVED & '|' & $HTTP_STATUS_REDIRECT & '|' & _
        $HTTP_STATUS_REDIRECT_METHOD & '|' & $HTTP_STATUS_NOT_MODIFIED & '|' & $HTTP_STATUS_REDIRECT_KEEP_VERB & '|' & _
        $HTTP_STATUS_BAD_REQUEST & '|' & $HTTP_STATUS_DENIED & '|' & $HTTP_STATUS_FORBIDDEN & '|' & _
        $HTTP_STATUS_BAD_METHOD & '|'

Example()

Func Example()
    Local $iPort = 80, _
            $sDomain = 'www.hearthpwn.com'

    ; Open the handles.
    Local Const $hOpen = _WinHttpOpen()
    Local Const $hConnect = _WinHttpConnect($hOpen, $sDomain, $iPort)
    Local Const $hRequest = _WinHttpOpenRequest($hConnect)


    ; Send a request.
    _WinHttpSendRequest($hRequest)

    ; Wait for a response.
    _WinHttpReceiveResponse($hRequest)

    ; Retrieve the http status and check if matches the OK status code.
    Local $sRequestCode = _WinHttpQueryHeaders($hRequest, $WINHTTP_QUERY_STATUS_CODE)

    ; Check if the status code is in the $HTTP_GOOD_STATUSCODE string.
    Local $fIsUp = StringInStr($HTTP_GOOD_STATUSCODE, '|' & $sRequestCode & '|') > 0

    ; Display value of $fIsUp.
    MsgBox($MB_SYSTEMMODAL, '', 'The return was: ' & $fIsUp & @CRLF)

    ; Close the handles.
    _WinHttpCloseHandle($hRequest)
    _WinHttpCloseHandle($hConnect)
    _WinHttpCloseHandle($hOpen)
EndFunc   ;==>Example

Return True

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

16 minutes ago, rootx said:

Return True

Not on my end... just checked.

Ideas?

 

EDIT: the script you send is actually updated a bit and it does return true in fact. However, below still doesn't work. No idea why. The site is up and runs smoothly.

_INetGetSource("http://www.hearthpwn.com")
Edited by Seminko

Share this post


Link to post
Share on other sites

#4 ·  Posted (edited)

Ok, tried to see what error I get with @extended and apparently as per THIS thread the error means:

Quote

12039 ERROR_INTERNET_HTTP_TO_HTTPS_ON_REDIR
The application is moving from a non-SSL to an SSL connection because of a redirect.

Ok so logically I tried httpS://www.hearthpwn.com and now what we get is the same error the other way around:

Quote

12040 ERROR_INTERNET_HTTPS_TO_HTTP_ON_REDIR
The application is moving from an SSL to an non-SSL connection because of a redirect.

 

Edited by Seminko

Share this post


Link to post
Share on other sites

Really? Nobody has such error before? :(

Share this post


Link to post
Share on other sites

Guys? My script is dead without getting this sorted.

Noone's seen this happen ever?

Share this post


Link to post
Share on other sites

Seminko,

The site you are using looks a lot like a game to me - how about persuading me that it is not?

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

Share this post


Link to post
Share on other sites

Hi Melba,

I see you're not coming to rescue me. Not yet at least :)

I hope I've checked the most up to date rules found here --> https://www.autoitscript.com/forum/topic/169097-forum-rules/

The rules state:

Quote

1. Do not ask for help with AutoIt scripts, post links to, or start discussion topics on the following subjects:

  • Launching, automation or script interaction with games or game servers, regardless of the game.

My script does not launch, automate or in any way interact with a game or game servers.

The site I linked is related to a game, yes. However, my script only interacts with the site itself, it copies card information into an excel file instead of me spending hours doing it myself.

So PLEASE, guide me!

Very much obliged

 

Share this post


Link to post
Share on other sites

Seminko,

OK, I am convinced - but alas I cannot offer any help.

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

Share this post


Link to post
Share on other sites

#10 ·  Posted

Bumping again. There has to be a solution.

Share this post


Link to post
Share on other sites

#13 ·  Posted

6 hours ago, mikell said:

Hmm this works for me...

Msgbox(0,"", BinarytoString(InetRead("http://www.hearthpwn.com")))

 

Well, it doesn't return an error, yes. But what does it return for you? It returns 0 for me. I use _InetGetSource to get the source code.

 

6 hours ago, AndyG said:
#include <Inet.au3>
$a=_INetGetSource("http://www.hearthpwn.com")
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $a = ' & $a & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

shows me the sourcecode...what i am doing wrong?

Does it? All I get is:

@@ Debug(58) : $a = 
>Error code: 13

-------------

Mikell, could you try with _INetGetSource? If it works for you both than the issue has to be on my end. Maybe some internet option?

Share this post


Link to post
Share on other sites

#14 ·  Posted

My previous code - as does the one from AndyG I presume - returns the whole source code of the page
_INetGetSource is nothing but a clean  BinarytoString(InetRead(...))

You could try to set a non-AutoIt user agent using  HttpSetUserAgent()

Share this post


Link to post
Share on other sites

#15 ·  Posted

55 minutes ago, mikell said:

My previous code - as does the one from AndyG I presume - returns the whole source code of the page
_INetGetSource is nothing but a clean  BinarytoString(InetRead(...))

You could try to set a non-AutoIt user agent using  HttpSetUserAgent()

I tried below code but the result is still the same:

#include <Inet.au3>
HttpSetUserAgent("MyUserAgent")
$a=_INetGetSource("http://www.hearthpwn.com")
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $a = ' & $a & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

It's strange that I'm apparently the first one reporting such issue.

Share this post


Link to post
Share on other sites

#16 ·  Posted

What about this ?

$oHTTP = ObjCreate("Microsoft.XMLHTTP")
$oHTTP.Open("GET", "http://www.hearthpwn.com", 0)
$oHTTP.Send()
$sData = $oHTTP.ResponseText
Msgbox(0,"", $sData)

 

Share this post


Link to post
Share on other sites

#17 ·  Posted

8 hours ago, mikell said:

What about this ?

$oHTTP = ObjCreate("Microsoft.XMLHTTP")
$oHTTP.Open("GET", "http://www.hearthpwn.com", 0)
$oHTTP.Send()
$sData = $oHTTP.ResponseText
Msgbox(0,"", $sData)

 

The Msgbox is empty :(

Share this post


Link to post
Share on other sites

#18 ·  Posted (edited)

Let's play at lower level:

TCPStartup()
If @error Then _ReportError(@error, @extended, @ScriptLineNumber - 1)
Local $sIPAddress = TCPNameToIP("www.hearthpwn.com")
If @error Then _ReportError(@error, @extended, @ScriptLineNumber - 1)
Local $iSocket = TCPConnect($sIPAddress, 80)
If @error Then _ReportError(@error, @extended, @ScriptLineNumber - 1)
TCPSend($iSocket, "GET /?cookieTest=1 HTTP/1.1" & @CRLF & _
    "Host: www.hearthpwn.com" & @CRLF & _
    "Connection: close" & @CRLF & @CRLF)
If @error Then _ReportError(@error, @extended, @ScriptLineNumber - 1)
Local $sData = "", $hTimer = TimerInit()
While 1
    $sRecv = _TCPRecv($iSocket, 2048)
    If $sRecv Then $sData &= $sRecv
    If @error Or @extended Then
        If @error Then _ReportError(@error, @extended, @ScriptLineNumber - 1)
        ExitLoop
    EndIf
    If $sRecv Then $hTimer = TimerInit()
    If TimerDiff($hTimer) > 3000 Then ExitLoop
    Sleep(10)
WEnd
ConsoleWrite("Response received: " & $sData & @CRLF)
TCPCloseSocket($iSocket)
TCPShutdown()

Func _ReportError($iError, $iExtended, $iLine)
    ConsoleWrite("Error: " & $iError & @CRLF & _
        "Extended: " & $iExtended & @CRLF & _
        "Line: " & $iLine & @CRLF)
    TCPShutdown()
    Exit
EndFunc

Func _TCPRecv($iMainsocket, $iMaxLen, $iFlag = 0)
    If IsArray($iMainsocket) And (UBound($iMainsocket, 0) = 1) And (UBound($iMainsocket) > 0) Then $iMainsocket = $iMainsocket[0]
    If $iFlag = Default Then $iFlag = 0
    $iMainsocket = Number($iMainsocket)
    $iMaxLen = Number($iMaxLen)
    $iFlag = Number($iFlag)
    If $iMainsocket < 0 Or _
            $iMaxLen < 1 Or _
            Not ($iFlag = 0 Or $iFlag = 1 Or $iFlag = 2) Then Return SetError(-4, 0, -1) ; invalid parameter

    Local $hWs2 = DllOpen("Ws2_32.dll")
    If @error Then Return SetError(-2, 0, -1) ;missing DLL
    Local $bError = 0, $nCode = 0, $nExtended = 0

    If Not $bError Then
        $aRet = DllCall($hWs2, "int", "ioctlsocket", "uint", $iMainsocket, "long", 0x8004667e, "ulong*", 1) ;FIONBIO
        If @error Then
            $bError = -1
        ElseIf $aRet[0] <> 0 Then ;SOCKET_ERROR
            $bError = 1
        EndIf
    EndIf

    Local $tBuf = DllStructCreate("char[" & $iMaxLen & "]")
    $aRet = DllCall($hWs2, "int", "recv", "uint", $iMainsocket, "ptr", DllStructGetPtr($tBuf), "int", $iMaxLen, "int", 0)
    If @error Then
        $bError = -1
    ElseIf ($aRet[0] = -1) Or ($aRet[0] = 4294967295) Then ;SOCKET_ERROR
        $bError = 1
        $aRet = DllCall($hWs2, "int", "WSAGetLastError")
        If @error Then
            $bError = -1
        ElseIf $aRet[0] = 0 Or $aRet[0] = 10035 Then ;WSAEWOULDBLOCK
            $nCode = -10 ;internal function value, it means no error
        EndIf
    ElseIf $aRet[0] = 0 Then
        $bError = 1
        $nCode = -10
        $nExtended = 1 ;connection closed
    Else
        Local $sResult = DllStructGetData($tBuf, 1) ;data
        If BitAND($iFlag, 2) = 2 Then ;EOT
            If StringRight($sResult, 1) = Chr(3) Then
                $sResult = StringTrimRight($sResult, 1)
                $nExtended = 2 ;End of Text reached
            EndIf
        EndIf
        If BitAND($iFlag, 1) = 1 Then $sResult = Binary($sResult)
    EndIf

    If $bError < 0 Then
        $nCode = -1 ;internal error
        $nReturn = "" ;failure
    ElseIf $bError > 0 Then
        If Not $nCode Then
            $aRet = DllCall($hWs2, "int", "WSAGetLastError")
            If @error Then
                $nCode = -1
            Else
                $nCode = $aRet[0]
            EndIf
            If $nCode = 0 Then $nCode = -3 ;undefined error
        EndIf
        If $nCode = -10 Then $nCode = 0
        $nReturn = ""
    Else
        $nReturn = $sResult
    EndIf
    DllClose($hWs2)
    Return SetError($nCode, $nExtended, $nReturn)
EndFunc   ;==>_TCPRecv

Post here the console output.

Edited by j0kky
1 person likes this

Share this post


Link to post
Share on other sites

#19 ·  Posted

Strange strange, your code, j0kky, actually returned the source. Here is the console:

Response received: HTTP/1.1 200 OK
Date: Tue, 22 Nov 2016 16:39:34 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: close
Set-Cookie: __cfduid=d865e594f7af57763539bb79728c06fb01479832774; expires=Wed, 22-Nov-17 16:39:34 GMT; path=/; domain=.hearthpwn.com; HttpOnly
Cache-Control: no-cache
Pragma: no-cache
Expires: -1
Vary: Accept-Encoding
X-AspNetMvc-Version: 1.1
X-AspNet-Version: 4.0.30319
X-Cobalt-Cache: Cached
X-UA-Compatible: IE=edge,chrome=1
X-Frame-Options: SAMEORIGIN
Server: cloudflare-nginx
CF-RAY: 305dd17796573e32-PRG

e673

.
.
.

site code

So I tried _INetGetSource again, just to make sure it has not started working all of the sudden. I has not.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0