ParoXsitiC Posted August 16, 2006 Share Posted August 16, 2006 Using the following UDF: expandcollapse popupFunc _HTTPConnect($host, $port = 80) Dim $ip = TCPNameToIP ( $host ) Dim $socket = TCPConnect ( $ip, 80 ) If ($socket == -1) Then SetError(1, @error) Return -1 EndIf $_HTTPLastSocket = $socket SetError(0) Return $socket EndFunc Func _HTTPClose($socket = -1) If $socket == -1 Then If $_HTTPLastSocket == -1 Then SetError(1) Return 0 EndIf $socket = $_HTTPLastSocket EndIf TCPCloseSocket($socket) SetError(0) Return 1 EndFunc Func _HTTPRead($socket = -1, $flag = 0) If $socket == -1 Then If $_HTTPLastSocket == -1 Then SetError(1) Return EndIf $socket = $_HTTPLastSocket EndIf Dim $timer = TimerStart() Dim $performancetimer = TimerStart() Dim $downloadtime = 0 Dim $headers[1][2] ; An Array of the headers found Dim $numheaders = 0 ; The number of headers found Dim $body = "" ; The body of the message Dim $HTTPVersion ; The HTTP version of the server (almost always 1.1) Dim $HTTPResponseCode ; The HTTP response code like 200, or 404 Dim $HTTPResponseReason ; The human-readable response reason, like "OK" or "Not Found" Dim $bytesreceived = 0 ; The total number of bytes received Dim $data = "" ; The entire raw message gets put in here. Dim $chunked = 0 ; Set to 1 if we get the "Transfer-Encoding: chunked" header. Dim $chunksize = 0 ; The size of the current chunk we are processing. Dim $chunkprocessed = 0 ; The amount of data we have processed on the current chunk. Dim $contentlength ; The size of the body, if NOT using chunked transfer mode. Dim $part = 0 ; Refers to what part of the data we're currently parsing: ; 0 - Nothing parsed, so HTTP response should come next ; 1 - Currently parsing headers ; 2 - Currently waiting for the next chunk size - this is skipped if the transfer-encoding is not chunked ; 3 - Currently waiting for or parsing body data ; 4 - Currently parsing footers While 1 Sleep(10) Dim $recv = TCPRecv($socket,16) If @error <> 0 Then ;ConsoleWrite("Server closed connection") ;@error appears to be -1 after the server closes the connection. A good way to tell that we're finished, because we always send ;the "Connection: close" header to the server. ; !!! This is no longer used because we can now tell that we're done by checking the content-length header or properly handling ; chunked data. EndIf If $recv <> "" Then $bytesreceived = $bytesreceived + StringLen($recv) $timer = TimerStart() $data &= $recv ConsoleWrite("Bytes downloaded: "&$bytesreceived&@CRLF) EndIf Dim $split = StringSplit($data,@CRLF,1) $data = "" Dim $i For $i=1 To $split[0] If $i=$split[0] Then If $part < 2 OR $chunked = 1 Then ; This is tricky. The last line we've received might be truncated, so we only want to process it under special cases. ; Non chunked data doesn't always send a CRLF at the end so there's no way to tell if this is truly the last line without parsing it. ; However, we don't want to parse it if it's only a partial header or something. ; The solution: We will only process this last line if we're at the body section and the transfer-encoding is NOT chunked. $data = $split[$i] ExitLoop EndIf EndIf Dim $newpart = $part Switch $part Case 0 ; Nothing parsed, so HTTP response should come next If $split[$i] <> "" Then Dim $regex = StringRegExp($split[$i],"^HTTP/([0-9.]+) ([0-9]+) ([a-zA-Z0-9 ]+)$",3) If @extended = 0 Then SetError(5) Return $split[$i] Else $HTTPVersion = $regex[0] $HTTPResponseCode = $regex[1] $HTTPResponseReason = $regex[2] If $HTTPResponseCode <> 100 Then $newpart = 1 EndIf EndIf EndIf Case 1, 4 ; Currently parsing headers or footers ;If the line is blank, then we're done with headers and the body is next If $split[$i] == "" Then If $part = 1 Then If $chunked Then $newpart = 2 Else $newpart = 3 EndIf ElseIf $part = 4 Then ; If $part is 4 then we're processing footers, so we're all done now. ExitLoop 2 EndIf Else ;The line wasn't blank ;Check to see if the line begins with whitespace. If it does, it's actually ;a continuation of the previous header Dim $regex = StringRegExp($split[$i],"^[ \t]+([^ \t].*)$",3) If @extended = 1 Then If $numheaders == 0 Then SetError(6) Return $split[$i] EndIf $headers[$numheaders-1][1] &= $regex[0] Else;The line didn't start with a space Dim $regex = StringRegExp($split[$i],"^([^ :]+):[ \t]*(.*)$",3) If @extended = 1 Then ;This is a new header, so add it to the array $numheaders = $numheaders + 1 ReDim $headers[$numheaders][2] $headers[$numheaders-1][0] = $regex[0] $headers[$numheaders-1][1] = $regex[1] ; There are a couple headers we need to know about. We'll process them here. If $regex[0] = "Transfer-Encoding" AND $regex[1] = "chunked" Then $chunked = 1 ElseIf $regex[0] = "Content-Length" Then $contentlength = Int($regex[1]) EndIf Else SetError(6) Return $split[$i] EndIf EndIf EndIf Case 2 ; Awaiting chunk size $regex = StringRegExp($split[$i],"^([0-9a-f]+);?.*$",3) If @extended = 0 Then SetError(8) Return $split[$i] EndIf $chunksize = $regex[0] $chunksize = Dec($chunksize) $chunkprocessed = 0 If $chunksize == 0 Then $newpart = 4 Else $newpart = 3 EndIf Case 3 ; Awaiting body data $body &= $split[$i] $chunkprocessed = $chunkprocessed + StringLen($split[$i]) If $chunked Then If $chunkprocessed >= $chunksize Then $newpart = 2 Else $body &= @CRLF $chunkprocessed = $chunkprocessed + 2; We add 2 for the CRLF we stipped off. EndIf Else If $chunkprocessed >= $contentlength Then ExitLoop 2 Else If $i < $split[0] Then ; Only add a CRLF if this is not the last line received. $body &= @CRLF $chunkprocessed = $chunkprocessed + 2; We add 2 for the CRLF we stipped off. EndIf EndIf EndIf Case Else ; This should never happen EndSwitch $part = $newpart Next If $bytesreceived == 0 AND TimerDiff($timer) > $_HTTPRecvTimeout Then SetError(3) Return 0 ElseIf $bytesreceived > 0 AND TimerDiff($timer) > $_HTTPRecvTimeout Then ConsoleWrite($body) SetError(4) Return $bytesreceived EndIf WEnd $downloadtime = TimerDiff($performancetimer) ConsoleWrite("Performance: Download time: "&$downloadtime&@CRLF) Switch $flag Case 0 SetError(0) Return $body Case 1 Dim $return[5] $return[0] = $HTTPResponseCode $return[1] = $HTTPResponseReason $return[2] = $HTTPVersion $return[3] = $headers $return[4] = $body SetError(0) Return $return Case Else SetError(7) Return 0 EndSwitch EndFunc Func _HTTPGetCookies($host, $page, $cookies="", $socket = -1) Dim $command If $socket == -1 Then If $_HTTPLastSocket == -1 Then SetError(1) Return EndIf $socket = $_HTTPLastSocket EndIf $command = "GET "&$page&" HTTP/1.1"&@CRLF $command &= "Host: " &$host&@CRLF $command &= "Cookie: "&$cookies&@CRLF&@CRLF Dim $bytessent = TCPSend($socket, $command) If $bytessent == 0 Then SetExtended(@error) SetError(2) return 0 EndIf SetError(0) Return $bytessent EndFunc Func _HTTPPostCookies($host, $page, $data = "", $cookies="", $socket = -1) Dim $command If $socket == -1 Then If $_HTTPLastSocket == -1 Then SetError(1) Return EndIf $socket = $_HTTPLastSocket EndIf Dim $datasize = StringLen($data) $command = "POST "&$page&" HTTP/1.1"&@CRLF $command &= "Host: " &$host&@CRLF $command &= "User-Agent: "&$_HTTPUserAgent&@CRLF $command &= "Connection: close"&@CRLF $command &= "Content-Type: application/x-www-form-urlencoded"&@CRLF $command &= "Content-Length: "&$datasize&@CRLF $command &= "Cookie: "&$cookies&@CRLF&@CRLF $command &= $data&@CRLF Dim $bytessent = TCPSend($socket, $command) If $bytessent == 0 Then SetExtended(@error) SetError(2) return 0 EndIf SetError(0) Return $bytessent EndFunc I thought that maybe it just needed a delay between the two requests so as a quick fix I tried just using TCPRevc($Socket, 10) between each request, this does improve the sucess ratio but doe's not work 100% of the time. Is it possible to send GET and POST HTTP Requests this fast to a server and not wait for a response? Also, I am having problems with the opening and closing of sockets. It seems like in theory I would only have to open the socket once, send all my GETs and POSTs and then close the socket, however the script seems to only want to work when I open and close the socket for each request. The HTTPRead function in the UDF is very slow. I would ideally want to just send requests and not wait for any response, but if this is not possible then I will just use wininet functions to get and post data instead. Link to comment Share on other sites More sharing options...
ParoXsitiC Posted August 16, 2006 Author Share Posted August 16, 2006 Apparently I'll want to pipeline all the GET requests and then do the POST requests. I cant find any examples on how to pipeline HTTP requests though. Link to comment Share on other sites More sharing options...
ParoXsitiC Posted August 16, 2006 Author Share Posted August 16, 2006 Update: I've tried using WinHttp.WinHttpRequest.5.1 object but I am unable to set cookies between send() methods. So that won't work. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now