Sign in to follow this  
Followers 0
mary

HTTP pipelining

20 posts in this topic

#1 ·  Posted (edited)

Hi !

as explained here http://en.wikipedia.org/wiki/HTTP_pipelining , HTTP pipelining may make faster http requests, specially when server updates data in real time ( let say every 10 milliseconds) .

also, this may be an alternative for multi-threading way of making a httpget (or _InetGetSource...).

my question is:

how to make HTTP pipelining with autoit ? ( i mean, is it just to add a "keep-alive" in header parameter of _InetGetSource or something elese ?)

thinks for any help

Edited by mary

Share this post


Link to post
Share on other sites



Have you tried using the MSXML2.ServerXMLHTTP ojbect its faster than any others I have tried

http://www.autoitscript.com/forum/index.ph...c=42520&hl=

I don't think so, here some tests (for speed, i'd just do a 100 loop and calculate the average of time consuming )

MSXML2.ServerXMLHTTP ===> 203 ms

Winhttp ===> 195 ms

_InetGetSource ===> 182 ms

all works synchronymly ( no way to do asynchronym call with autoit)

Share this post


Link to post
Share on other sites

#4 ·  Posted (edited)

Hi !

as explained here http://en.wikipedia.org/wiki/HTTP_pipelining , HTTP pipelining may make faster http requests, specially when server updates data in real time ( let say every 10 milliseconds) .

also, this may be an alternative for multi-threading way of making a httpget (or _InetGetSource...).

my question is:

how to make HTTP pipelining with autoit ? ( i mean, is it just to add a "keep-alive" in header parameter of _InetGetSource or something elese ?)

thinks for any help

I don't think your article on Wikipedia is acurate. Create the following .reg file and Merge it.

REGEDIT4

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings]
"MaxConnectionsPer1_0Server"=dword:A
"MaxConnectionsPerServer"=dword:2
;

Note that the values following "dword:" are in hexadecimal, therefore use these letters for settings greater than 9 - 10=A, 11=B, 12=C, 13=D, 14=E, 15=F.

This will definitly work with IE7 and I think as far back as IE4 (hence the use of the REGEDIT4 line.)

EDIT: I forgot the semicolon on the last line. It MUST be there

Edited by GEOSoft

George

Question about decompiling code? Read the decompiling FAQ and don't bother posting the question in the forums.

Be sure to read and follow the forum rules. -AKA the AutoIt Reading and Comprehension Skills test.***

The PCRE (Regular Expression) ToolKit for AutoIT - (Updated Oct 20, 2011 ver:3.0.1.13) - Please update your current version before filing any bug reports. The installer now includes both 32 and 64 bit versions. No change in version number.

Visit my Blog .. currently not active but it will soon be resplendent with news and views. Also please remove any links you may have to my website. it is soon to be closed and replaced with something else.

"Old age and treachery will always overcome youth and skill!"

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

sorry, but you misntrestood HTTP pipelining....(if you what to understand it , sniff some http paquets and you will see the difference)

: exemple:

Without pipelining: 2 successive calls to an HTTP_get( site) ===> 8 transactions before getting data (client ----> server)

With pipelining: 2 successive calls to an HTTP_get( site) ===> 5 transactions before getting data (client ----> server)

anyway, thinks for your participation

Edited by mary

Share this post


Link to post
Share on other sites

hmm...

thats how i understand it:

http 1.0 or 1.1 'Connection: Close' for 2 Transactions

Connect
 Request 
 Reply
Disconnect

Connect
 Request 
 Reply
Disconnect
http 1.1 'Connection: persistent' for 2 Transactions
Connect
 Request
 Reply
 Request
 Reply
Disconnect
http 1.1 pipelining for 2 Transactions

Connect
 Request
 Request
 Reply
 Reply
Disconnect

All of this can be done with Autoit's Tcp functions


CoProc Multi Process Helper libraryTrashBin.nfshost.com store your AutoIt related files here!AutoIt User Map

Share this post


Link to post
Share on other sites

here is a minimal example:

TCPStartup()

$sReply1  = "GET /autoit3/ HTTP/1.1" & @CRLF
$sReply1 &= "Host: www.autoitscript.com" & @CRLF & @CRLF

$sReply2  = "GET /autoit3/docs/ HTTP/1.1" & @CRLF
$sReply2 &= "Host: www.autoitscript.com" & @CRLF & @CRLF


$hSock = TCPConnect(TCPNameToIP("www.autoitscript.com"),80)
TCPSend($hSock,$sReply1 & $sReply2)

; I dont want to write a header parser
; so a simple timeout has to do it for now
$iTimer = TimerInit()
While TimerDiff($iTimer) < 500
    $sTmp = TCPRecv($hSock,1024)
    If StringLen($sTmp) > 0 Then
        ConsoleWrite($sTmp)
        $iTimer = TimerInit()
    EndIf
WEnd

TCPCloseSocket($hSock)
TCPShutdown()

but what exactly are you trying to do?


CoProc Multi Process Helper libraryTrashBin.nfshost.com store your AutoIt related files here!AutoIt User Map

Share this post


Link to post
Share on other sites

here is a minimal example:

TCPStartup()

$sReply1  = "GET /autoit3/ HTTP/1.1" & @CRLF
$sReply1 &= "Host: www.autoitscript.com" & @CRLF & @CRLF

$sReply2  = "GET /autoit3/docs/ HTTP/1.1" & @CRLF
$sReply2 &= "Host: www.autoitscript.com" & @CRLF & @CRLF
$hSock = TCPConnect(TCPNameToIP("www.autoitscript.com"),80)
TCPSend($hSock,$sReply1 & $sReply2)

; I dont want to write a header parser
; so a simple timeout has to do it for now
$iTimer = TimerInit()
While TimerDiff($iTimer) < 500
    $sTmp = TCPRecv($hSock,1024)
    If StringLen($sTmp) > 0 Then
        ConsoleWrite($sTmp)
        $iTimer = TimerInit()
    EndIf
WEnd

TCPCloseSocket($hSock)
TCPShutdown()

but what exactly are you trying to do?

Thinks for replay !

persistent connection= HTTP pipelining

HTTP pipelining means that you use one connexion session for many HTTP requests ( depends on "keep-alive" header).

Ethereal shows that well (it is faster than opening many session)

anyway, my goal is to detect the startup of an event in a siteweb as soon as possible ( the server is updating in realtime)

Share this post


Link to post
Share on other sites

i'd rather say HTTP pipelining = abusing a persistent connection :whistle:

at least the example i did was far from spec.

and you realy care about a tenth of a second?

well if yes then post your test script and i'll ad a TCP* function based method...

and btw:

if there is no 'Connection: Close' in the header 'Connection: Keep-Alive' is assumed in http 1.1 (at least on Apache)

another 24 byte traffic saved :P


CoProc Multi Process Helper libraryTrashBin.nfshost.com store your AutoIt related files here!AutoIt User Map

Share this post


Link to post
Share on other sites

#11 ·  Posted (edited)

thinks piccaso !

Yes tenth of a second is very important for my project :

While 1

$htm=_InetGetSource("www.theserver.com/cgi?course=14")

If StringInStr($htm, "Ok, Start now !") then
  Action() ; Action is a function to submit a form (no problem here)
  Exitloop
EndIf

Wend

with 4 instances of my script, i can do 8 call _InetGetSource per second but maybe there is an other faster solution

Edited by mary

Share this post


Link to post
Share on other sites

just one problem.

www.theserver.com/cgi?course=14

doesent exist

no it just an exemple

Share this post


Link to post
Share on other sites

but i need to know which server the host is running, which http version the cgi script uses, see the reply headers, and so on...

to make it accurate


CoProc Multi Process Helper libraryTrashBin.nfshost.com store your AutoIt related files here!AutoIt User Map

Share this post


Link to post
Share on other sites

#15 ·  Posted (edited)

but i need to know which server the host is running, which http version the cgi script uses, see the reply headers, and so on...

to make it accurate

ok, here is the server reponse header

HTTP/1.x 200 OK
Connection: Keep-Alive
Transfer-Encoding: chunked
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Date: Wed, 21 Mar 2007 23:22:10 GMT
Content-Type: text/html;charset=UTF-8
Server: Betex HTTP Server, Apache-Coyote/1.1
X-Powered-By: Servlet 2.4; Tomcat-5.0.28/JBoss-4.0.1sp1 (build: CVSTag=JBoss_4_0_1_SP1 date=200502160314)
Set-Cookie: Tags=mi=20334553; Expires=Thu, 22-Mar-2007 23:22:11 GMT; Path=/
Set-Cookie: abcd=44593ce629a0;expires=Thu, 22-Mar-07 11:22:11 GMT;path=/
Cache-Control: no-store
Pragma: no-cache

(i found a good library supporting http pipelining: Libcurl http://curl.haxx.se/ but it is hard to convert in autoit script, and autoit dont support callback futures :whistle:

Edited by mary

Share this post


Link to post
Share on other sites

come on, this is ridiculous

ridiculous ????

anyway, thinks and sorry for my poor english

Share this post


Link to post
Share on other sites

i dont want to do something i cant test local.

i dont want to spend much time on writing something general-purpose that may work and post it.

... wait for your reply ... change something and post it... ad so on

if you dont want to post it in an open forum you can pm it to me too

but you probably knew that bevore...


CoProc Multi Process Helper libraryTrashBin.nfshost.com store your AutoIt related files here!AutoIt User Map

Share this post


Link to post
Share on other sites

here is a minimal example:

TCPStartup()

$sReply1  = "GET /autoit3/ HTTP/1.1" & @CRLF
$sReply1 &= "Host: www.autoitscript.com" & @CRLF & @CRLF

$sReply2  = "GET /autoit3/docs/ HTTP/1.1" & @CRLF
$sReply2 &= "Host: www.autoitscript.com" & @CRLF & @CRLF
$hSock = TCPConnect(TCPNameToIP("www.autoitscript.com"),80)
TCPSend($hSock,$sReply1 & $sReply2)

; I dont want to write a header parser
; so a simple timeout has to do it for now
$iTimer = TimerInit()
While TimerDiff($iTimer) < 500
    $sTmp = TCPRecv($hSock,1024)
    If StringLen($sTmp) > 0 Then
        ConsoleWrite($sTmp)
        $iTimer = TimerInit()
    EndIf
WEnd

TCPCloseSocket($hSock)
TCPShutdown()

but what exactly are you trying to do?

Hey, I just have a quick question: say I wanted to grab the source of a website every minute or so, would you use this technique? or _InetGetSource()? And also, which way is faster?

Thanks,

Kurt


Awaiting Diablo III..

Share this post


Link to post
Share on other sites

#20 ·  Posted (edited)

Hey, I just have a quick question: say I wanted to grab the source of a website every minute or so, would you use this technique? or _InetGetSource()? And also, which way is faster?

Thanks,

Kurt

while 1

$htm=_InetGetSource()?
 ; ...instructions
sleep(60 * 1000)

Wend

Libcurl is the faster library i know (but no idea how convert it in autoit UDF)

Edited by mary

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0