Sign in to follow this  
Followers 0
Shimgreen

Http help needed please.

11 posts in this topic

#1 ·  Posted (edited)

I'm making a script that will help with automating, parsing, and collecting information from certain websites threw the tcp commands.

I cant figure out why i cannot see the data that the server has sent.

I know data is being sent but for some reason it doesnt show up in the msgbox or edit control!

I know its sending data though because when I use a invalid syntax header it will show the error message from the web server.

For Example:

I change

TCPSend ($socket,"GET / HTTP/1.0" & @CRLF & $head)

to

TCPSend($socket,"GET/HTTP/1.0" & @CRLF & $head)

so web server will produce the 400 error showing this,

HTTP/1.1 400 Bad Request

Content-Type: text/html

Date: Tue, 05 Oct 2010 05:57:48 GMT

Connection: close

Content-Length: 35

<h1>Bad Request (Invalid Verb)</h1>

Thats the only time i can get the socket to see the data.....

Can someone help because im lost.

#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <EditConstants.au3>
#include <StaticConstants.au3>

TCPStartUp()

    GUICreate("My GUI")
    $myedit = GUICtrlCreateEdit("" & @CR, 0,0, 400, 380, $ES_MULTILINE+$WS_VSCROLL)
    GUISetState(@SW_SHOW)

$var = Ping("208.109.208.136")
If @error = 0 Then 
 Msgbox(0,"Status","Online, Latency is:" & $var)
Else
 Msgbox(0,"Status"," 1 = Host is offline 2 = Host is unreachable 3 = Bad destination 4 = Other errors " & @CR & @CR & @error)
EndIf

$socket = TCPConnect("208.109.208.136", 80)
If $socket = -1 Then
Msgbox(0,"Status","Error")
Else 
$head = "Accept: text/plain & @CRLF & Accept-Charset: utf-8 & @CRLF & User-Agent: Mozilla/5.0 (Linux; X11) & @CRLF &  Accept-Language: en-US & @CRLF & Cache-Control: no-cache & @CRLF &     Host: www.chain-hq.com & @CRLF"
TCPSend ($socket,"GET / HTTP/1.0" & @CRLF & $head)
EndIf
    While 1 
$recv = TCPRecv($socket, 65400)
if $recv <> "" then 
msgbox(0,"Data:", $recv, 0)
GUICtrlSetData ($myedit, $recv )
EndIf
        $msg = GUIGetMsg()
        If $msg = $GUI_EVENT_CLOSE Then ExitLoop

    WEnd
    GUIDelete()
Edited by Shimgreen

Share this post


Link to post
Share on other sites



bump.

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

The head has to be terminated with a two linebreaks.

Edited by ProgAndy

*GERMAN* [note: you are not allowed to remove author / modified info from my UDFs]My UDFs:[_SetImageBinaryToCtrl] [_TaskDialog] [AutoItObject] [Animated GIF (GDI+)] [ClipPut for Image] [FreeImage] [GDI32 UDFs] [GDIPlus Progressbar] [Hotkey-Selector] [Multiline Inputbox] [MySQL without ODBC] [RichEdit UDFs] [SpeechAPI Example] [WinHTTP]UDFs included in AutoIt: FTP_Ex (as FTPEx), _WinAPI_SetLayeredWindowAttributes

Share this post


Link to post
Share on other sites

thanks for that info but it still doesnt solve the problem after adding the 2nd line break.

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

Your head is badly formated. The way you're doing it is including @crlf within it instead of a line break, which I don't think is what you want.

$head = "Accept: text/plain "& @CRLF & "Accept-Charset: utf-8" & @CRLF & "User-Agent: Mozilla/5.0 (Linux; X11)"& @CRLF & "Accept-Language: en-US" & @CRLF & "Cache-Control: no-cache" & @CRLF & "Host: www.chain-hq.com" & @CRLF

This should do it. But still, it doesn't recieve anything :S

By the way, I was also looking for something like this, I would also appreciate some help to solve it ;)

Edit: I think header is incomplete, can it be?

Edited by jiglei

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

try something like

"Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"& @LF & 
"Accept-Charset: utf-8" & @LF & 
"User-Agent: Mozilla/5.0 (Linux; X11)"& @LF & 
"Accept-Language: en-us,en;q=0.5" & @LF & 
"Cache-Control: no-cache" & @LF & 
"Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7"

Once you have done that, use ConsoleWrite to test what you are sending. @LFCR is a windows deal, it sends '\r\n', you don't want that for HTTP.

Edit: Fixed code tags

Edited by bo8ster

Post your code because code says more then your words can. SciTe Debug mode - it's magic: #AutoIt3Wrapper_run_debug_mode=Y. Use Opt("MustDeclareVars", 1)[topic="84960"]Brett F's Learning To Script with AutoIt V3[/topic][topic="21048"]Valuater's AutoIt 1-2-3, Class... is now in Session[/topic]Contribution: [topic="87994"]Get SVN Rev Number[/topic], [topic="93527"]Control Handle under mouse[/topic], [topic="91966"]A Presentation using AutoIt[/topic], [topic="112756"]Log ConsoleWrite output in Scite[/topic]

Share this post


Link to post
Share on other sites

=) =) !!! Thank you guys for helping me figure it out, both of you. I was able to create a valid header that works and i get the 200 signal ;). One Minute I will post the header you want to send.

Share this post


Link to post
Share on other sites

#8 ·  Posted (edited)

Here it is, try this correct header out for yourself to recieve data from the website.

$head = "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" & @CRLF & "Accept-Charset: utf-8" & @CRLF & "User-Agent: Mozilla/5.0 (Linux; X11)" & @CRLF & "Accept-Language: en-us,en;q=0.5" & @CRLF & "Cache-Control: no-cache" &@CRLF & "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7" & @CRLF & @CRLF

$head = "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" & @CRLF & "Accept-Charset: utf-8" & @CRLF & "User-Agent: Mozilla/5.0 (Linux; X11)" & @CRLF & "Accept-Language: en-us,en;q=0.5" & @CRLF & "Cache-Control: no-cache" &@CRLF & "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7" & @CRLF & @CRLF

Edited by Shimgreen

Share this post


Link to post
Share on other sites

Great to hear you got it to work. I didn't expect @CRLF to work, guess I was wrong.

Good deal!


Post your code because code says more then your words can. SciTe Debug mode - it's magic: #AutoIt3Wrapper_run_debug_mode=Y. Use Opt("MustDeclareVars", 1)[topic="84960"]Brett F's Learning To Script with AutoIt V3[/topic][topic="21048"]Valuater's AutoIt 1-2-3, Class... is now in Session[/topic]Contribution: [topic="87994"]Get SVN Rev Number[/topic], [topic="93527"]Control Handle under mouse[/topic], [topic="91966"]A Presentation using AutoIt[/topic], [topic="112756"]Log ConsoleWrite output in Scite[/topic]

Share this post


Link to post
Share on other sites

I just read that initial lines and headers should end in CRLF, though you should gracefully handle lines ending in just LF.

So I guess its rude of me to use CRLF, though still workable.

http://www.jmarshall.com/easy/http/

Share this post


Link to post
Share on other sites

Maybe a little but it works and thats all that matters. The encoding may be saving you, not sure.

If you find other sites don't work that is a good place to start ;)


Post your code because code says more then your words can. SciTe Debug mode - it's magic: #AutoIt3Wrapper_run_debug_mode=Y. Use Opt("MustDeclareVars", 1)[topic="84960"]Brett F's Learning To Script with AutoIt V3[/topic][topic="21048"]Valuater's AutoIt 1-2-3, Class... is now in Session[/topic]Contribution: [topic="87994"]Get SVN Rev Number[/topic], [topic="93527"]Control Handle under mouse[/topic], [topic="91966"]A Presentation using AutoIt[/topic], [topic="112756"]Log ConsoleWrite output in Scite[/topic]

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0