Jump to content

HTTP & gZip encoding

Shanheavel
 Share

Recommended Posts

Shanheavel Wayfarer

Shanheavel

Posted
Posted (edited)

Hello,

I've made program which uses TCP and HTTP protocols for browsing webpages.

TCPStartup()
$IP = TCPNameToIP("google.pl")
$ServerSocket = TCPConnect($IP, 80)
TCPSend($ServerSocket, $Request)
...

Sometimes received packets are gZiped I know that I can remove "Accept-Encoding: gzip,deflate" from request, but I want use it).

I tried lots of Zip UDFs but they don't work.

...

I tested all.

Example:

Received data from server:

HTTP/1.1 200 OK

Date: Sun, 14 Aug 2011 10:38:55 GMT

Server: Apache

Cache-Control: no-cache

Vary: Accept-Encoding

Content-Encoding: gzip

Content-Length: 982

Keep-Alive: timeout=15, max=100

Connection: Keep-Alive

Content-Type: text/plain; charset=UTF-8

‹ ťUMoŰF=G€˙Ă– *ęQÔ‡ÓP&ŔR´·:§\Ś!9"W"w‰ÝĄ(˛čˇF‹ü zń©±ţWwEZ¦0r!¸óńŢěçńdp&#AET] o)Ü)g

[h­Vp2xeÇ<*sdj4qmŻJ)ʱG䏓Á‹WöđeY€”ĂŃxu!PĘcŤ#ó‚®lW)ŤRßwg#’ń„2{´Ôľ?Íóđ8sZ^M«1Ý’(Ó¸ľf/©ÂśHUgčC6‰ŕ%‹˝Rd6ÍGAx©1ÇKFĂŕĚŃÉA‡Ac_{

`“Ôoť-ÁŘXBôČôMQ[[ű ôŻ€Ő5l¸wČ

s

‚Á™jf“vHÇ#e˘DFĄş-ż˘±J=w1)vşT^(s—‚dbć[熑FV00.ÓçÎqQ˛R‚=ĎGŮBVęy˝y;™ś.¬ŕÎuć´I:ŰąEn‘î‘|ÂĽfđ-G7źÍµ?ěŻ1Ů_ý˛żÚ€íVw”Óéb1˙É

žŚ|fż”

« mĄíÎzśóééۉk=ß3Y~Q>Ćň;Č©í¦w4‹Ét:›[AßŮ㹏đ>Śb\„bŰU=”Ůdćj”‡O#mW©JU©JŰ

ŕčŮŢw?wľ\4Ź\ű#Ę6ŔîMt6źÎgş»}ç4NűQ·{uŘ••€ő”Jqv´vG΢ŚFx0~Đ‘öhHĎ2ľEáO'fUś6¸[ŰRhö­Tw(íYëFă48"°CôVQ‘"ň-ÇH…Ƹ4

â±°î6|Ĺą:|Źţ(Î3ůP€¤WľőNÜřg+šXÁ˙-żNĄ€h݉ôSRĄ

Ďq

žóhśH8Ă|\de1îĆEZĽ„ş”)Ż~ÔÓô]÷Íű˘©CŮ Ů_AlôŠ"IDý ľ­h%#qČúDÎ+$űżH¤ŕzÖśŐQÚ%š†|Ű—Z–$ä"FáM‹‘<Ł1/IÄ3.Ľő¤Ş%Yq¦^KÚ çΊݒǔ%ž9B|¬řÉFr˛F-™

hšýµľ9ŮŠűë›~0Ý"­BwőŔ]Ł«'÷aÝ!…őţJ?±)(FŤ†ĘKIeCJ© ˘7źIeoGA‰¬ľ~YőźŚ`4#”$\עżÇ

7Ę7>ś7µ‚5Qş ˘{Ú•SYdP{”e”á’?ęëX÷Q€Ů'OwEë2ĄĚ›ż{ÜÓůÄ

#1őńv¸˙[Ş

ĚČƇ?j·Ý÷ř?|Ž°ň

Get binary body:

$ServerRecv = TCPRecv($ServerSocket, 1024^2)
$HeaderEnd = StringInStr($ServerRecv, @CRLF & @CRLF)
$Body = StringTrimLeft($ServerRecv, $HeaderEnd + 3)
$Binary = StringToBinary($Body)

And I use for example: 

_Zlib_UncompressBinary($Binary, BinaryLen($Binary))

Maybe I do something wrong?

Edited by Adrian777
guinness Universalist

guinness

Posted
Posted

How about using 7-Zip? >>

UDF List:

 
_AdapterConnections() • _AlwaysRun() • _AppMon() • _AppMonEx() • _ArrayFilter/_ArrayReduce • _BinaryBin() • _CheckMsgBox() • _CmdLineRaw() • _ContextMenu() • _ConvertLHWebColor()/_ConvertSHWebColor() • _DesktopDimensions() • _DisplayPassword() • _DotNet_Load()/_DotNet_Unload() • _Fibonacci() • _FileCompare() • _FileCompareContents() • _FileNameByHandle() • _FilePrefix/SRE() • _FindInFile() • _GetBackgroundColor()/_SetBackgroundColor() • _GetConrolID() • _GetCtrlClass() • _GetDirectoryFormat() • _GetDriveMediaType() • _GetFilename()/_GetFilenameExt() • _GetHardwareID() • _GetIP() • _GetIP_Country() • _GetOSLanguage() • _GetSavedSource() • _GetStringSize() • _GetSystemPaths() • _GetURLImage() • _GIFImage() • _GoogleWeather() • _GUICtrlCreateGroup() • _GUICtrlListBox_CreateArray() • _GUICtrlListView_CreateArray() • _GUICtrlListView_SaveCSV() • _GUICtrlListView_SaveHTML() • _GUICtrlListView_SaveTxt() • _GUICtrlListView_SaveXML() • _GUICtrlMenu_Recent() • _GUICtrlMenu_SetItemImage() • _GUICtrlTreeView_CreateArray() • _GUIDisable() • _GUIImageList_SetIconFromHandle() • _GUIRegisterMsg() • _GUISetIcon() • _Icon_Clear()/_Icon_Set() • _IdleTime() • _InetGet() • _InetGetGUI() • _InetGetProgress() • _IPDetails() • _IsFileOlder() • _IsGUID() • _IsHex() • _IsPalindrome() • _IsRegKey() • _IsStringRegExp() • _IsSystemDrive() • _IsUPX() • _IsValidType() • _IsWebColor() • _Language() • _Log() • _MicrosoftInternetConnectivity() • _MSDNDataType() • _PathFull/GetRelative/Split() • _PathSplitEx() • _PrintFromArray() • _ProgressSetMarquee() • _ReDim() • _RockPaperScissors()/_RockPaperScissorsLizardSpock() • _ScrollingCredits • _SelfDelete() • _SelfRename() • _SelfUpdate() • _SendTo() • _ShellAll() • _ShellFile() • _ShellFolder() • _SingletonHWID() • _SingletonPID() • _Startup() • _StringCompact() • _StringIsValid() • _StringRegExpMetaCharacters() • _StringReplaceWholeWord() • _StringStripChars() • _Temperature() • _TrialPeriod() • _UKToUSDate()/_USToUKDate() • _WinAPI_Create_CTL_CODE() • _WinAPI_CreateGUID() • _WMIDateStringToDate()/_DateToWMIDateString() • Au3 script parsing • AutoIt Search • AutoIt3 Portable • AutoIt3WrapperToPragma • AutoItWinGetTitle()/AutoItWinSetTitle() • Coding • DirToHTML5 • FileInstallr • FileReadLastChars() • GeoIP database • GUI - Only Close Button • GUI Examples • GUICtrlDeleteImage() • GUICtrlGetBkColor() • GUICtrlGetStyle() • GUIEvents • GUIGetBkColor() • Int_Parse() & Int_TryParse() • IsISBN() • LockFile() • Mapping CtrlIDs • OOP in AutoIt • ParseHeadersToSciTE() • PasswordValid • PasteBin • Posts Per Day • PreExpand • Protect Globals • Queue() • Resource Update • ResourcesEx • SciTE Jump • Settings INI • SHELLHOOK • Shunting-Yard • Signature Creator • Stack() • Stopwatch() • StringAddLF()/StringStripLF() • StringEOLToCRLF() • VSCROLL • WM_COPYDATA • More Examples...

Updated: 22/04/2018

Shanheavel Wayfarer

Shanheavel

Posted
  • Author
Posted (edited)

How about using 7-Zip? >>

I've tried it day ago. If I save body of received packet (as string) to "file.txt", rename to "file.zip" and extract (by 7zip) I've "file" and error:

"CRC failed in <filename>. The file is corrupt."

I can rename it again to "file.txt", open in notepad and I've correct packet but this errors interfere with my "browser".

Edited by Adrian777
smartee Universalist

smartee

Posted
Posted

@Adrian777: Post a runnable snippet of code that demonstrates your problem. Then we can offer something other than guesses and links you've already tried :mellow:

guinness Universalist

guinness

Posted
Posted (edited)

This is what I use to decrypt a compressed file using 7-Zip (must be supported by 7-Zip.) As smartee said please post some reproducible code and/or website that you're having the problem with, because it could be the simplest of errors you've somehow overlooked :mellow:

Func _7Zip_Decrypt($sZipFile, $sDestinationFolder = @ScriptDir, $sPassword = "")
    If FileExists($sZipFile) = 0 Then
        Return SetError(1, 0, 0)
    EndIf
    If StringRight($sDestinationFolder, 1) <> "\" Then
        $sDestinationFolder &= "\"
    EndIf
    If FileExists($sDestinationFolder) = 0 Then
        DirCreate($sDestinationFolder)
    EndIf
    If $sPassword <> "" Then
        $sPassword = "-p" & '"' & $sPassword & '" '
    EndIf
    If FileExists(@ScriptDir & "\" & "7za.exe") = 0 Then
        FileInstall("7za.exe", @ScriptDir & "\" & "7za.exe", 0)
    EndIf
    Return RunWait('7za.exe' & ' x "' & $sZipFile & '" ' & $sPassword & "-y -o" & '"' & $sDestinationFolder & '"', "", @SW_HIDE)
EndFunc   ;==>_7Zip_Decrypt
Edited by guinness

UDF List:

 
_AdapterConnections() • _AlwaysRun() • _AppMon() • _AppMonEx() • _ArrayFilter/_ArrayReduce • _BinaryBin() • _CheckMsgBox() • _CmdLineRaw() • _ContextMenu() • _ConvertLHWebColor()/_ConvertSHWebColor() • _DesktopDimensions() • _DisplayPassword() • _DotNet_Load()/_DotNet_Unload() • _Fibonacci() • _FileCompare() • _FileCompareContents() • _FileNameByHandle() • _FilePrefix/SRE() • _FindInFile() • _GetBackgroundColor()/_SetBackgroundColor() • _GetConrolID() • _GetCtrlClass() • _GetDirectoryFormat() • _GetDriveMediaType() • _GetFilename()/_GetFilenameExt() • _GetHardwareID() • _GetIP() • _GetIP_Country() • _GetOSLanguage() • _GetSavedSource() • _GetStringSize() • _GetSystemPaths() • _GetURLImage() • _GIFImage() • _GoogleWeather() • _GUICtrlCreateGroup() • _GUICtrlListBox_CreateArray() • _GUICtrlListView_CreateArray() • _GUICtrlListView_SaveCSV() • _GUICtrlListView_SaveHTML() • _GUICtrlListView_SaveTxt() • _GUICtrlListView_SaveXML() • _GUICtrlMenu_Recent() • _GUICtrlMenu_SetItemImage() • _GUICtrlTreeView_CreateArray() • _GUIDisable() • _GUIImageList_SetIconFromHandle() • _GUIRegisterMsg() • _GUISetIcon() • _Icon_Clear()/_Icon_Set() • _IdleTime() • _InetGet() • _InetGetGUI() • _InetGetProgress() • _IPDetails() • _IsFileOlder() • _IsGUID() • _IsHex() • _IsPalindrome() • _IsRegKey() • _IsStringRegExp() • _IsSystemDrive() • _IsUPX() • _IsValidType() • _IsWebColor() • _Language() • _Log() • _MicrosoftInternetConnectivity() • _MSDNDataType() • _PathFull/GetRelative/Split() • _PathSplitEx() • _PrintFromArray() • _ProgressSetMarquee() • _ReDim() • _RockPaperScissors()/_RockPaperScissorsLizardSpock() • _ScrollingCredits • _SelfDelete() • _SelfRename() • _SelfUpdate() • _SendTo() • _ShellAll() • _ShellFile() • _ShellFolder() • _SingletonHWID() • _SingletonPID() • _Startup() • _StringCompact() • _StringIsValid() • _StringRegExpMetaCharacters() • _StringReplaceWholeWord() • _StringStripChars() • _Temperature() • _TrialPeriod() • _UKToUSDate()/_USToUKDate() • _WinAPI_Create_CTL_CODE() • _WinAPI_CreateGUID() • _WMIDateStringToDate()/_DateToWMIDateString() • Au3 script parsing • AutoIt Search • AutoIt3 Portable • AutoIt3WrapperToPragma • AutoItWinGetTitle()/AutoItWinSetTitle() • Coding • DirToHTML5 • FileInstallr • FileReadLastChars() • GeoIP database • GUI - Only Close Button • GUI Examples • GUICtrlDeleteImage() • GUICtrlGetBkColor() • GUICtrlGetStyle() • GUIEvents • GUIGetBkColor() • Int_Parse() & Int_TryParse() • IsISBN() • LockFile() • Mapping CtrlIDs • OOP in AutoIt • ParseHeadersToSciTE() • PasswordValid • PasteBin • Posts Per Day • PreExpand • Protect Globals • Queue() • Resource Update • ResourcesEx • SciTE Jump • Settings INI • SHELLHOOK • Shunting-Yard • Signature Creator • Stack() • Stopwatch() • StringAddLF()/StringStripLF() • StringEOLToCRLF() • VSCROLL • WM_COPYDATA • More Examples...

Updated: 22/04/2018

Shanheavel Wayfarer

Shanheavel

Posted
  • Author
Posted (edited)

@guinness

With normal ziped files (compressed by program) 7-Zip returns 0. With my received packet it returns 2.

@My code:

Send.au3:

; --- Config ---
Global Const $IP = "87.98.235.245" ; server IP
Global Const $RequestFile = "request.txt" ; file with request to send
Global Const $RecvFile = "recv.txt" ; file to write received data
; --- End ---
Global $ServerRecv = ""

; --- System ---
TCPStartup()

$ServerSocket = TCPConnect($IP, 80)
If @error Then MsgBox(16, 1, @error)
ConsoleWrite("Connected with " & $IP & @CR)
TCPSend($ServerSocket, FileRead($RequestFile))
ConsoleWrite("Send ... " & @CR)
Sleep(200)
$ServerRecv = TCPRecv($ServerSocket, 1024 ^ 2, 1)
If @error Then MsgBox(16, 2, @error)
If $ServerRecv <> Binary("") Then
    ConsoleWrite("Recv... " & @CR)
    MsgBox(32, StringLen($ServerRecv), $ServerRecv)
    FileWrite($RecvFile, $ServerRecv)
EndIf
TCPCloseSocket($ServerSocket)
TCPShutdown()
; -------

I think it's simple! =)

My request:

POST /ajax/logon.php?t=login HTTP/1.1

Host: www.margonem.pl

User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:5.0.1) Gecko/20100101 Firefox/5.0.1

Accept: text/html, */*

Accept-Language: pl,en-us;q=0.7,en;q=0.3

Accept-Encoding: gzip,deflate

Accept-Charset: ISO-8859-2,utf-8;q=0.7,*;q=0.7

Connection: keep-alive

Content-Type: application/x-www-form-urlencoded; charset=UTF-8

X-Requested-With: XMLHttpRequest

Referer: http://www.margonem.pl/

Content-Length: 26

Cookie: ads=0-1; __utma=1.1328332501.1313174251.1313174151.1313174151.1; __utmb=1.1.10.1313174151; __utmc=1; __utmz=1.1313174151.1.7.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)

Pragma: no-cache

Cache-Control: no-cache

l=grimreaper&p=devilmaycry

This is example login request. After send we should received packet with ziped body. Edited by Adrian777
guinness Universalist

guinness

Posted
Posted (edited)

Please read the Forum rules as the website you specified is for a game >> http://www.autoitscript.com/forum/forum-2/announcement-13-forum-rules/

Edited by guinness

UDF List:

 
_AdapterConnections() • _AlwaysRun() • _AppMon() • _AppMonEx() • _ArrayFilter/_ArrayReduce • _BinaryBin() • _CheckMsgBox() • _CmdLineRaw() • _ContextMenu() • _ConvertLHWebColor()/_ConvertSHWebColor() • _DesktopDimensions() • _DisplayPassword() • _DotNet_Load()/_DotNet_Unload() • _Fibonacci() • _FileCompare() • _FileCompareContents() • _FileNameByHandle() • _FilePrefix/SRE() • _FindInFile() • _GetBackgroundColor()/_SetBackgroundColor() • _GetConrolID() • _GetCtrlClass() • _GetDirectoryFormat() • _GetDriveMediaType() • _GetFilename()/_GetFilenameExt() • _GetHardwareID() • _GetIP() • _GetIP_Country() • _GetOSLanguage() • _GetSavedSource() • _GetStringSize() • _GetSystemPaths() • _GetURLImage() • _GIFImage() • _GoogleWeather() • _GUICtrlCreateGroup() • _GUICtrlListBox_CreateArray() • _GUICtrlListView_CreateArray() • _GUICtrlListView_SaveCSV() • _GUICtrlListView_SaveHTML() • _GUICtrlListView_SaveTxt() • _GUICtrlListView_SaveXML() • _GUICtrlMenu_Recent() • _GUICtrlMenu_SetItemImage() • _GUICtrlTreeView_CreateArray() • _GUIDisable() • _GUIImageList_SetIconFromHandle() • _GUIRegisterMsg() • _GUISetIcon() • _Icon_Clear()/_Icon_Set() • _IdleTime() • _InetGet() • _InetGetGUI() • _InetGetProgress() • _IPDetails() • _IsFileOlder() • _IsGUID() • _IsHex() • _IsPalindrome() • _IsRegKey() • _IsStringRegExp() • _IsSystemDrive() • _IsUPX() • _IsValidType() • _IsWebColor() • _Language() • _Log() • _MicrosoftInternetConnectivity() • _MSDNDataType() • _PathFull/GetRelative/Split() • _PathSplitEx() • _PrintFromArray() • _ProgressSetMarquee() • _ReDim() • _RockPaperScissors()/_RockPaperScissorsLizardSpock() • _ScrollingCredits • _SelfDelete() • _SelfRename() • _SelfUpdate() • _SendTo() • _ShellAll() • _ShellFile() • _ShellFolder() • _SingletonHWID() • _SingletonPID() • _Startup() • _StringCompact() • _StringIsValid() • _StringRegExpMetaCharacters() • _StringReplaceWholeWord() • _StringStripChars() • _Temperature() • _TrialPeriod() • _UKToUSDate()/_USToUKDate() • _WinAPI_Create_CTL_CODE() • _WinAPI_CreateGUID() • _WMIDateStringToDate()/_DateToWMIDateString() • Au3 script parsing • AutoIt Search • AutoIt3 Portable • AutoIt3WrapperToPragma • AutoItWinGetTitle()/AutoItWinSetTitle() • Coding • DirToHTML5 • FileInstallr • FileReadLastChars() • GeoIP database • GUI - Only Close Button • GUI Examples • GUICtrlDeleteImage() • GUICtrlGetBkColor() • GUICtrlGetStyle() • GUIEvents • GUIGetBkColor() • Int_Parse() & Int_TryParse() • IsISBN() • LockFile() • Mapping CtrlIDs • OOP in AutoIt • ParseHeadersToSciTE() • PasswordValid • PasteBin • Posts Per Day • PreExpand • Protect Globals • Queue() • Resource Update • ResourcesEx • SciTE Jump • Settings INI • SHELLHOOK • Shunting-Yard • Signature Creator • Stack() • Stopwatch() • StringAddLF()/StringStripLF() • StringEOLToCRLF() • VSCROLL • WM_COPYDATA • More Examples...

Updated: 22/04/2018

bogQ Universalist

bogQ

Posted
Posted

probably

http://www.margonem.pl/

TCP server and client - Learning about TCP servers and clients connection
Au3 oIrrlicht - Irrlicht project
Au3impact - Another 3D DLL game engine for autoit. (3impact 3Drad related)



460px-Thief-4-temp-banner.jpg
There are those that believe that the perfect heist lies in the preparation.
Some say that it’s all in the timing, seizing the right opportunity. Others even say it’s the ability to leave no trace behind, be a ghost.

 

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...