Jump to content

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Find out more here. X
X


Photo

[UDF] Simple TCP

TCP Client Server Encrypted

  • Please log in to reply
10 replies to this topic

#1 matwachich

matwachich

    Polymath

  • Active Members
  • PipPipPipPip
  • 224 posts

Posted 12 November 2011 - 02:36 PM

Here i am again!
Today, i got a new UDF i want to share!

It's a clone of my TCPClass, but without POO, only classic functions!

It's faster, easier, and the client is more performant, cause it's now very simple to handle mutiple clients per script.

Here is v 1.0 (deprecated) (63 downloads)

Update v 1.2
Here is the new version.
Change Log:
Legend: ------- +: Addition, -: Remouved/Deprecated, *: Modified, !: Bug correction Change Log: ----------- === Version 1.2 === (24/11/2011) *: Now UTF-8 String are correctly processed (thanks jchd!) === Version 1.1.1 === (21/11/2011) +: Added 2 helper functions (see documentation) !: Another bug corrected in _TCPServer_SocketList. Thanks Tetdoss (again!) === Version 1.1 === (20/11/2011) +: ZLib data compression, optional (disabled by default) !: Client's properties were not reset when a client disconnects !: Incorrect _TCPServer_SocketList. Now works well (thanks Tetdoss) === Version 1.0 === (11/11/2011) First public version


Archive:
Attached File  SimpleTCP_1.2.zip   118.26KB   746 downloads

Edited by matwachich, 24 November 2011 - 04:11 PM.








#2 davidkim

davidkim

    Adventurer

  • Active Members
  • PipPip
  • 112 posts

Posted 14 November 2011 - 07:13 AM

thanks matwachich
I interest am many quite in AutoitScript.From that is [http://cafe.naver.com/autoitscript] Korea of cafe(blog) to be operating, English cannot well.Many help it requests.To read, it stands it thanks.

#3 matwachich

matwachich

    Polymath

  • Active Members
  • PipPipPipPip
  • 224 posts

Posted 25 November 2011 - 10:29 AM

Update

#4 autoall

autoall

    Seeker

  • Active Members
  • 21 posts

Posted 28 November 2011 - 02:05 PM

very nice job!

#5 denchu

denchu

    Seeker

  • Active Members
  • 6 posts

Posted 23 May 2012 - 08:12 AM

Hi. Very good job.
Is there a function to flush buffer of client after read received data? I plan to use your udf in my scripts that already have a file transfer routine. I transfer big binary files(about several gigobytes). This amount of data cannot be transfered by one time and even can't be held in memory. I transfer it by chunks. Script read the chunk from file and send it by native tcpsend function, in many cases script don't send the chunk by one tcpsend command and split it on "sub"chunks. There is part of my script:
if not BinaryLen($sBuffer[$x]) then $sBuffer[$x] = FileRead($f_fopen[$x], $chunk_size) $f_Byte = TCPsend($aSocket[$x], $sBuffer[$x]) ; TCPSend returns the number of bytes sent If @error  Then     ConsoleWrite('!Lost Connection')     FileClose($f_fopen[$x]) else      $sBuffer[$x] = BinaryMid($sBuffer[$x], $f_Byte+1)      $f_size[$x]-=$f_byte      if not $f_size[$x] then          ConsoleWrite('+Upload Complited')          FileClose($f_fopen[$x])      endif endif



Is it possible to receive data by chunks with your UDF ? Read data from buffer then flush buffer save to disk and read again.

And a second question. How the clinet.connect func works, it sending some keepalive packets or just reconnect socket if needed or i need to connect manualy every time when i needs to send data after some time(10min for example).

#6 matwachich

matwachich

    Polymath

  • Active Members
  • PipPipPipPip
  • 224 posts

Posted 23 May 2012 - 07:06 PM

For the 2nd question, you don't need to connect every time you want to send data, you only need to reconnect when _TCPClient_IsConnected returns 0.

For the first question, for now, NO. But i think it's simply modifiable, but the problem: i don't have any time these days to focus on this (exams!) sorry.

You will have to study the function _TCPxxx_Process to see how it works

#7 denchu

denchu

    Seeker

  • Active Members
  • 6 posts

Posted 23 May 2012 - 10:05 PM

When i try to send binary file data over _TCPServer_Send without compression and encoding i have string on other end. I inserted some debugging command(consolewrite(isbinary($var)) in all used scripts and found that binary data become string after calling TCPRecv function in _TCPClient.au3. I suggests a bug in TCPRecv function that recognize data as string. So i alweys use cryptpass in my script.

Next problem i have is if i sends file in chunks, i have data lost on other end. For example i transfer file of size 400kb by chunks of 122880bytes. By consolewrite i know that server sends 4 chunks but client receives only 2 - first and last. If i try to transfer bigger file i have 3 of 6 chunks received etc. It is code of my send function.
AutoIt         
local $aSocket=_TCPServer_Socketlist() for $x=1 to $aSocket[0][0]       if _TCPServer_ClientPropertyGet($aSocket[$x][0],0)='SendingBinaryFileData' then            _ConsoleWrite('SendingBinaryFileData')            _SendChunk($aSocket[$x][0])       endif Next func _SendChunk($f_Socket) local $f_fopen=_TCPServer_ClientPropertyGet($f_Socket,2);get handle of file to send local $f_size=_TCPServer_ClientPropertyGet($f_Socket,4);get file size to send _ConsoleWrite('Need to send '&$f_size) _Consolewrite("Read next chunk") local $Buffer=FileRead($f_fopen, $chunk_size)   $f_size-=BinaryLen($Buffer) _CONSOLEWRite("isbinary="&isbinary($Buffer)) $f_Byte=_TCPServer_Send($f_Socket, $Buffer) ; it returns amount of data sent in crypted state with chr(2) and chr(3) - useless info If @error Then        _ConsoleWrite('! Lost Connection! error='&@Error, 1)          FileClose($f_fopen)        _TCPServer_ClientPropertySet($f_Socket, 0,'')         return -1 elseif $f_size<=0 then        _ConsoleWrite('+ Upload Complited', 0)        FileClose($f_fopen)       _TCPServer_ClientPropertySet($f_Socket, 0,'') EndIf _TCPServer_ClientPropertySet($f_Socket,4,$f_size) return 1 endfunc


#8 denchu

denchu

    Seeker

  • Active Members
  • 6 posts

Posted 24 May 2012 - 07:50 AM

Next problem i have is if i sends file in chunks, i have data lost on other end. For example i transfer file of size 400kb by chunks of 122880bytes. By consolewrite i know that server sends 4 chunks but client receives only 2 - first and last. If i try to transfer bigger file i have 3 of 6 chunks received etc. It is code of my send function.


I have solved second problem! Just a little patch to your code. You flush the buffer every time when __Check_Buffer() founds new data between chr(2) and chr(3). At first i changed chr(2) and chr(3) to chr(2)&chr(2)&chr(2)&chr(2) because of chr(2) is often placed in binary files. And at main i replace buffer flushing to shrink buffer to last border of last packet. It's a part of your code(TCPClient.au3) after line 455.

AutoIt         
 ; Check buffer ok $tmp = __TCPClient_CheckBuffer($iClient) If IsArray($tmp) Then   For $elem In $tmp    If $__TCPClient_Clients[$iClient][$__TCPc_CB_RECV] Then _     Call($__TCPClient_Clients[$iClient][$__TCPc_CB_RECV], $iClient, $elem)   Next   ; ---   $__TCPClient_Clients[$iClient][$__TCPc_BUFFER] = StringRegExpReplace($__TCPClient_Clients[$iClient][$__TCPc_BUFFER],'.*'&Chr(3)&Chr(3)&Chr(3)&Chr(3),'') ;it was ""(flush) here EndIf EndFunc Func __TCPClient_CheckBuffer($iClient) If $__TCPClient_Clients[$iClient][$__TCPc_BUFFER] = "" Then Return 0 ; --- $bet = _StringBetween($__TCPClient_Clients[$iClient][$__TCPc_BUFFER],Chr(2)&Chr(2)&Chr(2)&Chr(2),Chr(3)&Chr(3)&Chr(3)&Chr(3)) If IsArray($bet) Then   For $i = 0 To UBound($bet) - 1    $bet[$i] = __TCPClient_dCry($iClient, $bet[$i])   Next   Return $bet Else   Return 0 EndIf EndFunc


Another way to solve this issue is sending some confirmation packet from client every chunk and send next chunk only after recieving that packet on server.

P.S.: Taking data from buffer by some marker(chr(2)) it's a bad idea because there is no guarantee that your marker will never included in transfered data. Better way is make some packet header, for example chr(2)&$Packetdatasize&chr(2). When script checks the buffer it find header, proof that packet is longer that $PacketDataSize+header_lenght and then get $Packetdatasize of data from buffer.

Edited by denchu, 24 May 2012 - 08:02 AM.


#9 denchu

denchu

    Seeker

  • Active Members
  • 6 posts

Posted 25 May 2012 - 02:34 PM

I rewrote several functions in your UDF, so you may use my changes to improve your UDF:
AutoIt         
Func _TCPServer_Send($iSocket, $Data) If Not _TCPServer_IsStarted() Then Return SetError(-1, 0, 0) If Not __TCPServer_IsSocketValid($iSocket) Then Return SetError(1, 0, 0) ; --- ;Consolewrite('Data to send is '&VarGetType($Data)&@crlf) $Data=__TCPServer_Cry($Data) local $DataType=isbinary($data) Local $DataSize=Binarylen($DATA) local $Header=chr(2)&chr(3)&chr(2)&Hex($DataType,2)&Hex($DataSize,16)&chr(3)&chr(2)&chr(3) local $f_byte=TcpSend($__TCPServer_Sockets[$iSocket][0],$Header) if @error then   return 0 endif if $f_byte<>binarylen($Header) then   Seterror(1,$f_byte,0)   return 0 endif local $failtimer=timerinit() ;Consolewrite('After servercrypt '&VarGetType($Data)&@crlf) while binaryLen($Data)   if $DataType then $Data=Binary($Data)   $f_byte= TcpSend($__TCPServer_Sockets[$iSocket][0],$DATA)   if @error then    return 0   endif   if $f_byte > 0 then    $failtimer=timerinit()    $Data=BinaryMid($Data,$f_byte+1)   Elseif TimerDiff($failtimer)>1000 then    seterror(2,BinaryLen($Data),0)    return 0   endif   local $tmprec=TCPRecv($__TCPServer_Sockets[$iSocket][0],1)   if $tmprec=chr(2) or ($tmprec=chr(3) and binaryLen($Data)>0) then    seterror(3,$tmprec,0)    return 0   endif   if $tmprec=chr(3) and not binarylen($Data) then return 1   $failtimer=timerinit()   while $tmprec<>chr(3)    if TimerDiff($failtimer)>=$__TCPSrv_Info_TIMEOUT Then ; Timed-out!     If $__TCPSrv_Info_CB_TIMEDOUT <> "" Then _      Call($__TCPSrv_Info_CB_TIMEDOUT, $iSocket, $__TCPServer_Sockets[$iSocket][1], BinaryLen($__TCPServer_Sockets[$iSocket][2]))     seterror(4,BinaryLen($Data),0)     return 0    endif    if $tmprec=chr(2) then     seterror(5,BinaryLen($Data),0)     return 0    endif    $tmprec=TCPRecv($__TCPServer_Sockets[$iSocket][0],1)   wend wend return 1 EndFunc

AutoIt         
Func _TCPServer_Process() If Not _TCPServer_IsStarted() Then Return SetError(-1, 0, 0) ; --- Local $hNewSocket, $recv, $tmp ; --- ; Check new connexions $hNewSocket = TCPAccept($__TCPSrv_Info_SOCKET) If $hNewSocket <> -1 Then   __TCPServer_StoreNewClient($hNewSocket) EndIf ; --- ; Process Connected clients For $i = 1 To $__TCPServer_Sockets[0][0]   If $__TCPServer_Sockets[$i][0] = -1 Then ContinueLoop   ; ---   $recv = TCPRecv($__TCPServer_Sockets[$i][0], 24)   ; ---   ; Check disconnection   If @error Then    If $__TCPSrv_Info_CB_LOST <> "" Then _     Call($__TCPSrv_Info_CB_LOST, $i, $__TCPServer_Sockets[$i][1])    ; ---    TCPCloseSocket($__TCPServer_Sockets[$i][0])    __TCPServer_ResetSocket($i)   EndIf   ; ---   ; Check Receiving   If $recv Then ;Header received    $__TCPServer_Sockets[$i][3] = TimerInit()    if stringleft($recv,3)<>chr(2)&chr(3)&chr(2) or stringright($recv,3)<>chr(3)&chr(2)&chr(3) then     SetError(3,'Bad Header')     Consolewrite('Bad Header received')     TCPRecv($__TCPServer_Sockets[$i][0], 999999999)     continueloop    endif    local $Header=StringMid($recv,4,18)    Local $DataSize=Dec(StringRight($Header,16))    Local $DataType=Dec(StringLeft($Header,2))    ConsoleWrite('Header='&$HEader&' Datasize='&$DataSize&'bytes datatype='&$DataType&@crlf)    $__TCPServer_Sockets[$i][2]=''    if $DataType<>0 and $DataType<>1 then     SetError(1,$DataType)     Consolewrite('Bad datatype in header')     TCPRecv($__TCPServer_Sockets[$i][0], $DataSize)     continueloop    endif    #region recieving data    While BinaryLen($__TCPServer_Sockets[$i][2])<$DataSize     $Recv=TCPRecv($__TCPServer_Sockets[$i][0], $DataSize-BinaryLen($__TCPServer_Sockets[$i][2]),$DataType)     if $recv then      $__TCPServer_Sockets[$i][2]&=$recv      If $__TCPSrv_Info_CB_RECEIVING <> "" Then _        Call($__TCPSrv_Info_CB_RECEIVING, $i, $__TCPServer_Sockets[$i][1], BinaryLen($__TCPServer_Sockets[$i][2]))      $__TCPServer_Sockets[$i][3]=TimerInit()     elseif timerdiff($__TCPServer_Sockets[$i][3])>= $__TCPSrv_Info_TIMEOUT then      If $__TCPSrv_Info_CB_TIMEDOUT <> "" Then _        Call($__TCPSrv_Info_CB_TIMEDOUT, $i, $__TCPServer_Sockets[$i][1], BinaryLen($__TCPServer_Sockets[$i][2]))      ; ---      TCPSend($__TCPServer_Sockets[$i][0],chr(2))      SetError(2,BinaryLen($__TCPServer_Sockets[$i][2]))      $__TCPServer_Sockets[$i][2] = ""      exitloop     endif     ;Consolewrite('+Received '&BinaryLen($__TCPServer_Sockets[$i][2])&' of '&$DataSize&'bytes'&@crlf)    wend    ;Consolewrite('Calling _TCP_RECV function, data is '& __TCPServer_dCry($__TCPServer_Sockets[$i][2]) &@crlf)    if BinaryLen($__TCPServer_Sockets[$i][2])=$DataSize then     TCPSend($__TCPServer_Sockets[$i][0],chr(3))    else     TCPSend($__TCPServer_Sockets[$i][0],chr(2))    endif    If $__TCPSrv_Info_CB_RECV <> "" Then _       Call($__TCPSrv_Info_CB_RECV, $i, $__TCPServer_Sockets[$i][1], __TCPServer_dCry($__TCPServer_Sockets[$i][2]))    $__TCPServer_Sockets[$i][2]=''    #endregion    ; ---   EndIf   ; ---   ; Check timeout   If $__TCPServer_Sockets[$i][2] Then    If TimerDiff($__TCPServer_Sockets[$i][3]) >= $__TCPSrv_Info_TIMEOUT Then ; Timed-out!     If $__TCPSrv_Info_CB_TIMEDOUT <> "" Then _      Call($__TCPSrv_Info_CB_TIMEDOUT, $i, $__TCPServer_Sockets[$i][1], BinaryLen($__TCPServer_Sockets[$i][2]))     ; ---     $__TCPServer_Sockets[$i][2] = ""    EndIf   EndIf   ; --- Next ; --- Return 1 EndFunc

Send and recieve functions have no "string compare" commands and does not stuck on big data chunks. Now script make packet with 24bytes of header with information about data length and data type(bynary/string). Searching for magic sequence in all amount of data is not needed anymore. I have no problems with > 100Mb files, transfer speed is about 5Mb/s. Plain functions that i wrote in the past have much speed but have no stability of this.

AutoIt         
Func __TCPServer_Cry($Data) ; --- Local $bin = 1 If Not IsBinary($Data) Then   $bin = 0   $Data = StringToBinary($Data, 4) EndIf ; --- If $__TCPSrv_Info_CRYKEY Then $Data = _Crypt_EncryptData($Data, $__TCPSrv_Info_CRYKEY, $CALG_USERKEY) ; --- If $__TCPSrv_Info_COMPRESSION Then $Data = _ZLIB_Compress($Data, $__TCPSrv_Info_COMPRESSION) ; --- If Not $bin Then $Data = StringTrimLeft($Data, 2) ; --- Return $Data EndFunc Func __TCPServer_dCry($Data) ; --- Local $bin = 0 If StringLeft($Data, 2) = "0x" Then   $bin = 1   $Data = Binary($Data) Else   $Data = Binary("0x" & $Data) EndIf ; --- If $__TCPSrv_Info_COMPRESSION Then $Data = _ZLIB_Uncompress($Data) ; --- If $__TCPSrv_Info_CRYKEY Then $Data = _Crypt_DecryptData($Data, $__TCPSrv_Info_CRYKEY, $CALG_USERKEY) If Not $bin Then $Data = BinaryToString($Data, 4) ; --- Return $Data EndFunc

crypt and decrypt functions now made zlib compression(if needed) and prepare string to transfer even if encryption is disabled.

#10 matwachich

matwachich

    Polymath

  • Active Members
  • PipPipPipPip
  • 224 posts

Posted 25 May 2012 - 05:52 PM

Good work!
Gonna check all this when i have time.

#11 matwachich

matwachich

    Polymath

  • Active Members
  • PipPipPipPip
  • 224 posts

Posted 02 August 2012 - 05:33 PM

I really like you ideas!
Can you post the complete server and client scripts? I wanna study them more.





Also tagged with one or more of these keywords: TCP Client Server, Encrypted

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users