Jump to content

Autoit-Socket-IO - Networking in AutoIt made simple!


Recommended Posts

  • Replies 60
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Version 2.x.x and 3.x.x has been moved to branch 3.x About Autoit-Socket-IO Autoit-Socket-IO is a event driven TCP/IP wrapper heavily inspired from Socket.IO with focus on user friendliness

Hello. I am not dead! I suggest that everyone that uses this UDF in production, updates to this version. Version 1.5.0 (This update DOES NOT break scripts) Added AutoIt like docs under

What maybe would work too is to $nPing = ping() the server right before TCPConnect and setting this $nPing, if not unusally high, as the current tcptimeout. Then connect. And after success reseting th

I have the same issue,

I made some minor edits and decided to run the Server on my dedicated server (run via script) and run (via script) the client on my computer. It would not connect at all. I used a port that has been used for years. Other example Autoit Server & Clients examples work, but the client does not need to connect with V4 or V3 of this. I can connect to the AutoIT Socket Server with a different Client example, but it does not have the features I want (was testing purposes only). Thus this leads me into believing Server has no issues but the Client has something not allowing it to connect.

Tried Default Chat Examples in V1/V3/V4 with the the same ip/port the linked below example.

This, for example, connects instantly and works fine, so the only conclusion is the Client does not connect at all to an external Server.

Doug

Edit:

@Rurorita helped me and figured it out. You need to have (Opt('TCPTimeout', X)) inside Io_Connect, Calling it before the Io_Connect or after has no effect. V4 console prints 

Quote

SocketIO.au3: Because you are using Autoit version 3.3.14.5 Opt('TCPTimeout') has been set to 5. You could manually use another value by putting Opt('TCPTimeout', 5) (once) after _Io_Connect or _Io_listen. Why this is done you could read more about here: https://www.autoitscript.com/trac/autoit/ticket/3575

 

which is wrong. Very thankful to Rurorita for assistance and figuring out the problem.

Edited by TehDoug
Link to post
Share on other sites

The issue comes in effect when 1. the ping is greater then the timeout or 2. the server doesnt accept in time. So if the server is on load then the timeout can appear too.

He only had to set a timeout coresponding to his ping then it worked nicely. However if the timeout is set too high then lags can come in effect.

But this issue seems to variate and i have to adress it in the framework once i understand why it behaves so strange.

Link to post
Share on other sites

Thought I'd give a better way of handling higher pings:

Took it from a different Example, instead of timeout 100 (sometimes still does not connect, 150 100% does) but it makes input lag. Below seems to fix those issues will need fixing up, just though I'd post an example of a re-try loop rather than TimeOut which causes pauses.

Local $nMaxTimeout = 10
      Local $socket = TCPConnect($sAddress, $iPort)
      If @error = 10060 Then
            ; Timeout occurs try again
            $nMaxTimeout -= 1
            If $nMaxTimeout < 0 Then
                MsgBox(BitOR($MB_SYSTEMMODAL, $MB_ICONHAND), "", "Could not connect, after " & 10 - $nMaxTimeout & " TimeOut")
                Return False
            EndIf
        ElseIf @error Then
            $iError = @error
            ; The server is probably offline/port is not opened on the server.
            MsgBox(BitOR($MB_SYSTEMMODAL, $MB_ICONHAND), "", "Could not connect, Error code: " & $iError)
            Return False
         EndIf

 

Link to post
Share on other sites

What maybe would work too is to $nPing = ping() the server right before TCPConnect and setting this $nPing, if not unusally high, as the current tcptimeout. Then connect. And after success reseting the tcptimeout. maybe thats enough to fix any lag in the hope that tcpsend is not influenced by the timeout. if so then it maybe helps to have the ping for each socket stored. I will do some research and probably implement something into the framework to combat this.

Link to post
Share on other sites

Hi Guys,

I appriciate the love and I want to answer all the questions, however im working full time so I havent had the chance to make any updates to the IO project, when its vacation time i will probably be a little bit more active.

Link to post
Share on other sites
  • 3 weeks later...

To those who messaged me about the modified variant. I will release the Modified SocketIO UDF under the name SocketIOEx on Github and this forum. The framework around SocketIoEx will simply be called _netcode UDF.

It will not be so soon, as i need to address certain issues and thoughts. I want to make the SocketIOEx more redundant, more DDOS proof and faster overall. For that i, for example, need to recode certain Autoit Functions like TCPSend() and TCPRecv(), as those come with known issues. _netcode UDF will be a plug and play UDF, it manages the whole network stuff and also reacts to certain bugs that may appear and so on. It will be really easy to use, someone could call the Send function with hundreds of MB of data and the _netcode would manage that. Also, whats not done yet, i plan to add a feature where a Client could get parts of the same data set from different providers to maxout the download speed. Similiar to torrent.

But currently im working on Support for Tor, its already working, but the Tor app comes with a couple of issues.

Link to post
Share on other sites
5 hours ago, Rurorita said:

But currently im working on Support for Tor, its already working, but the Tor app comes with a couple of issues.

Hi, I am curious, what kind of issues are you experiencing with Tor? I may be able to help you as I made a GUI for Tor a few years back. Also, how are you adding support for Tor in your UDF? Are you directly interfacing with the SOCKS proxy?

EasyCodeIt - A cross-platform AutoIt implementation

DcodingTheWeb Forum - Follow for updates and Join for discussion

Link to post
Share on other sites
11 hours ago, TheDcoder said:

Hi, I am curious, what kind of issues are you experiencing with Tor? I may be able to help you as I made a GUI for Tor a few years back. Also, how are you adding support for Tor in your UDF? Are you directly interfacing with the SOCKS proxy?

Hi, the issue i was speaking of yesterday was actually an issue with SocketIO. When i was sending packets greater then what could be transmitted per second then the Recv function of SocketIO wasnt returning the full packet. It has nothing todo with Tor, it is just that i never had in mind before that an internet connection could actually be so slow that when the recv function is called, that the packet isnt yet fully there and ready to be processed. At first i was thinking that tor was limited in some form, but actually i just have to recode this function. Also i have to find a way how to ask the Tor app when we are connected to the .onion adress and how to check when a connection is dead.

For now what i do is that both parties, the server and the client, start a tor session.

The Server Side runs a listener like usually and also starts up a Tor Session creating a Hidden Service, which then connects to the listener.

The Client Side starts a regular tor session with the Socks interface enabled and then connects to that. This tor Session is then commanded to connect to the .onion address.

It is really basic and working quite nice and fast. I just have to change certain parts of SocketIO to handle incomplete packets, but in a way it can not be misused.

I made an RSA example and modified it a little to use it with tor. The transmitted packets are small and therefore it works really fast. Even if i spam the Send function.

Spoiler

rja6dJA.png

But i also coded an desktop Screenshot example. Each packet is atleast 200kb in size. But thats not a speed that the tor app can manage to send per second always. So i ended up with incomplete packets.

Edited by Rurorita
Link to post
Share on other sites

@Rurorita The "packets" you are referring to may not exist, the TCP packets may be split into smaller ones by any one of the network relays... and you don't really have to worry about this as the OS collects all these packets and puts their data into a buffer (from which the TCPRecv function reads from).

That means you should not rely on the receive function to give you the whole information in a single call, always buffer the input until you have the information you need... this means calling the receive function multiple times and storing the result in a string.

EasyCodeIt - A cross-platform AutoIt implementation

DcodingTheWeb Forum - Follow for updates and Join for discussion

Link to post
Share on other sites
14 minutes ago, TheDcoder said:

@Rurorita The "packets" you are referring to may not exist, the TCP packets may be split into smaller ones by any one of the network relays... and you don't really have to worry about this as the OS collects all these packets and puts their data into a buffer (from which the TCPRecv function reads from).

That means you should not rely on the receive function to give you the whole information in a single call, always buffer the input until you have the information you need... this means calling the receive function multiple times and storing the result in a string.

This isnt the issue. And Socketio is already doing that. Looping until the buffer is empty. The loop is Exited once the TCPRecv returns "". The issue is that the buffer doesnt contains the whole packet yet because its gets emptied right in the transmission. If the whole packet is 100kb in size and only 50kb got transmitted yet then the buffer only hold this 50kb. Once we empty that buffer, we only have an incomplete packet. Im already addressing that.

Edited by Rurorita
Link to post
Share on other sites
6 minutes ago, Rurorita said:

Looping until the buffer is empty. The loop is Exited once the TCPRecv returns "".

This approach will not work because the buffer being empty doesn't mean that you got all packets, more of them may be on the way. You need to use protocol specific logic to determine if you have a received all data. IRC for example, ends each "command" with a new line so that the client knows it got all the necessary information to process that specific "command".

EasyCodeIt - A cross-platform AutoIt implementation

DcodingTheWeb Forum - Follow for updates and Join for discussion

Link to post
Share on other sites
Just now, TheDcoder said:

This approach will not work because the buffer being empty doesn't mean that you got all packets, more of them may be on the way. You need to use protocol specific logic to determine if you have a received all data. IRC for example, ends each "command" with a new line so that the client knows it got all the necessary information to process that specific "command".

Right thats the point. But i also need to take in account that certain parts of a packet can get missing and that can also be the packet seperator. So i have to Validate.

Link to post
Share on other sites
25 minutes ago, Rurorita said:

But i also need to take in account that certain parts of a packet can get missing and that can also be the packet seperator.

No need, that is why TCP exists, it always makes sure that the data you receive is intact and is in the same order as it was sent.

EasyCodeIt - A cross-platform AutoIt implementation

DcodingTheWeb Forum - Follow for updates and Join for discussion

Link to post
Share on other sites
2 minutes ago, TheDcoder said:

No need, that is why TCP exists, it always makes sure that the data you receive is intact and is in the same order as it was sent.

That is what TCP is meant to prevent. But it does still happen. Im including certain packet validation technics and also have a feature where lost or bad packets get rerequested.

Link to post
Share on other sites
5 minutes ago, Rurorita said:

But it does still happen.

I don't think that is likely at all, otherwise all applications depending on TCP would be broken. The more probable situation is that there is something in your code which is sending data in the incorrect order/format.

EasyCodeIt - A cross-platform AutoIt implementation

DcodingTheWeb Forum - Follow for updates and Join for discussion

Link to post
Share on other sites
Just now, TheDcoder said:

I don't think that is likely at all, otherwise all applications depending on TCP would be broken. The more probable situation is that there is something in your code which is sending data in the incorrect order/format.

It doesnt happen in my code, im just coding preventions technics for the case. In reality it does happen, no clue how often. But i spoke to alot of devs and alot of them told me about similiar issues. I try to prevent the whole bad or missing packet issue and make things easy to use.

Link to post
Share on other sites

@Rurorita I have written several programs in AutoIt which use TCP for communication, and I have never experienced issues with corrupted data :think:

I suggest you investigate the issue on a lower-level, the issue is on the sender's side according to what you have told me.

EasyCodeIt - A cross-platform AutoIt implementation

DcodingTheWeb Forum - Follow for updates and Join for discussion

Link to post
Share on other sites

What is TCP?

The Transmission Control Protocol (TCP) is a communications standard that enables application programs and computing devices to exchange messages over a network. It is designed to send packets across the internet and ensure the successful delivery of data and messages over networks.

 

Fully agree... this is handled on the protocal level you don't need to manage this on the application level 😉

Link to post
Share on other sites
  • 2 weeks later...

Hey guys and girls,

"C:\Users\Administrator\Desktop\PubTestwithaccounts\Includes\Networking\Dependencies\Autoit-Serialize-1.0.0\Serialize.au3" (40) : ==> Array variable has incorrect number of subscripts or subscript dimension range exceeded.:
Local $val = $part[1]
Local $val = ^ ERROR

 

Func _UnSerialize(Const $source)
    Local Const $parts = StringSplit($source, "#")

    For $i = 1 To $parts[0]
        Local $part = StringSplit($parts[$i], '|', 2)
        Local $type = $part[0]
        Local $val = $part[1]

        Switch $type
            Case "s"
                Return BinaryToString($val)
            Case "a"
                Return __Serialize_UnSerializeArray($val)
            Case "o"
                Return __Serialize_UnSerializeScriptingDictionary($val)
            Case "b"
                Return $val == 1
            Case "Int32"
                Return Number($val, 1)
            Case "Int64"
                Return Number($val, 2)
            Case "Ptr"
                Return Ptr($val)
            Case "Binary"
                Return Binary($val)
            Case "Double"
                Return Number($val, 3)
            Case "Keyword"
                Return Null
        EndSwitch

    Next

EndFunc   ;==>UnSerialize

 

I guess it might be to do with none client connections, as it is hosted on my dedicated server which had a problem with RDP bots flooding trying to login with no password. But I also can't confirm that is the issue.

Anyone know a way to prevent this? I tried making sure "#" existed but that caused other issues and don't want to spaghetti code it myself.

Also closing the Client seems to Disconnect twice:

Quote

+[2021-04-30 20:31:22][SERVER][SUCCESS] Successfully fired event "auth" "$socket = 1148", "$mySocket = 1132"
>[2021-04-30 20:33:24][SERVER][INFO]    Attempting to fire event "disconnect" "$socket = 1232", "$mySocket = 1132"
+[2021-04-30 20:33:24][SERVER][SUCCESS] Successfully fired event "disconnect" "$socket = 1232", "$mySocket = 1132"
>[2021-04-30 20:33:24][SERVER][INFO]    Attempting to fire event "disconnect" "$socket = 1148", "$mySocket = 1132"
+[2021-04-30 20:33:24][SERVER][SUCCESS] Successfully fired event "disconnect" "$socket = 1148", "$mySocket = 1132"

D x

Edit:

it does seem likes its from a bot connection, it would seem there needs to be a way to prevent none Autoit-Socket-IO clients from connecting as the server can't handle any incoming packets it's not designed to handle and thus crashes the server. I'm currently trying to check if it contains "|" if not return "Broke" then in __Io_HandleRecvdPackage if = "Broke" then return back to loop. Not sure if this is the best way but will be giving it a try to see if that prevents it.

Edited by TehDoug
Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By tarretarretarre
      About AutoIt-Events
      AutoIt-Events is an event Observer and is a core dependency for Autoit-Socket-IO but can be used for any Autoit project.
      Example
      #include "Event.au3" ; Subscribe listeners _Event_Listen(UserCreatedEvent, SendWelcomeMail) _Event_Listen(UserCreatedEvent, RegisterNewsLetter) ; Fire event _Event(UserCreatedEvent, @UserName, "tarre.islam@gmail.com") Func UserCreatedEvent(Const ByRef $oEvent, $name, $email) ; via $oEvent you can pass data to its listeners $oEvent.add("name", $name) $oEvent.add("email", $email) $oEvent.add("id", 1) EndFunc Func SendWelcomeMail(Const $oEvent) MsgBox(64, "Welcome mail sent", "Welcome mail sent to " & $oEvent.item("name") & " with email " & $oEvent.item("email")) EndFunc Func RegisterNewsLetter(Const $oEvent) MsgBox(64, "News letter registred", "News letter bound to user id " & $oEvent.item("id")) EndFunc  
      The code is also available at Github
       
      Autoit-Events-1.0.0.zip
    • By tarretarretarre
      Version 2.x.x and 3.x.x has been moved to branch 3.x
      About Autoit-Socket-IO
      Autoit-Socket-IO is a event driven TCP/IP wrapper heavily inspired from Socket.IO with focus on user friendliness and long term sustainability.
      I constantly want to make this UDF faster and better, so if you have any suggestions or questions (beginner and advanced) Do not hesitate to ask them, I will gladly help!
      Key features
      Simple API 99% data-type serialization thanks to Autoit-Serialize Can easily be extended with your own functionality thanks to Autoit-Events "Educational" examples Data encryption thanks to _<Crypt.au3> Limitations
      Speed. This UDF will sacrifice some speed for convenience Read more in the official thread
    • By TheXman
      Purpose (from Microsoft's website)
      The HTTP Server API enables applications to communicate over HTTP without using Microsoft Internet Information Server (IIS). Applications can register to receive HTTP requests for particular URLs, receive HTTP requests, and send HTTP responses. The HTTP Server API includes SSL support so that applications can exchange data over secure HTTP connections without IIS.
      Description
      There have been several times in the past that I wanted to either retrieve information from one of my PCs or execute commands on one of my PCs, whether it was from across the world or sitting on my couch.  Since AutoIt is one of my favorite tools for automating just about anything on my PC, I looked for ways to make to make it happen.  Setting up a full blown IIS server seemed like overkill so I looked for lighter weight solutions.  I though about creating my own AutoIt UDP or TCP server but that just wasn't robust enough,  Then I found Microsoft's HTTP Server API and it looked very promising.  After doing a little research into the APIs, I found that it was flexible & robust enough to handle just about any of the tasks that I required now and in the future.  So a while back I decided to wrap the API functionality that I needed into an AutoIt UDF file to allow me to easily create the functionality I needed at the time.  It has come in very handy over the years.  Of course it wasn't all wrapped up with a nice little bow like it is now.  That only happened when I decided to share it with anyone else who could use it or learn from it.
      The example file that I included is a very granular example of the steps required to get a lightweight HTTP Server up and listening for GET requests.  The UDF is a wrapper for the Microsoft APIs.  That means to do anything over and above what I show in the example, one would probably have to have at least a general knowledge of APIs or the ability to figure out which APIs/functions to use, what structures and data is needed to be passed to them, and in what order.  However, the UDF gives a very solid foundation on which to build upon.  Of course, if anyone has questions about the UDF or how to implement any particular functionality, I would probably help to the extent that I could or point you in the right direction so that you can figure out how to implement your own solution.
      The APIs included in the UDF are the ones that I needed in the past to do what I needed to do.  If any additional APIs need to be added to the UDF file, please make those suggestions in the related forum topic.
      Being that this is basically an AutoIt wrapper for the Microsoft API functions, there's no need to create AutoIt-specific documentation.  All of the UDF functions, structures, constants, and enumerations are named after their Microsoft API counterparts.  Therefore, you can refer to Microsoft's extensive documentation of their HTTP Server API.  As stated earlier, if there is one or more APIs that you find yourself needing for your particular solution, please suggest it in the related Example Scripts forum topic.
      Related Links
      Microsoft HTTP Server API - Start Page
      Microsoft HTTP Server API - API v2 Reference
      Microsoft HTTP Server API - Programming Model
    • By YogendraAtluri
      Hi, 
      I am new to AutoIT scripting and I am still learning. I am trying to communicate with a Labview application that acts like a server. it basically takes commands from the client. But for some commands, it also send back some data. 
      When i am sending commands from my script, i can see that the labview is getting them. But i am not able to get anything back. I tried different code pieces that are available online in the forum.
      This is the working piece of code which i been using to send data.
      #cs This module is used to establish tcp connection with lab view #ce #include <File.au3> Func SendCmd($cmd) TCPStartup() Local $IpAddress="192.168.10.101" Local $Port="5353" $Labview = TCPConnect($IpAddress,$Port) If @error Then ConsoleWrite('!--> TCPConnect error number ( ' & @error & ' ).' & @CRLF) TCPCloseSocket($Labview) TCPShutdown() Exit EndIf TCPSend($Labview, $cmd & @CRLF) TCPCloseSocket($Labview) TCPShutdown() EndFunc SendCmd("wt42d")  
      This is slightly modified code to send and receive data, which is not working. I am not getting any response back
      SendCmd("galil") Func SendCmd($cmd) TCPStartup() Local $IpAddress="192.168.10.101" Local $Port="5353" $Labview = TCPConnect($IpAddress,$Port) If @error Then ConsoleWrite('!--> TCPConnect error number ( ' & @error & ' ).' & @CRLF) TCPCloseSocket($Labview) TCPShutdown() Exit EndIf TCPSend($Labview, $cmd & @CRLF) $ip = @IPAddress1 ;create listening socket $Listensocket = TCPListen($ip, $Port) ConsoleWrite("Listening to Socket - " & $Listensocket & @CRLF) If $Listensocket = -1 Then ConsoleWrite("Exiting..." & @CRLF) Exit EndIf ;Accept incoming clients and recieve info While 1 $connectedsocket = TCPAccept($Listensocket) ConsoleWrite("Connecting to Socket - " & $connectedsocket & "Error -" & @error & @CRLF) If $ConnectedSocket >= 0 Then $ip2 = TCPRecv($connectedsocket,1000000) EndIf WEnd TCPCloseSocket($connectedsocket) TCPCloseSocket($Labview) TCPShutdown() EndFunc I am not getting anything back. I am getting the following output in the console
      +>Setting Hotkeys...--> Press Ctrl+Alt+Break to Restart or Ctrl+BREAK to Stop. Listening to Socket - 544 Connecting to Socket - -1Error -0 Connecting to Socket - -1Error -0 Connecting to Socket - -1Error -0 Connecting to Socket - -1Error -0 Connecting to Socket - -1Error -0 Connecting to Socket - -1Error -0 Connecting to Socket - -1Error -0 Connecting to Socket - -1Error -0 Connecting to Socket - -1Error -0 its going through that loop forever. i need to force stop it.
      But when i open putty and send the same command, i am getting response right away. 
      Can someone please help me with that.
      Thanks in advance
      Regards
      Yogendra
    • By argumentum
      I can TCP/IP in AutoIt, hence, make a HTTP deamon. Now, how can I HTTPS to use SSL !??
      Well, Apache has this "mod_proxy.so" module that can let me have SSL and what not is in Apache.
      All that is needed is to tell Apache what I wanna do by editing httpd.conf .
      # Implements a proxy/gateway for Apache. # 1. Open /Applications/XAMPP/etc/httpd.conf # 2. Enable the following Modules by removing the # at the front of the line. # - LoadModule rewrite_module modules/mod_rewrite.so # - LoadModule proxy_module modules/mod_proxy.so # - LoadModule proxy_http_module modules/mod_proxy_http.so # # 3. Copy and Paste below to the bottom of httpd.conf # <IfModule mod_proxy.c> ProxyRequests On <Proxy *> Order deny,allow Allow from all </Proxy> ProxyVia Off ProxyPreserveHost Off ProxyPass /home/ http://127.0.0.1:84/home/ ProxyPassReverse /home/ http://127.0.0.1:84/home/ SetEnv proxy-nokeepalive 1 # ..since we are not using "keep-alive", we are using "close" </IfModule> ...et voila  
      I'm using XAMPP ( https://www.apachefriends.org/download.html )
      and this is my solution to avoid coding in PHP, as I feel more comfortable coding in AutoIt.
      A "muli-thread or concurrency" can be done by forking the socket ( https://www.autoitscript.com/forum/topic/199177-fork-udf-ish/ )
      but responses are under 20 ms., so I feel fine with a single thread.
      I modified an example ( attached below ), so can try out the concept.
      PS: I am not an Apache guru. I just discovered this and it opens a world of possibilities. In my case, I'm thinking of an API to query SQLite 
      PS2: I'm not gonna make Poll but do click like if you do  
       
      201673-json-http-post-serverlistener.au3
×
×
  • Create New...