Jump to content

[Solved] IPC between system and user

Recommended Posts

Hi, interesting problem.:) TCP/UDP should work. NamedPipes and regular shared memory likely won't work. I would guess Mailslot and injecting Windows message queue won't work either (or only one-way at best), but haven't tested this. I did just run a successful test with my HighMem  UDF (link in sig, Test 3 scripts A and B, any number of processes can share access to the same region(s) ) where one script is ADMIN and the other is USER  (note that Scite output is not visible for admin when starting Scite as user and script has #RequireAdmin), but comms and beeps upon successful I/O still work, as does user output in the other Scite window); haven't ever tested HighMem with SYSTEM account, but it might be worth a shot (I'd be interested to hear how it went); it does require x64-mode (machine/OS/Au3) though.;)

EDIT: So I couldn't resist, and hooked up wraithdu's adaptation of arcker's great services UDF to elevate script HighMem_Test3_A.au3 to  SYSTEM level, and to my delight and surprise, communicating to/from user-level script works fine both ways.:D And because the shared mem is virtual, access is fast, and there's no size restrictions (easily >4GB).

Edited by RTFC
more testing
Link to post
Share on other sites
4 hours ago, RTFC said:

Hi, interesting problem

yeap, and the "Sorry, this test ONLY works if the following three conditions are all met:", none are.

Is all 32bit compile as it should work in as many OS versions as possible. 

In what I'm looking to do is to send commands only, so, one-way only is fine, as I can make each script send the one-way back, by including the originating "address" as part of the message. Even as UDP would be the obviously simple solution, the memShare would be perfect to avoid the firewalls and whatnot, interrupting communications between processes.

I was looking at _WinAPI_CreateFileMapping() and I was thinking of having a go at it, but then again, I have no idea what all those DllStruct are. So I'm not well suited to make an IPC UDF out of it. If you'd like to give it try, I figure, it would only take rearranging the example to be a UDF ( but other than copy/paste, I have no idea of the working of the code ).


Link to post
Share on other sites
23 minutes ago, argumentum said:

Is all 32bit compile as it should work in as many OS versions as possible. 

Luckily for you, I just tested the original HighMem Test3 scripts running as 32-bit and everything still works; of course, you're shared memory region should be significantly less than 2 GB in that case. Just disable "#AutoIt3Wrapper_UseX64=Y" everywhere, and remove the @Autoitx64 macro checks.

23 minutes ago, argumentum said:

Definitely the way to go, IMHO. Under the hood, that's exactly what HighMem uses too.^_^

Edited by RTFC
Link to post
Share on other sites
2 minutes ago, RTFC said:

running as 32-bit and everything still works; of course, you're shared memory region should be significantly less than 2 GB

cool, All I'll ask my other script is to "restart" or, "whatYouUpto", so that count as less than 2GB ;) 
I'll do the mods. and try right now.

Link to post
Share on other sites
7 minutes ago, AdamUL said:


is restricted to 64KB (minus 29-byte header) on localhost per transfer, and to datagrams (424 bytes, minus 12-byte header?) between machines on a LAN (AFAIK). And if you're going to send datagram-sized packets, you might as well use TCP. But both MailSlot and TCP ports can be blocked by AV software/firewalls, so more prone to unexpected failure in (unknown/changing) user environments. And I have no clue how to tell AV to allow MailSlot traffic if it doesn't already by default. So it comes with caveats. Nevertheless, great UDF, and a bedrock-reliable form of IPC if it is not blocked.

Link to post
Share on other sites
25 minutes ago, RTFC said:

Just disable "#AutoIt3Wrapper_UseX64=Y" everywhere, and remove the @Autoitx64 macro checks

Did that and worked :) 
Now, these examples write an INI file, in my case, declaring 1 MB, the files has:


So I can assume that only the PID is needed and the ALLOC 1 & 2 will always be the same. If that is so, all that is needed is the PID of a process, in a one-way-ish ( as in I send "234" and I can get an "ACK" back as to, get acknowledgment of data received and change the data and "flag" ready for next msg.
Along with _Crypt_DecryptData() to make it not accept data from where the CryptKey is obviously wrong. ..... 

If all this is so, would you like to put it together as, a universal ( for the AutoIt universe anyway, tho, it will be interoperational with any other language in Windows ), UDF in the example forum, to have a final and definitive solution to the IPC, that always come up every so often ?

Link to post
Share on other sites

@RTFC, looking at the HighMem UDF examples, I see:

If @error Then
    ConsoleWrite("Error: Allocation failed with error: " & @error & @CRLF)
    Exit(1) ;   on Exit, _HighMem_CleanUp() is automatically invoked

and when AutoIt crashes, it returns 1. Could you change it to "2", to know that it failed the script and is not a crash ?

PS: I just realized that what I posted here is not related to the UDF and ... is just nonsense, sowy :( 

Edited by argumentum
Link to post
Share on other sites
15 hours ago, argumentum said:

Did that and worked


Regarding those  structs I used in that Test3 example, the first one is just a dummy buffer to illustrate the relative offset you need to access the second struct. Note that the absolute address of the allocation in virtual memory is PID-specific, so each process obtains its own mapping of the same virtual region, and the (same) offset of a particular struct will be relative to that PID-specific base within each process. With me still?:blink: I'm also using the PID to generate a process-specifc unique name for the allocated region, so multiple processes can each allocate multiple regions (and multiple structs within those, if necessary), and each process can predict a memory-mapped region's name once it knows what process (PID) created it. The first allocation within each region of course has an offset of zero, so if you only need it for basic comms, maybe that's enough (two processes each create one write buffer, read by the other, so then you don't need to bother with mutex negotiation to ensure you're not both writing to the same allocation at the same time).

15 hours ago, argumentum said:

would you like to put it together as, a universal ( for the AutoIt universe anyway, tho, it will be interoperational with any other language in Windows ), UDF

Errm, not really, sorry, but thanks for asking.:lol: I only cobbled HighMem together for my E4A matrix computing environment, because I need to manipulate large datasets (>4GB, hence x64-mode required). If you think it might be of more general use in an IPC context, just write your own wrapper UDFs and include HighMem.au3 (I think only the test scripts use x64-mode anyhow, the main UDF does not require it, as we just discovered). It shouldn't take more than a rewrite of the Test3 scripts in UDF form. Maybe provide some snazzy user options such as storing strings in a 64K string struct, or some predefined codes for basic comms operations. Have a look at my Pool environment for some examples, if you wish.

It's definitely doable and probably fun (basic functionality is already present), but I simply cannot spare the time for new UDFs these days, drowning in CUDA at the moment...:( But it's great that you find it useful!

Edited by RTFC
Link to post
Share on other sites
7 minutes ago, RTFC said:

With me still?:blink:


7 minutes ago, RTFC said:

Maybe provide som snazzy user options such as storing strings in a 64K string struct, or some predefined codes for basic comms operations

I only wish I had an idea of what you are saying, code wise.

I'll wait for someone to put it together as I'm not the one that can tackle this. But is good to know that is doable. That means that is just a matter of time, until such UDF popup in the forum.

meanwhile I'll do it the UDP way. Thanks for your help, as you've been quite enlightening for me on this subject. :) 

Link to post
Share on other sites
13 hours ago, argumentum said:

I'll do it the UDP way

In that case, be aware of firewall/AV restrictions on UDP port usage that may affect user environments. You may have to actively enable access first; but since your processes run at admin/system level anyhow, they have the required permissions to do so.


13 hours ago, argumentum said:

I only wish I had an idea of what you are saying, code wise

Apologies for being obscure. I hope you're not abandoning the HighMem option because of unfamiliarity with structs.

Structs are easy. You can think of them as a database record containing one or more fields, with each field defined (e.g., by you) to hold a specific type of data (integer, string, double precision value, and so forth). Or easier still, a struct is like a big IKEA wardrobe you design yourself, with drawers of different sizes (different variable types) and each as deep as you want it to be: one may hold a single coin (integer), another a big pile of books (strings). They can be tiny (a few bytes) to humongous in size (GBs). Best of all, you have the option of deciding exactly where you want that wardrobe to be placed in the room (memory), for example in the SW corner, 5 cm from the west wall and 10 cm from the southern one (using the optional second parameter in DllStructCreate, a 32/64 bit memory pointer). You can even define multiple wardrobes to physically overlap, mapping to the same space, but each with a different internal structure, so the same content (data) can be interpreted differently depending on whether you open a book drawer (string) in wardrobe 1, or a numeric one (e.g., integers) in wardrobe 2. So despite being fixed (structures), they're actually very flexible (to work with).:D In the IPC example, several processes are mapping the same-shaped wardrobe(s) as defined in the Test3 .ini file, and then each one in turn opens a drawer, takes something out (DllStructGetData), and sticks something else back in (DllStructSetData). It's no different than accessing an internal $variable, other than interacting with a memory address (DllStructGetPtr) directly rather than indirectly. As long as you know where the wardrobe sits and what (type and size of) drawers it contains (and in what order), you can manipulate their content to your heart's content. The only issue in IPC is that virtual memory is remapped to a different base address by each process, so the room is (nominally) different, but the wardrobe's position relative to its SW corner is the same. That's why those relative offsets were parsed in the example scripts (clumsily through an .ini, because it was just for illustration). DllCalls also rely heavily on them, whenever more than a single piece of data needs to be parsed or returned. Much of AutoIt's functionality, and indeed Windows OS itself, relies on dll calls (to various Windows kernel dlls), and many of those in turn rely on structs to get the job done. Therefore it might be worth your time to get to know them, especially since many programming languages provide access to them (my E4A matrices are structs, and I manipulate them from three very different programming environments). Of course, it's up to you...:mellow:

Edited by RTFC
Link to post
Share on other sites
6 hours ago, RTFC said:

Apologies for being obscure

You were not obscure at all, actually, well presented. It is I, that is in a hurry, and "looking for a rounder circle" without much experience in drawing a line.
@Michel Claveau posted a dll to do the _WinAPI_CreateFileMapping() and @trancexx pointed him to the DLL calls she used in her code.
You yourself use those calls in your code, so what I'll do is to use a mix of what I have in hand, and that is the example from the help file, mainly due to that in the help files example, looks to me ala mailslot, by just declaring, what in that UDF is called "address", that is in fact a name of sorts, and in this is called "The name of the file mapping object to be opened" in the FileMapping example file. So by looking to read that name object, no other info is needed. To make it a unique mapping name, I'll use "ScriptName + PID" and that would be all that is needed to talk to that script, very much so, as only the PID is needed to for the WM_COPYDATA ( if one can find the window ).

That is the approach I'm taking until I get more familiar with WinAPI FileMapping. I've been using mailslots for the longest time and I know what I can get out of it, and it's caveats, but I have no clue of the caveats in FileMapping.

So I've dropped the UDP idea and I'm jumping to this, as its the best approach, for a A/V proof, system/Admin/User proof, IPC. Unless I find caveats that send me back to the drawing board :) 

PS: ..and get to understand DllStructs, that in comparison, is like "why should I learn to drive a car, I already know how to walk there", but I now have to walk an ocean and if I knew how to drive a car, it would be easy to drive a boat. So yes, in due time, due to my "I don't have time to get into that now", I'll take the time to learn and build experience working with DllStructs. And thank you for your push towards it  :) 

Edited by argumentum
Link to post
Share on other sites

ok, 1st draft. Working on making it flawless but has its problems :( 
I get a lot of @extended 183 when on "rapid fire" mouse clicking. 


...work in progress :) 

P.S.: Writing the other way around, Sender as Receiver, as it make more sense ;) 

Edited by argumentum
writing the other way around
Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By SEuBo
      Hi there,
      while I created an example script to generate and execute a function during runtime, I stumbled across a neat way to share data between running autoit scripts.
      This is done using the amazing magic of  AutoItObject_Internal . (You'll need at least Version 3.0.0 of AutoItObject_Internal)
      Using this UDF, you can create a shared data storage, basically an empty "AutoitObject_Internal-"Object which you can then use to write / read data Inline. no set/get methods, just
      #include "AutoItSharedData.au3" $oShare = _AutoIt_SharedData_CreateOrAttach("MyCustomID") $oShare.some_data = 'foo' and you're done. any other script accessing this data will have to do:
      #include "AutoItSharedData.au3" $oShare = _AutoIt_SharedData_CreateOrAttach("MyCustomID") ConsoleWrite($oShare.some_data & @LF)  
      Basically it's Larsj's Implementing IRunningObjectTable Interface, but you dont have a Dictionary, but an IDIspatch Object instead.
      There are already a bunch of IPC options available - and this is another one.
      Example Script 1
      Example Script 2
      To test: run Example Script 1, Then run example Script 2.. or the other way around.
      Example Script 3
      To test: run Example_sharedata3.au3.
       Example SharedData4:
      /Edit: Please note that there's a limitation with the Running object table :
      The Script accessing a variable first, will be the "server" for this variable. This means, access to that variable from other scripts should only be possible, as long the "server" script is running! Use appropriate Object Error handlers in case you don't want the surviving "clients" to crash.
      Feedback and/or improvements appreciated
      version 2.0
      Removed need for AutoItObject, as AutoItObject_Internal now comes with ROT support Added UDF Header Fixed typo on "#include AutoItObjectInternal.au3" -> "#include AutoItObject_Internal.au3" Added ObjGet() after registering the object fails (in case 2 programs tried to register the same ID simultaneously) Updated Examples & zip archive. Cheers,
    • By argumentum
      so in https://www.autoitscript.com/forum/topic/193254-solved-ipc-between-system-and-user/ I asked around about IPCs and got all the answers I was looking for.
      Now the question is: what IPC is most "resilient" on an overwhelmed PC, meaning, the CPU is at 100%, memory is top out and, as is always, need to rely on the IPC.
      ..and all this happened because I open over 100 GUIs at once 😜
      ..but it happens sporadically on low CPU or memory demand anyways.
      ..should I sleep() some time before running another instance ?
      I did not know if to make the question in technical, chat, ..or here. So it's here.   
      Since you will ask what I've tried, I've used the IPC from the Fork UDFish ( WM_COPYDATA that can do Admin/user mix ) and the FMIPC file mapping,  that work under the same conditions.
      So, how do you handle IPC if it fails ?
    • By tatane
      I would like to send an array from a script to a another. This array has 1000 rows and 4 columns with this kind of data :
      1st row  =     528  ;  31  ;   HOSTNAME|1|02:45:47|abcdefgh|username|5   ;   old
      2nd row = ...
      What IPC should I use ?
    • By argumentum
      I was in need of an IPC (Interprocess communication) between system, admin and user levels, and ended up writing this UDF to suit my wantings.
      Hope you find it useful too.
      Works from WinXP/Server2003 to the now current Win10/Server2016.
      It communicates between any mix of x32, x64, Admin, User.

      In the zip file, there is the UDF and an example: FMIPC(v0.2018.04.04).zip
      Special thanks to @RTFC for the help in the support forum   
    • By JohnWIlling
      I am in the need for a simple synchronous Client/Server communication.  I found several examples and references to various kinds of Inter-Process Communications such as TCP, Named Pipes, Mail Slots, Shared Memory, Memory Mapped Files, and simple Files.  I wanted to see what the best solutions would be.  I began developing a library and slowly began adding each of the IPC methods and ended up with a library with a very simple synchronous “ASCII” API where the application can choose which method to use at startup.
      For the Server side, a Server app must initialize communication by calling:
      Func InitConnection($cType = $cDefaultType, $ResourceName = "", $bBlock = $cDefaultBlocking, $fSleepFunc = "", $iBufSize = $DEFAULT_BUFSIZE)
      The optional arguments allow the app to specify the connection type (such as: $cNamedPipe, $cFile, $cTCP, $cSharedMem, $cMailSlot), a value for the resource name (such as the file, named pipe name, TCP port number, etc.), the communication buffer size, and a callback function for when the “read” is waiting for data.
      A “File Descriptor” is returned and must be used in the future API calls.
      The Server side must then call:
      Func StartConnection($iFD)
      This call waits for a Client to connect.  The Server then calls:
                      Func ReadData($iFD, ByRef $sData)
      To read a Request from the Client and then calls:
                      Func WriteData($iFD, ByRef $sData)
      To send the reply back to the Client.
      When communication with the Client is done, the Server app will call:
      Func StopConnection($iFD)
      When the Server app is done with the communications it will call:
      Func EndConnection($iFD)
      For the Client side, a Client app must open the communication by calling:
      Func OpenConnection($cType = $cDefaultType, $ResourceName = "", $bBlock = $cDefaultBlocking, $fSleepFunc = "", $iBufSize = $DEFAULT_BUFSIZE)
      The optional arguments allow the app to specify the connection type (such as: $cNamedPipe, $cFile, $cTCP, $cSharedMem, $cMailSlot), a value for the resource name (such as the file, named pipe name, TCP port number, etc.), the communication buffer size, and a callback function for when the “read” is waiting for data.
      A “File Descriptor” is returned and must be used in the future API calls.
      The Client side then send a request to the Server app by calling:
                      Func WriteData($iFD, ByRef $sData)
      To read a Response from the Server by calling:
                      Func ReadData($iFD, ByRef $sData)
      To end the connection to the Server by calling:
      Func CloseConnection($iFD)
      Within the IPC_IO.AU3 library, each IPC method is ether:
      ·         “stream” based where data is read/written by calling _WinAPI_ReadFile/TCPRecv or _WinAPI_WriteFile/ TCPSend
      ·         “direct” based for Shared memory where the Client reads the data directly from the Server App’s memory and the Server directly reads the Client App’s memory
      In processing a request, the “ReadData” process starts by checking if data is ready to be read by calling the routine: “ReadStart”, then it reads in the size of the request by calling “ReadSize”, it then reads in the Ascii Request by calling “ReadBuffer”, then the sequence is completed by calling “ReadEnd”. 
      The Write Process follows the same sequence with “WriteData” calling “WriteStart”, “WriteSize”, “WriteBuffer”, “WriteEnd”.
      My testing showed that the performance of sending and receiving of a 10k file took:
      ·         "Shared Memory" was the fastest, at 0.007468 Sec
      ·         “Named Pipes” at 0.015954
      ·         “Mail Slots” at 0.016427
      ·         “File Based” at 0.270287
      ·         “TCP” at 0.994884
  • Create New...