Jump to content
argumentum

[Solved] IPC between system and user

Recommended Posts

RTFC

Hi, interesting problem.:) TCP/UDP should work. NamedPipes and regular shared memory likely won't work. I would guess Mailslot and injecting Windows message queue won't work either (or only one-way at best), but haven't tested this. I did just run a successful test with my HighMem  UDF (link in sig, Test 3 scripts A and B, any number of processes can share access to the same region(s) ) where one script is ADMIN and the other is USER  (note that Scite output is not visible for admin when starting Scite as user and script has #RequireAdmin), but comms and beeps upon successful I/O still work, as does user output in the other Scite window); haven't ever tested HighMem with SYSTEM account, but it might be worth a shot (I'd be interested to hear how it went); it does require x64-mode (machine/OS/Au3) though.;)

EDIT: So I couldn't resist, and hooked up wraithdu's adaptation of arcker's great services UDF to elevate script HighMem_Test3_A.au3 to  SYSTEM level, and to my delight and surprise, communicating to/from user-level script works fine both ways.:D And because the shared mem is virtual, access is fast, and there's no size restrictions (easily >4GB).

Edited by RTFC
more testing
  • Thanks 1

Share this post


Link to post
Share on other sites
Bilgus

Depending on how much data you are transfering you could use the eventlog functions..

I only mention it because you can get notifications with _EventLog__Notify ( $hEventLog, $hEvent )

Edited by Bilgus
  • Like 1

Share this post


Link to post
Share on other sites
argumentum
4 hours ago, RTFC said:

Hi, interesting problem

yeap, and the "Sorry, this test ONLY works if the following three conditions are all met:", none are.

Is all 32bit compile as it should work in as many OS versions as possible. 

In what I'm looking to do is to send commands only, so, one-way only is fine, as I can make each script send the one-way back, by including the originating "address" as part of the message. Even as UDP would be the obviously simple solution, the memShare would be perfect to avoid the firewalls and whatnot, interrupting communications between processes.

I was looking at _WinAPI_CreateFileMapping() and I was thinking of having a go at it, but then again, I have no idea what all those DllStruct are. So I'm not well suited to make an IPC UDF out of it. If you'd like to give it try, I figure, it would only take rearranging the example to be a UDF ( but other than copy/paste, I have no idea of the working of the code ).

Thanks

Share this post


Link to post
Share on other sites
RTFC
23 minutes ago, argumentum said:

Is all 32bit compile as it should work in as many OS versions as possible. 

Luckily for you, I just tested the original HighMem Test3 scripts running as 32-bit and everything still works; of course, you're shared memory region should be significantly less than 2 GB in that case. Just disable "#AutoIt3Wrapper_UseX64=Y" everywhere, and remove the @Autoitx64 macro checks.

23 minutes ago, argumentum said:

Definitely the way to go, IMHO. Under the hood, that's exactly what HighMem uses too.^_^

Edited by RTFC

Share this post


Link to post
Share on other sites
AdamUL

Have you looked at the MailSlot UDF?  It may work for what you need.  

 

Adam

 

Share this post


Link to post
Share on other sites
argumentum
1 minute ago, AdamUL said:

Have you looked at the MailSlot UDF?  It may work for what you need. 

it does not :( 

Share this post


Link to post
Share on other sites
argumentum
2 minutes ago, RTFC said:

running as 32-bit and everything still works; of course, you're shared memory region should be significantly less than 2 GB

cool, All I'll ask my other script is to "restart" or, "whatYouUpto", so that count as less than 2GB ;) 
I'll do the mods. and try right now.

Share this post


Link to post
Share on other sites
RTFC
7 minutes ago, AdamUL said:

MailSlot

is restricted to 64KB (minus 29-byte header) on localhost per transfer, and to datagrams (424 bytes, minus 12-byte header?) between machines on a LAN (AFAIK). And if you're going to send datagram-sized packets, you might as well use TCP. But both MailSlot and TCP ports can be blocked by AV software/firewalls, so more prone to unexpected failure in (unknown/changing) user environments. And I have no clue how to tell AV to allow MailSlot traffic if it doesn't already by default. So it comes with caveats. Nevertheless, great UDF, and a bedrock-reliable form of IPC if it is not blocked.

Share this post


Link to post
Share on other sites
argumentum
25 minutes ago, RTFC said:

Just disable "#AutoIt3Wrapper_UseX64=Y" everywhere, and remove the @Autoitx64 macro checks

Did that and worked :) 
Now, these examples write an INI file, in my case, declaring 1 MB, the files has:

[PID]
PID=16120
[ALLOC1]
sizeInBytes=512
relativeOffset=0x00000000
[ALLOC2]
sizeInBytes=1024
relativeOffset=0x00010000

So I can assume that only the PID is needed and the ALLOC 1 & 2 will always be the same. If that is so, all that is needed is the PID of a process, in a one-way-ish ( as in I send "234" and I can get an "ACK" back as to, get acknowledgment of data received and change the data and "flag" ready for next msg.
Along with _Crypt_DecryptData() to make it not accept data from where the CryptKey is obviously wrong. ..... 

If all this is so, would you like to put it together as, a universal ( for the AutoIt universe anyway, tho, it will be interoperational with any other language in Windows ), UDF in the example forum, to have a final and definitive solution to the IPC, that always come up every so often ?

Share this post


Link to post
Share on other sites
argumentum

@RTFC, looking at the HighMem UDF examples, I see:

If @error Then
    ConsoleWrite("Error: Allocation failed with error: " & @error & @CRLF)
    Exit(1) ;   on Exit, _HighMem_CleanUp() is automatically invoked
EndIf

and when AutoIt crashes, it returns 1. Could you change it to "2", to know that it failed the script and is not a crash ?

PS: I just realized that what I posted here is not related to the UDF and ... is just nonsense, sowy :( 

Edited by argumentum
oops

Share this post


Link to post
Share on other sites
RTFC
15 hours ago, argumentum said:

Did that and worked

:)

Regarding those  structs I used in that Test3 example, the first one is just a dummy buffer to illustrate the relative offset you need to access the second struct. Note that the absolute address of the allocation in virtual memory is PID-specific, so each process obtains its own mapping of the same virtual region, and the (same) offset of a particular struct will be relative to that PID-specific base within each process. With me still?:blink: I'm also using the PID to generate a process-specifc unique name for the allocated region, so multiple processes can each allocate multiple regions (and multiple structs within those, if necessary), and each process can predict a memory-mapped region's name once it knows what process (PID) created it. The first allocation within each region of course has an offset of zero, so if you only need it for basic comms, maybe that's enough (two processes each create one write buffer, read by the other, so then you don't need to bother with mutex negotiation to ensure you're not both writing to the same allocation at the same time).

15 hours ago, argumentum said:

would you like to put it together as, a universal ( for the AutoIt universe anyway, tho, it will be interoperational with any other language in Windows ), UDF

Errm, not really, sorry, but thanks for asking.:lol: I only cobbled HighMem together for my E4A matrix computing environment, because I need to manipulate large datasets (>4GB, hence x64-mode required). If you think it might be of more general use in an IPC context, just write your own wrapper UDFs and include HighMem.au3 (I think only the test scripts use x64-mode anyhow, the main UDF does not require it, as we just discovered). It shouldn't take more than a rewrite of the Test3 scripts in UDF form. Maybe provide some snazzy user options such as storing strings in a 64K string struct, or some predefined codes for basic comms operations. Have a look at my Pool environment for some examples, if you wish.

It's definitely doable and probably fun (basic functionality is already present), but I simply cannot spare the time for new UDFs these days, drowning in CUDA at the moment...:( But it's great that you find it useful!

Edited by RTFC
typos
  • Like 1

Share this post


Link to post
Share on other sites
argumentum
7 minutes ago, RTFC said:

With me still?:blink:

 

7 minutes ago, RTFC said:

Maybe provide som snazzy user options such as storing strings in a 64K string struct, or some predefined codes for basic comms operations

I only wish I had an idea of what you are saying, code wise.

I'll wait for someone to put it together as I'm not the one that can tackle this. But is good to know that is doable. That means that is just a matter of time, until such UDF popup in the forum.

meanwhile I'll do it the UDP way. Thanks for your help, as you've been quite enlightening for me on this subject. :) 

Share this post


Link to post
Share on other sites
RTFC
13 hours ago, argumentum said:

I'll do it the UDP way

In that case, be aware of firewall/AV restrictions on UDP port usage that may affect user environments. You may have to actively enable access first; but since your processes run at admin/system level anyhow, they have the required permissions to do so.

 

13 hours ago, argumentum said:

I only wish I had an idea of what you are saying, code wise

Apologies for being obscure. I hope you're not abandoning the HighMem option because of unfamiliarity with structs.

Structs are easy. You can think of them as a database record containing one or more fields, with each field defined (e.g., by you) to hold a specific type of data (integer, string, double precision value, and so forth). Or easier still, a struct is like a big IKEA wardrobe you design yourself, with drawers of different sizes (different variable types) and each as deep as you want it to be: one may hold a single coin (integer), another a big pile of books (strings). They can be tiny (a few bytes) to humongous in size (GBs). Best of all, you have the option of deciding exactly where you want that wardrobe to be placed in the room (memory), for example in the SW corner, 5 cm from the west wall and 10 cm from the southern one (using the optional second parameter in DllStructCreate, a 32/64 bit memory pointer). You can even define multiple wardrobes to physically overlap, mapping to the same space, but each with a different internal structure, so the same content (data) can be interpreted differently depending on whether you open a book drawer (string) in wardrobe 1, or a numeric one (e.g., integers) in wardrobe 2. So despite being fixed (structures), they're actually very flexible (to work with).:D In the IPC example, several processes are mapping the same-shaped wardrobe(s) as defined in the Test3 .ini file, and then each one in turn opens a drawer, takes something out (DllStructGetData), and sticks something else back in (DllStructSetData). It's no different than accessing an internal $variable, other than interacting with a memory address (DllStructGetPtr) directly rather than indirectly. As long as you know where the wardrobe sits and what (type and size of) drawers it contains (and in what order), you can manipulate their content to your heart's content. The only issue in IPC is that virtual memory is remapped to a different base address by each process, so the room is (nominally) different, but the wardrobe's position relative to its SW corner is the same. That's why those relative offsets were parsed in the example scripts (clumsily through an .ini, because it was just for illustration). DllCalls also rely heavily on them, whenever more than a single piece of data needs to be parsed or returned. Much of AutoIt's functionality, and indeed Windows OS itself, relies on dll calls (to various Windows kernel dlls), and many of those in turn rely on structs to get the job done. Therefore it might be worth your time to get to know them, especially since many programming languages provide access to them (my E4A matrices are structs, and I manipulate them from three very different programming environments). Of course, it's up to you...:mellow:

Edited by RTFC
  • Like 1

Share this post


Link to post
Share on other sites
argumentum
6 hours ago, RTFC said:

Apologies for being obscure

You were not obscure at all, actually, well presented. It is I, that is in a hurry, and "looking for a rounder circle" without much experience in drawing a line.
@Michel Claveau posted a dll to do the _WinAPI_CreateFileMapping() and @trancexx pointed him to the DLL calls she used in her code.
You yourself use those calls in your code, so what I'll do is to use a mix of what I have in hand, and that is the example from the help file, mainly due to that in the help files example, looks to me ala mailslot, by just declaring, what in that UDF is called "address", that is in fact a name of sorts, and in this is called "The name of the file mapping object to be opened" in the FileMapping example file. So by looking to read that name object, no other info is needed. To make it a unique mapping name, I'll use "ScriptName + PID" and that would be all that is needed to talk to that script, very much so, as only the PID is needed to for the WM_COPYDATA ( if one can find the window ).

That is the approach I'm taking until I get more familiar with WinAPI FileMapping. I've been using mailslots for the longest time and I know what I can get out of it, and it's caveats, but I have no clue of the caveats in FileMapping.

So I've dropped the UDP idea and I'm jumping to this, as its the best approach, for a A/V proof, system/Admin/User proof, IPC. Unless I find caveats that send me back to the drawing board :) 

PS: ..and get to understand DllStructs, that in comparison, is like "why should I learn to drive a car, I already know how to walk there", but I now have to walk an ocean and if I knew how to drive a car, it would be easy to drive a boat. So yes, in due time, due to my "I don't have time to get into that now", I'll take the time to learn and build experience working with DllStructs. And thank you for your push towards it  :) 

Edited by argumentum
expand

Share this post


Link to post
Share on other sites
argumentum

ok, 1st draft. Working on making it flawless but has its problems :( 
I get a lot of @extended 183 when on "rapid fire" mouse clicking. 

<snip>

...work in progress :) 

P.S.: Writing the other way around, Sender as Receiver, as it make more sense ;) 

Edited by argumentum
writing the other way around

Share this post


Link to post
Share on other sites
argumentum

The above code is, unless any of you find a flaw, perfect.   B)

Now my question is:  can it be a GUIRegisterMsg($something , 'FileMappingRecvRead')  implemented ?

That'd be beautiful     :D 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • argumentum
      By argumentum
      I was in need of an IPC (Interprocess communication) between system, admin and user levels, and ended up writing this UDF to suit my wantings.
      Hope you find it useful too.
      Works from WinXP/Server2003 to the now current Win10/Server2016.
      It communicates between any mix of x32, x64, Admin, User.

      In the zip file, there is the UDF and an example: FMIPC(v0.2018.04.04).zip
      Special thanks to @RTFC for the help in the support forum   
    • JohnWIlling
      By JohnWIlling
      IPC_IO.AU3
      I am in the need for a simple synchronous Client/Server communication.  I found several examples and references to various kinds of Inter-Process Communications such as TCP, Named Pipes, Mail Slots, Shared Memory, Memory Mapped Files, and simple Files.  I wanted to see what the best solutions would be.  I began developing a library and slowly began adding each of the IPC methods and ended up with a library with a very simple synchronous “ASCII” API where the application can choose which method to use at startup.
      For the Server side, a Server app must initialize communication by calling:
      Func InitConnection($cType = $cDefaultType, $ResourceName = "", $bBlock = $cDefaultBlocking, $fSleepFunc = "", $iBufSize = $DEFAULT_BUFSIZE)
      The optional arguments allow the app to specify the connection type (such as: $cNamedPipe, $cFile, $cTCP, $cSharedMem, $cMailSlot), a value for the resource name (such as the file, named pipe name, TCP port number, etc.), the communication buffer size, and a callback function for when the “read” is waiting for data.
      A “File Descriptor” is returned and must be used in the future API calls.
      The Server side must then call:
      Func StartConnection($iFD)
      This call waits for a Client to connect.  The Server then calls:
                      Func ReadData($iFD, ByRef $sData)
      To read a Request from the Client and then calls:
                      Func WriteData($iFD, ByRef $sData)
      To send the reply back to the Client.
      When communication with the Client is done, the Server app will call:
      Func StopConnection($iFD)
      When the Server app is done with the communications it will call:
      Func EndConnection($iFD)
       
      For the Client side, a Client app must open the communication by calling:
      Func OpenConnection($cType = $cDefaultType, $ResourceName = "", $bBlock = $cDefaultBlocking, $fSleepFunc = "", $iBufSize = $DEFAULT_BUFSIZE)
      The optional arguments allow the app to specify the connection type (such as: $cNamedPipe, $cFile, $cTCP, $cSharedMem, $cMailSlot), a value for the resource name (such as the file, named pipe name, TCP port number, etc.), the communication buffer size, and a callback function for when the “read” is waiting for data.
      A “File Descriptor” is returned and must be used in the future API calls.
      The Client side then send a request to the Server app by calling:
                      Func WriteData($iFD, ByRef $sData)
      To read a Response from the Server by calling:
                      Func ReadData($iFD, ByRef $sData)
      To end the connection to the Server by calling:
      Func CloseConnection($iFD)
       
      Within the IPC_IO.AU3 library, each IPC method is ether:
      ·         “stream” based where data is read/written by calling _WinAPI_ReadFile/TCPRecv or _WinAPI_WriteFile/ TCPSend
      ·         “direct” based for Shared memory where the Client reads the data directly from the Server App’s memory and the Server directly reads the Client App’s memory
      In processing a request, the “ReadData” process starts by checking if data is ready to be read by calling the routine: “ReadStart”, then it reads in the size of the request by calling “ReadSize”, it then reads in the Ascii Request by calling “ReadBuffer”, then the sequence is completed by calling “ReadEnd”. 
      The Write Process follows the same sequence with “WriteData” calling “WriteStart”, “WriteSize”, “WriteBuffer”, “WriteEnd”.
       
      Results
      My testing showed that the performance of sending and receiving of a 10k file took:
      ·         "Shared Memory" was the fastest, at 0.007468 Sec
      ·         “Named Pipes” at 0.015954
      ·         “Mail Slots” at 0.016427
      ·         “File Based” at 0.270287
      ·         “TCP” at 0.994884
       
      IPC_IO.au3
      Client.au3
      Server.au3
    • dubi
      By dubi
      Hi,


       
      Background: i have a number of instances of the same application that I want to automate in parallel.

      Unfortunately, I cannot completely automate these instances in the background. So, from time to time these instances need to have the focus so that I can interact with the controls via “send” directly.

      Each of the application instances is controlled by a au3 complied script. Each script (called with a parameter) manages the automation of the respective application-instance. Each of the (complied) script (instances) is called by a (central) front end with a gui. The front end controls if the “focus” is available to do the “send” and “mouseclick” modifications. The central front end either allows a child to have the focus or prevents it to get the focus (in which case the child will wait and checks again). The code for the front end is included. Apologies for the lengthy explanation.


      #RequireAdmin #include <ButtonConstants.au3> #include <GUIConstantsEx.au3> #include <WindowsConstants.au3> #include <EditConstants.au3> #include <GUIEdit.au3> #include <ScrollBarConstants.au3> #include <Array.au3> Global Const $APF_ALLOWMULTILINE = 1 Global Const $APF_RIGHTALIGN = 6 Global Const $APF_RIGHTALIGNCOL = 2 Global Const $APF_RIGHTALIGNDATA = 4 Global Const $APF_PRINTROWNUM = 8 Global Const $APF_DEFAULT = $APF_PRINTROWNUM Global $PID[9], $FocusAvailable = True, $previousEditMsg Global $PID_waiting[0][2], $logfile $logfile = "D:\MultiInstance\Logfiles\FrontEnd.txt" If FileExists($logfile) Then FileDelete($logfile) #Region ### START Koda GUI section ### Form= $hGui_1 = GUICreate("Instanzenmanager", 493, 1226, 1807, 93) $grpInst1 = GUICtrlCreateGroup(" 1 ", 8, 8, 233, 121) $btnPause01 = GUICtrlCreateCheckbox("Pause", 16, 64, 50, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $tbnStop01 = GUICtrlCreateCheckbox("Stop", 16, 96, 218, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnMin01 = GUICtrlCreateCheckbox("Minimize", 72, 64, 66, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnWdh01 = GUICtrlCreateCheckbox("Restore", 144, 64, 90, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnStart01 = GUICtrlCreateCheckbox("Start", 16, 32, 218, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) GUICtrlSetBkColor(-1, 0x00FF00) GUICtrlCreateGroup("", -99, -99, 1, 1) $grpInst3 = GUICtrlCreateGroup(" 3 ", 8, 136, 233, 121) $btnPause03 = GUICtrlCreateCheckbox("Pause", 16, 192, 50, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $tbnStop03 = GUICtrlCreateCheckbox("Stop", 16, 224, 218, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnMin03 = GUICtrlCreateCheckbox("Minimize", 72, 192, 66, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnWdh03 = GUICtrlCreateCheckbox("Restore", 144, 192, 90, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnStart03 = GUICtrlCreateCheckbox("Start", 16, 160, 218, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) GUICtrlSetBkColor(-1, 0x00FF00) GUICtrlCreateGroup("", -99, -99, 1, 1) $grpInst2 = GUICtrlCreateGroup(" 2 ", 248, 8, 233, 121) $btnPause02 = GUICtrlCreateCheckbox("Pause", 256, 64, 50, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $tbnStop02 = GUICtrlCreateCheckbox("Stop", 256, 96, 218, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnMin02 = GUICtrlCreateCheckbox("Minimize", 312, 64, 66, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnWdh02 = GUICtrlCreateCheckbox("Restore", 384, 64, 90, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnStart02 = GUICtrlCreateCheckbox("Start", 256, 32, 218, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) GUICtrlSetBkColor(-1, 0x00FF00) GUICtrlCreateGroup("", -99, -99, 1, 1) $grpInst4 = GUICtrlCreateGroup(" 4 ", 248, 136, 233, 121) $btnPause04 = GUICtrlCreateCheckbox("Pause", 256, 192, 50, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $tbnStop04 = GUICtrlCreateCheckbox("Stop", 256, 224, 218, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnMin04 = GUICtrlCreateCheckbox("Minimize", 312, 192, 66, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnWdh04 = GUICtrlCreateCheckbox("Restore", 384, 192, 90, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnStart04 = GUICtrlCreateCheckbox("Start", 256, 160, 218, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) GUICtrlSetBkColor(-1, 0x00FF00) $Edit1 = GUICtrlCreateEdit("", 8, 720, 473, 497) $btnPauseAll = GUICtrlCreateCheckbox("Pause all", 8, 656, 474, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnStopAll = GUICtrlCreateCheckbox("Stop all", 7, 688, 474, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) GUISetState(@SW_SHOW) #EndRegion ### END Koda GUI section ### While 1 CheckGuiMsg() FileWrite($logfile, @HOUR & ":" & @MIN & ":" & @SEC & "> " & "CheckGuiMsg" & @CRLF) CheckClientMessages() FileWrite($logfile, @HOUR & ":" & @MIN & ":" & @SEC & "> " & "CheckClientMessages" & @CRLF) WEnd Func CheckGuiMsg() $nMsg = GUIGetMsg() Switch $nMsg Case $GUI_EVENT_CLOSE Exit Case $btnStart01 AddTextToEdit("Starting Instance 1") $PID[0] = Run("D:\XVM05\Entwicklung_FilterTest_Multi_ToFileV001.exe 1", @ScriptDir, Default, 3) Case $btnStart02 AddTextToEdit("Starting Instance 2") $PID[1] = Run("D:\XVM05\Entwicklung_FilterTest_Multi_ToFileV001.exe 2", @ScriptDir, Default, 3) Case $btnStart03 AddTextToEdit("Starting Instance 3") $PID[2] = Run("D:\XVM05\Entwicklung_FilterTest_Multi_ToFileV001.exe 3", @ScriptDir, Default, 3) Case $btnStart04 AddTextToEdit("Starting Instance 4") $PID[3] = Run("D:\XVM05\Entwicklung_FilterTest_Multi_ToFileV001.exe 4", @ScriptDir, Default, 3) Case $btnPause01 AddTextToEdit("Send Pause to Instance 1") StdinWrite($PID[0], "Pause") Case $btnPause02 StdinWrite($PID[1], "Pause") Case $btnPause03 StdinWrite($PID[2], "Pause") Case $btnPause04 StdinWrite($PID[3], "Pause") Case $tbnStop01 StdinWrite($PID[0], "Stop") Case $tbnStop02 StdinWrite($PID[1], "Stop") Case $tbnStop03 StdinWrite($PID[2], "Stop") Case $tbnStop04 StdinWrite($PID[3], "Stop") Case $btnPauseAll AddTextToEdit(@CRLF & "************Pause All not yet implemented**************" & @CRLF) Case $btnStopAll AddTextToEdit(@CRLF & "************Stop All not yet implemented***************" & @CRLF) EndSwitch EndFunc ;==>CheckGuiMsg Func CheckClientMessages() For $i = 0 To 3 FileWrite($logfile, @HOUR & ":" & @MIN & ":" & @SEC & "> " & $i & @CRLF) Local $a = TimerInit() $p = $PID[$i] $streamRead = StdoutRead($p) If $streamRead <> "" Then Switch $streamRead Case StringInStr($streamRead, "Focus Needed") > 0 If $FocusAvailable Then $FocusAvailable = False StdinWrite($p, "Focus Granted") Else EndIf Case StringInStr($streamRead, "Release Focus") > 0 StdinWrite($p, "Release Focus Received") $FocusAvailable = True Case Else EndSwitch EndIf FileWrite($logfile, @HOUR & ":" & @MIN & ":" & @SEC & "> " & $i & " " & round(TimerDiff($a),2) & @CRLF) Next EndFunc ;==>CheckClientMessages Func AddTextToEdit($text) If $previousEditMsg <> $text Then $previousEditMsg = $text GUICtrlSetData($Edit1, GUICtrlRead($Edit1) & @YEAR & "." & @MON & "." & @MDAY & " - " & @HOUR & ":" & @MIN & ":" & @SEC & "> " & $text & @CRLF) _GUICtrlEdit_Scroll($Edit1, $SB_SCROLLCARET) EndIf EndFunc ;==>AddTextToEdit



      My issue now is that the mechanism with with StdoutRead and StdinWrite is not efficient at all. The more instances I start the slower it gets. This is not just a bit slower (like a fraction of a second), but to the degree that the front end is not responding at all any longer (with 3 instances handling).

      So my questions are:

      1.       Is there a flaw in my implementation with StdoutRead and StdinWrite? (note that all works fine with 1 and also (slower) with 2 instances running) but actually breaks down with 3 instances running.

      2.       Can I optimize the currently used implementation so that I can control 30+ instances?

      3.       What other implementation do you see suitable for this approach?

      a.       I have already tried it with communication through files but observed that this is not sufficiently reliable with multiple instances.

      b.       Is Named Pipes a more performant approach (I am a bit scared of the effort to learn and implement this)

      c.       Any other method?


       
      Many thanks in advance

      -dubi





    • Grosminet
      By Grosminet
      Hi,
      I'm blocked on a strange issue concerning the use of '_WinAPI_ReadProcessMemory' to retrieve one 'String' between 2 cooperating applications based on the IPC method using a private 'Windows Message' handler (thanks to '_WinAPI_RegisterWindowMessage').
      Let's me explain what happens:
      1) - From a small GUI 'ipc-sender' application, the user can type any string (like 'abcde') and click a 'Send Data' button to exchange this info with another small 'ipc-receiver' application. the coding is done in such way ( '_DumpStruct()' method) that a trace of the data sent is dumped in an edit viewer inside the GUI: see the 'ipc-sender' script source below -->
      #Region ;**** Directives created by AutoIt3Wrapper_GUI **** #AutoIt3Wrapper_Outfile=ipc_sender.exe #EndRegion ;**** Directives created by AutoIt3Wrapper_GUI **** #include <GUIConstantsEx.au3> #include <SendMessage.au3> #include <MsgBoxConstants.au3> #include <WinAPI.au3> #include <WinAPISys.au3> #include <ProcessConstants.au3> #include <FontConstants.au3> #include <GuiEdit.au3> #include <ScrollBarsConstants.au3> #include <Array.au3> ; Author : Grosminet Global Const $WM_IPC_PRIVATE_Grosminet = _WinAPI_RegisterWindowMessage('ipc_sender_to_receiver') Global Const $sAPP_me = "ipc_sender" Global Const $sAPP_other = "ipc_receiver" Global $guiw = 1000, $guih = 300, $guix = (@desktopwidth - $guiw - 50), $guiy = $guih + 150, $sp = 10, $x = $sp, $y = $sp, $w, $hbut = 28, $h Global $hParentGUI, $hSendBut, $hlocalPID, $hSendEdit, $hRecEdit Global $debug = true, $info, $PIDAppMe, $hOtherProcess ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo $hParentGui = GUICreate($sAPP_me, $guiw, $guih, $guix, $guiy) $w = ($guiw - 3*$sp) / 4 $h = ($guih - 3* $sp) / 2 $hSendBut = GUICtrlCreateButton("Send data", $x, $y, $w, $hbut) $y += $hbut + $sp $hlocalPID = GUIctrlCreateLabel("PID=", $x, $y, $w, $h) $x += $w + $sp $y = $sp $hSendEdit = GUIctrlCreateEdit("abcde", $x, $y, 3* $w, $h) $x = $sp $y += $h + $sp $hRecEdit = GUIctrlCreateEdit("", $x, $y, 4* $w, $h) GUICtrlSetFont(-1, 9, $FW_NORMAL, Default, "Courier New") ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo GUISetState(@SW_SHOW, $hParentGui) GUICtrlSetData($hlocalPID, "PID= " & @AutoItPID) ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo ; Get the RECEIVER application 'process handle' ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo Opt("WinTitleMatchMode", 1) $hOtherProcess = WinGetHandle($sAPP_other) if @error then MsgBox($MB_SYSTEMMODAL, "ERROR", "Unable to retrieve handle of " & $sAPP_other & ", error= " & @error) exit endif $info = " Receiver application --> " & $sAPP_other & " - Handle= " & $hOtherProcess & @crlf _ShowInfo($info) ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo While 1 Switch GUIGetMsg() Case $GUI_EVENT_CLOSE _myExit() Case $hSendBut _SendDATA_to_X() EndSwitch WEnd ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo Func _SendDATA_to_X() Local $sValue_To_Send = GUICtrlRead($hSendEdit) Local $iStringSize = StringLen($sValue_To_Send) + 1 local $TagInfoStruct = "struct;wchar buf[" & $iStringSize & "];endstruct" Local $tValue_To_Send = DllStructCreate($TagInfoStruct) DllStructSetData($tValue_To_Send, "buf", $sValue_To_Send) ; Local $pValue_To_Send = DllStructGetPtr($tValue_To_Send) Local $iSizeStruct = DllStructGetSize($tValue_To_Send) $info = '_SendDATA_to_X(): Pointer to text= ' & $pValue_To_Send & " - Size of text= " & $iStringSize & " - Size of structure= " & $iSizeStruct & @CRLF _ShowInfo($info) $info = _DumpStruct($pValue_To_Send, $iSizeStruct) _ShowInfo($info) ; local $ret = _WinAPI_PostMessage($hOtherProcess, $WM_IPC_PRIVATE_Grosminet, $pValue_To_Send, $iSizeStruct) If not $ret Then MsgBox($MB_SYSTEMMODAL, "ERROR", "_SendDATA_to_X(): " & $sAPP_me & " --> _WinAPI_PostMessage error= " & _WinAPI_GetLastError()) else Local $sData_Sent = StringLeft(DllStructGetData($tValue_To_Send, "buf"), $iStringSize) $info = '................: --> Data sent = ' & $sData_Sent & @CRLF _ShowInfo($info) endif $pValue_To_Send = 0 $tValue_To_Send = 0 EndFunc ;==>_SendDATA_to_X ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo Func _ShowInfo($msg) if $debug then ConsoleWrite($msg) GUICtrlSetData($hRecEdit, GUICtrlRead($hRecEdit) & $msg) Local $iEnd = StringLen(GUICtrlRead($hRecEdit)) _GUICtrlEdit_SetSel($hRecEdit, $iEnd, $iEnd) _GUICtrlEdit_Scroll($hRecEdit, $SB_SCROLLCARET) Endfunc ; _ShowInfo ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo Func _myExit() GUIDelete() exit Endfunc ; _myExit ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo Func _DumpStruct($p_STRUCT, $iSizeStruct) ; $iSizeStruct = the size of the struct in bytes (DllStructGetSize) Local $TagStructDump = "struct;align;byte[" & $iSizeStruct & "];endstruct" Local $t_Struct = DllStructCreate($TagStructDump, $p_STRUCT) Local $i Local $structInfo = "" _ConsoleWriteInfo($structInfo, "Structure size: " & $iSizeStruct & " byte(s):" & @crlf) for $i = 0 to $iSizeStruct - 1 _ConsoleWriteInfo($structInfo, hex(DllStructGetData($t_Struct, 1, $i), 2) & " ") if (Mod($i+1, 8) = 0) then _ConsoleWriteInfo($structInfo, @CRLF) Endif Next _ConsoleWriteInfo($structInfo, @CRLF) return $structInfo EndFunc ; _DumpStruct ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo Func _ConsoleWriteInfo(ByRef $msg, $txt) $msg &= $txt EndFunc ; _ConsoleWriteInfo 2) - From a small GUI 'ipc-receiver' application, the user can check the values of data received thanks to the same '_DumpStruct()' method: --> see the 'ipc-receiver' script :
      #Region ;**** Directives created by AutoIt3Wrapper_GUI **** #AutoIt3Wrapper_Outfile=ipc_receiver.exe #EndRegion ;**** Directives created by AutoIt3Wrapper_GUI **** #include <Array.au3> #include <GUIConstantsEx.au3> #include <SendMessage.au3> #include <MsgBoxConstants.au3> #include <WinAPI.au3> #include <WinAPISys.au3> #include <ProcessConstants.au3> #include <FontConstants.au3> #include <GuiEdit.au3> #include <ScrollBarsConstants.au3> #include <WinAPIDiag.au3> ; Author : Grosminet Global Const $WM_IPC_PRIVATE_Grosminet = _WinAPI_RegisterWindowMessage('ipc_sender_to_receiver') Global Const $sAPP_me = "ipc_receiver" Global Const $sAPP_other = "ipc_sender" Global Const $sSenderEXE = @scriptdir & "\" & $sAPP_other & ".exe" Global $guiw = 1000, $guih = 300, $guix = (@desktopwidth - $guiw - 50), $guiy = 100, $sp = 10, $x = $sp, $y = $sp, $w, $hbut = 28, $h Global $hParentGUI, $hlocalPID, $hRecEdit Global $debug = true, $info, $hProcessOther, $PIDAppMe, $PIDAppOther, $iRead, $aret ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo $hParentGui = GUICreate($sAPP_me, $guiw, $guih, $guix, $guiy) $w = ($guiw - 2*$sp) $hlocalPID = GUIctrlCreateLabel("PID=", $x, $y, $w, $hbut) $y += $hbut + $sp $h = ($guih - $y - $sp) $hRecEdit = GUIctrlCreateEdit("", $x, $y, $w, $h) GUICtrlSetFont(-1, 9, $FW_NORMAL, Default, "Courier New") ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo GUIRegisterMsg($WM_IPC_PRIVATE_Grosminet, 'WM_FROM_APP') GUISetState(@SW_SHOW, $hParentGui) GUICtrlSetData($hlocalPID, "PID= " & @AutoItPID) ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo ; Get the SENDER application 'pid' ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo $PIDAppOther = ShellExecute($sSenderEXE) if $PIDAppOther = -1 then MsgBox($MB_SYSTEMMODAL, "ERROR", "Unable to start " & $sAPP_other & " --> error= " & @error) exit Endif sleep(500) $info = "Ready to receive ! Please send a text ..." & @CRLF _ShowInfo($info) ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo ; Get the SENDER application 'process handle' ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo $hProcessOther = _WinAPI_OpenProcess($PROCESS_VM_READ, False, $PIDAppOther) if @error Then $info = "_WinAPI_OpenProcess() error: " & @error & @crlf _ShowInfo($info) exit endif ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo While 1 Switch GUIGetMsg() Case $GUI_EVENT_CLOSE _myExit() EndSwitch WEnd ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo Func WM_FROM_APP($hWnd, $iMsg, $wParam, $lParam) $info = "..... METHOD 1: _WinAPI_CreateBuffer ....." & @crlf _ShowInfo($info) _Method_1($wParam, $lParam) ; $info = "..... METHOD 2: DllStructCreate .........." & @crlf _ShowInfo($info) _Method_2($wParam, $lParam) EndFunc ;==>WM_FROM_APP ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo Func _Method_1($wParam, $lParam) Local $iStrucSize_SENT = Int($lParam) Local $pBuffer = _WinAPI_CreateBuffer($iStrucSize_SENT) $aret = _WinAPI_ReadProcessMemory($hProcessOther, $wParam, $pBuffer, $iStrucSize_SENT, $iRead) ; $info = _DumpStruct($pBuffer, $iStrucSize_SENT) _ShowInfo($info) _ShowInfo(_WinAPI_GetString($pBuffer) & @crlf & "--------------------------------" & @crlf) _WinAPI_FreeMemory($pBuffer) EndFunc ; _Method_1 ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo Func _Method_2($wParam, $lParam) Local $iStrucSize_SENT = Int($lParam) local $TagInfoStruct = "struct;align;byte buf[" & $iStrucSize_SENT & "];endstruct" Local $tbuffer = DllStructCreate($TagInfoStruct) Local $iSizeStruct = DllStructGetSize($tbuffer) Local $pBuffer = DllStructGetPtr($tbuffer) $aret = _WinAPI_ReadProcessMemory($hProcessOther, $wParam, $pBuffer, $iStrucSize_SENT, $iRead) ; $info = _DumpStruct($pBuffer, $iStrucSize_SENT) _ShowInfo($info) _ShowInfo(_WinAPI_GetString($pBuffer) & @crlf & "--------------------------------" & @crlf) $pBuffer = 0 EndFunc ; _Method_2 ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo Func _ShowInfo($msg) if $debug then ConsoleWrite($msg) GUICtrlSetData($hRecEdit, GUICtrlRead($hRecEdit) & $msg) Local $iEnd = StringLen(GUICtrlRead($hRecEdit)) _GUICtrlEdit_SetSel($hRecEdit, $iEnd, $iEnd) _GUICtrlEdit_Scroll($hRecEdit, $SB_SCROLLCARET) Endfunc ; _ShowInfo ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo Func _myExit() _WinAPI_CloseHandle($hProcessOther) ProcessClose($PIDAppOther) GUIDelete() exit Endfunc ; _myExit ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo Func _DumpStruct($p_STRUCT, $iSizeStruct) ; $iSizeStruct = the size of the struct in bytes (DllStructGetSize) Local $TagStructDump = "struct;align;byte[" & $iSizeStruct & "];endstruct" Local $t_Struct = DllStructCreate($TagStructDump, $p_STRUCT) Local $i Local $structInfo = "" _ConsoleWriteInfo($structInfo, "Structure size: " & $iSizeStruct & " byte(s):" & @crlf) for $i = 0 to $iSizeStruct - 1 _ConsoleWriteInfo($structInfo, hex(DllStructGetData($t_Struct, 1, $i), 2) & " ") if (Mod($i+1, 8) = 0) then _ConsoleWriteInfo($structInfo, @CRLF) Endif Next _ConsoleWriteInfo($structInfo, @CRLF) return $structInfo EndFunc ; _DumpStruct ; ooooooooooooooooooooooooooooooooooooooooooooooooooooooo Func _ConsoleWriteInfo(ByRef $msg, $txt) $msg &= $txt EndFunc ; _ConsoleWriteInfo The ISSUE : systematically, the 3 first bytes received are 'corrupted' !!! ???
      NOTE: You must repeat several times sending the same string to check that  bytes sent" and "bytes received"  are equal EXCEPT the 3 first ones !
      I'm quite sure that my code is somewhere wrong ! BUT I'm not able to discover myself WHERE ! I have tried to use 2 methods to read and save the external memory bytes (using the '_WinAPI_CreateBuffer' function, and the 'DllStructCreate' function) --> BOTH give me back the same issue.
      --> So I suspect that my understanding of the '_WinAPI_ReadProcessMemory' function is maybe wrong and I do not correctly call this API.
      ??? Is it correct if I say, [according the MSDN 's ReadProcessMemory explanation or the #include <WinAPI.au3> library code of this function] :
      - the base address of memory to be read is the pointer received from my private WM handler --> i.e. $wParam (regarding my script receiver code)
      - the buffer pointer where to save bytes read (starting from $wParam) is the pointer created using '_WinAPI_CreateBuffer' or 'DllStructCreate + DllStructGetPtr' functions
      - the number of bytes to be read is the information provided by the $lParam variable (regarding my script receiver code)
      - AND of course, the external memory base-address will only be readable if the 'ipc-sender' application handler is correctly declared ($hProcessOther = _WinAPI_OpenProcess($PROCESS_VM_READ, False, $PIDAppOther)).
      There is probably other methods to share strings between cooperating applications, and surely more simple and elegant ones, BUT I'm focusing on these scripts where in fact the types of data to share are not limited to the 'String' type, but could concern any kind of structure.
      Any advice or help to explain me what happens would be welcome.
      Great Thanks in advance for your time passed to help me...
      Alain.
      These are my environment characteristics:
      AutoIT : 3.3.14.2
      OS: Windows 7 Home Premium Service Pack 1 / 7601
       
       
      ipc_receiver.au3
      ipc_sender.au3
    • RTFC
      By RTFC
      Pool provides multiple AutoIt processes with an infrastructure for exchanging messages, data, and remote-control instructions, on a single machine or on a local area network. It is modelled partly on HTCondor, but also incorporates user-defined messages (UDM), LANchat, mails, data Container shipping management, and remote power control and remote execution. That last feature means one process can send lines of AutoIt script to another process, and run it there (including batch control and synchronisation). Consider it my take on inter-process communication (IPC).
       
      Pool incorporates many contributions (a few are included in the bundle, other tiny snippets are part of the main script), from other forum members without whom this project simply would never even have been conceivable. In particular, I hereby want to acknowledge the excellent work of Zatorg, wraithdu, W0uter, Ward, ValeryVal, spudw2k, trancexx, This-is-Me, Nomad, Nine, Manadar, LazyCat, Lakes, Kip, Kealper, Jos, Greencan, FredAI, evilertoaster, dragan, Chimp, and AdamUL. Some codes merely helped to clarify my novice understanding of various IPC issues, others have been gratefully integrated. Three contributors need special mention: Kip & Zatorg for event-driven TCP, and trancexx for MailSlot; together these form the backbone of everything else. The only reason I still dare to publish this under my own name is because I did add over ten thousand lines of code myself. Nevertheless, Pool may serve as a showcase for how various members' example scripts can be combined into a comprehensive environment. Many, many thanks to everyone involved!
       
      Pool is designed to be simple to use. Only a few functions are public, and of those, you really only need to study these four control UDFs:
      _Pool_Send_Command(): Pool's Command & Control Centre (see list of commands below)
      _Pool_Send_UserDefinedMsg(): define your own outgoing msgs
      _Pool_Receive_UserDefinedMsg(): process your own incoming msgs
      _Pool_Send_ExeQcall(): send one or more AutoIt instructions to target's Exe-Queue (ExeQ)
       
      and these ones for data transfers:
      _Pool_Container_Create(): associate shipping Container with data
      _Pool_Container_Destroy(): release Container memory and all associated Sharing relations
      _Pool_Container_CreateShare(): associate destination with Container
      _Pool_Container_DestroyShare(): dissolve sharing association for destination
       
      To get an idea of what _Pool_Send_Command() can do, here's the list of commands:
       
       

       
      Pool is big. It has got many dozens of fixed and dynamic user-defined settings to create specific applications with (see #Region Globals). Please take the time to read the extensive Remarks section at the top of the main script (Pool.au3), including the warnings and limitations. Furthermore, I've provided two example environments in the subdir Examples in the bundle: CLNtest (messages and remote execution) and CLNmaster + CLNslave (data Container shipping). If you run these on a single machine, you'll need to start PostOffice_Solo; on multiple machines, one should start first with PostOffice_Server, the rest with PostOffice_Client, once the Server is initialised. See the readme.txt in the Examples subdir for additional help. Another example (multi-processing a la Condor) is shipped with release 2.4+ of Eigen4AutoIt.
      Since this is an Example script, you can find a number of interesting design ideas in Pool:
       
       
       
       Pool.v0.7.7z Pool bundle, second beta release, version 0.7
        IMPORTANT CAVEAT 
      This is an experimental beta release. Some parts of Pool have never been tested, others only in the simplest possible setting (the largest network I ever tested comprised one desktop and three laptops (one with busted radio) on a crappy Wifi router; I've never tested it on a cabled network). The (W)LAN part still has many issues and may not be able to handle more than a few machines. There's no callstack error handler. Event-driven TCP remains glitchy, and some of my MailSlots keep malfunctioning. I also lack access to Windows 8.* test environments.
      In addition, many issues you may encounter will be due to specific timings/event sequencing (which makes it hard to debug) and/or your specific infrastructure. That means that, most likely, I cannot recreate your Pool bugs (even with your scripts), so I cannot fix them either. Therefore, I will not be offering bug support for users at this point; there's still far too much I need to fix in my own environment to worry about yours, I'm sorry (also, I've been working on this for four months flat, and need a break from it). However, I am of course open to suggestions, remarks, criticism, and kind words (preferably the latter ).
      Finally, I will be exceptionally busy with work for the foreseeable future, so it may take me more time than in the past to respond to your posts.
      Have fun playing in the Pool.
×

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.