Jeemo

Piping binary data to child's StdIn stream from DLLStruct buffer

3 posts in this topic

#1 ·  Posted (edited)

Hello AutoIt community,

I'm working on a project where I need to copy the bits of a file to a child process's StdIn stream. It's almost complete, but I have one final snag. 

The script uses DLLStructCreate to create a 64K buffer that holds the file's binary contents in memory. It iterates through the contents of the desired file in 64K segments, first pulling the bits into the buffer, then dumping the contents of the buffer out to the child process's StdIn stream. The child process then ultimately reconstitutes the file on a remote system.

The child process does build the file in the target directory, but the problem is that the reconstituted file is always slightly larger than the original because it's always a multiple of 64K (when you look at the file properties, the size is always identical to the "size on disk" value). Storage consumption is not a concern since it's technically not occupying more space than the original, but file integrity is a concern. Not every file shows any signs of corruption when opening it, but some of the reconstituted files are completely useless because of this.

I know that the problem lies within the last iteration of reading the source file; for example, if there are only 24K bits remaining to be read, that data gets stored in the DLLStruct along with 40K of zeros to fill the entire 64K buffer. When the file is reconstituted, this padding of zeroes is unfortunately also included. So my challenge is to try to figure out how to ignore these trailing zeroes while reading the final <64K bits of the file, or at least only send part of the contents of the DLLStruct buffer to StdIn. Does anyone know how to go about doing this?

Thanks in advance,

Jeemo

Edited by Jeemo

An emoticon is worth a dozen words.

Share this post


Link to post
Share on other sites



@JohnOne, that is definitely correct.

I've been able to resolve the issue by creating a temporary DLLStruct during the final iteration of the file, and defining the size of it to be equal to the size of the remaining bytes to be read, then transferring the contents of the primary DLLStruct to the temp one. Doing this truncates whatever data from the primary DLLStruct that doesn't fit in the temp one, which is exactly what I need since only the padded zeroes get truncated.


An emoticon is worth a dozen words.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Similar Content

    • dubi
      By dubi
      Hi,


       
      Background: i have a number of instances of the same application that I want to automate in parallel.

      Unfortunately, I cannot completely automate these instances in the background. So, from time to time these instances need to have the focus so that I can interact with the controls via “send” directly.

      Each of the application instances is controlled by a au3 complied script. Each script (called with a parameter) manages the automation of the respective application-instance. Each of the (complied) script (instances) is called by a (central) front end with a gui. The front end controls if the “focus” is available to do the “send” and “mouseclick” modifications. The central front end either allows a child to have the focus or prevents it to get the focus (in which case the child will wait and checks again). The code for the front end is included. Apologies for the lengthy explanation.


      #RequireAdmin #include <ButtonConstants.au3> #include <GUIConstantsEx.au3> #include <WindowsConstants.au3> #include <EditConstants.au3> #include <GUIEdit.au3> #include <ScrollBarConstants.au3> #include <Array.au3> Global Const $APF_ALLOWMULTILINE = 1 Global Const $APF_RIGHTALIGN = 6 Global Const $APF_RIGHTALIGNCOL = 2 Global Const $APF_RIGHTALIGNDATA = 4 Global Const $APF_PRINTROWNUM = 8 Global Const $APF_DEFAULT = $APF_PRINTROWNUM Global $PID[9], $FocusAvailable = True, $previousEditMsg Global $PID_waiting[0][2], $logfile $logfile = "D:\MultiInstance\Logfiles\FrontEnd.txt" If FileExists($logfile) Then FileDelete($logfile) #Region ### START Koda GUI section ### Form= $hGui_1 = GUICreate("Instanzenmanager", 493, 1226, 1807, 93) $grpInst1 = GUICtrlCreateGroup(" 1 ", 8, 8, 233, 121) $btnPause01 = GUICtrlCreateCheckbox("Pause", 16, 64, 50, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $tbnStop01 = GUICtrlCreateCheckbox("Stop", 16, 96, 218, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnMin01 = GUICtrlCreateCheckbox("Minimize", 72, 64, 66, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnWdh01 = GUICtrlCreateCheckbox("Restore", 144, 64, 90, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnStart01 = GUICtrlCreateCheckbox("Start", 16, 32, 218, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) GUICtrlSetBkColor(-1, 0x00FF00) GUICtrlCreateGroup("", -99, -99, 1, 1) $grpInst3 = GUICtrlCreateGroup(" 3 ", 8, 136, 233, 121) $btnPause03 = GUICtrlCreateCheckbox("Pause", 16, 192, 50, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $tbnStop03 = GUICtrlCreateCheckbox("Stop", 16, 224, 218, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnMin03 = GUICtrlCreateCheckbox("Minimize", 72, 192, 66, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnWdh03 = GUICtrlCreateCheckbox("Restore", 144, 192, 90, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnStart03 = GUICtrlCreateCheckbox("Start", 16, 160, 218, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) GUICtrlSetBkColor(-1, 0x00FF00) GUICtrlCreateGroup("", -99, -99, 1, 1) $grpInst2 = GUICtrlCreateGroup(" 2 ", 248, 8, 233, 121) $btnPause02 = GUICtrlCreateCheckbox("Pause", 256, 64, 50, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $tbnStop02 = GUICtrlCreateCheckbox("Stop", 256, 96, 218, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnMin02 = GUICtrlCreateCheckbox("Minimize", 312, 64, 66, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnWdh02 = GUICtrlCreateCheckbox("Restore", 384, 64, 90, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnStart02 = GUICtrlCreateCheckbox("Start", 256, 32, 218, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) GUICtrlSetBkColor(-1, 0x00FF00) GUICtrlCreateGroup("", -99, -99, 1, 1) $grpInst4 = GUICtrlCreateGroup(" 4 ", 248, 136, 233, 121) $btnPause04 = GUICtrlCreateCheckbox("Pause", 256, 192, 50, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $tbnStop04 = GUICtrlCreateCheckbox("Stop", 256, 224, 218, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnMin04 = GUICtrlCreateCheckbox("Minimize", 312, 192, 66, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnWdh04 = GUICtrlCreateCheckbox("Restore", 384, 192, 90, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnStart04 = GUICtrlCreateCheckbox("Start", 256, 160, 218, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) GUICtrlSetBkColor(-1, 0x00FF00) $Edit1 = GUICtrlCreateEdit("", 8, 720, 473, 497) $btnPauseAll = GUICtrlCreateCheckbox("Pause all", 8, 656, 474, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) $btnStopAll = GUICtrlCreateCheckbox("Stop all", 7, 688, 474, 25, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_PUSHLIKE)) GUISetState(@SW_SHOW) #EndRegion ### END Koda GUI section ### While 1 CheckGuiMsg() FileWrite($logfile, @HOUR & ":" & @MIN & ":" & @SEC & "> " & "CheckGuiMsg" & @CRLF) CheckClientMessages() FileWrite($logfile, @HOUR & ":" & @MIN & ":" & @SEC & "> " & "CheckClientMessages" & @CRLF) WEnd Func CheckGuiMsg() $nMsg = GUIGetMsg() Switch $nMsg Case $GUI_EVENT_CLOSE Exit Case $btnStart01 AddTextToEdit("Starting Instance 1") $PID[0] = Run("D:\XVM05\Entwicklung_FilterTest_Multi_ToFileV001.exe 1", @ScriptDir, Default, 3) Case $btnStart02 AddTextToEdit("Starting Instance 2") $PID[1] = Run("D:\XVM05\Entwicklung_FilterTest_Multi_ToFileV001.exe 2", @ScriptDir, Default, 3) Case $btnStart03 AddTextToEdit("Starting Instance 3") $PID[2] = Run("D:\XVM05\Entwicklung_FilterTest_Multi_ToFileV001.exe 3", @ScriptDir, Default, 3) Case $btnStart04 AddTextToEdit("Starting Instance 4") $PID[3] = Run("D:\XVM05\Entwicklung_FilterTest_Multi_ToFileV001.exe 4", @ScriptDir, Default, 3) Case $btnPause01 AddTextToEdit("Send Pause to Instance 1") StdinWrite($PID[0], "Pause") Case $btnPause02 StdinWrite($PID[1], "Pause") Case $btnPause03 StdinWrite($PID[2], "Pause") Case $btnPause04 StdinWrite($PID[3], "Pause") Case $tbnStop01 StdinWrite($PID[0], "Stop") Case $tbnStop02 StdinWrite($PID[1], "Stop") Case $tbnStop03 StdinWrite($PID[2], "Stop") Case $tbnStop04 StdinWrite($PID[3], "Stop") Case $btnPauseAll AddTextToEdit(@CRLF & "************Pause All not yet implemented**************" & @CRLF) Case $btnStopAll AddTextToEdit(@CRLF & "************Stop All not yet implemented***************" & @CRLF) EndSwitch EndFunc ;==>CheckGuiMsg Func CheckClientMessages() For $i = 0 To 3 FileWrite($logfile, @HOUR & ":" & @MIN & ":" & @SEC & "> " & $i & @CRLF) Local $a = TimerInit() $p = $PID[$i] $streamRead = StdoutRead($p) If $streamRead <> "" Then Switch $streamRead Case StringInStr($streamRead, "Focus Needed") > 0 If $FocusAvailable Then $FocusAvailable = False StdinWrite($p, "Focus Granted") Else EndIf Case StringInStr($streamRead, "Release Focus") > 0 StdinWrite($p, "Release Focus Received") $FocusAvailable = True Case Else EndSwitch EndIf FileWrite($logfile, @HOUR & ":" & @MIN & ":" & @SEC & "> " & $i & " " & round(TimerDiff($a),2) & @CRLF) Next EndFunc ;==>CheckClientMessages Func AddTextToEdit($text) If $previousEditMsg <> $text Then $previousEditMsg = $text GUICtrlSetData($Edit1, GUICtrlRead($Edit1) & @YEAR & "." & @MON & "." & @MDAY & " - " & @HOUR & ":" & @MIN & ":" & @SEC & "> " & $text & @CRLF) _GUICtrlEdit_Scroll($Edit1, $SB_SCROLLCARET) EndIf EndFunc ;==>AddTextToEdit



      My issue now is that the mechanism with with StdoutRead and StdinWrite is not efficient at all. The more instances I start the slower it gets. This is not just a bit slower (like a fraction of a second), but to the degree that the front end is not responding at all any longer (with 3 instances handling).

      So my questions are:

      1.       Is there a flaw in my implementation with StdoutRead and StdinWrite? (note that all works fine with 1 and also (slower) with 2 instances running) but actually breaks down with 3 instances running.

      2.       Can I optimize the currently used implementation so that I can control 30+ instances?

      3.       What other implementation do you see suitable for this approach?

      a.       I have already tried it with communication through files but observed that this is not sufficiently reliable with multiple instances.

      b.       Is Named Pipes a more performant approach (I am a bit scared of the effort to learn and implement this)

      c.       Any other method?


       
      Many thanks in advance

      -dubi





    • Quarrel
      By Quarrel
      Opt("WinTitleMatchMode", 2) ControlClick("codebuffer1.au3", "Tab", 354, "left") ControlSend("codebuffer1.au3", "Tab", 354, "{F5}")
      So after using SciTE with AutoIT for a few months, I decided I'd like to test different buffer code while working on an included page on a different buffer. So send a control click to Scite to click then send F5? I was thinking it would work but it does not for me. Using the finder tool it appears the tab is Control ID 354 and with WinTitleMatchMode set to '2' I was expecting results... Can anyone show me the error of my ways?
    • SpinningCone
      By SpinningCone
      OK so i'm trying to build a tool part of which adds a GUI to a command line tool.  my tests seemed to work fine but when i ported my function into a GUI they break, the first time i invoke the commandline it returns nothing, then each subsequent time i press the button that calls to command line it ignores the current command and re-sends the last command. if i press the button a second time the command updates.    IE let's say  i have a command line app with a counter as one of the inputs.

      Press button to generate output -> "0" returned
      Press Button again with same paramaters -> "Hello world 1" returned
      Change counter to "2" press button  -> "Hello world 1" returned
      press the button again ->  "Hello world 2" returned
       
      THis was boggling my mind so i stripped out the GUI and took the function that was being called in the GUI and make it a hotkey call.   now the exact same code that calls the commandline has different behavior



      Press hotkey to generate output -> "0" returned
      Press hotkey again with same paramaters -> "Hello world 1" returned
      Change counter to "2" press hotkey-> "Hello world 2" returned
       
       
      So a hotkey act's "normally" bot only after the initial "0" is returned.
       
      if I perform the function call for the command line before the hotkey while loop it functions properly the first time. :   This seems like some sort of autoit bug but i have no idea how to work around it. : example code
       
       
       
      #include <Constants.au3> #include <array.au3> #include <ButtonConstants.au3> #include <EditConstants.au3> #include <GUIConstantsEx.au3> #include <StaticConstants.au3> #include <WindowsConstants.au3> Opt("GUIOnEventMode", 1) #Region ### START Koda GUI section ### Form=c:\users\datwater\documents\autoit\projects\otp tool\otp tool.kxf $Form1 = GUICreate("Form1", 392, 448, 191, 148) GUISetOnEvent($GUI_EVENT_CLOSE, "Form1Close") $Label1 = GUICtrlCreateLabel("seed", 23, 57, 35, 17) $txtSeed = GUICtrlCreateInput("", 66, 55, 213, 21) $btnRandomSeed = GUICtrlCreateButton("Create Seed", 287, 53, 74, 25) GUICtrlSetOnEvent(-1, "btnRandomSeedClick") $txtCounter = GUICtrlCreateInput("5", 72, 96, 71, 21) $txtLength = GUICtrlCreateInput("6", 223, 96, 48, 21) $Label2 = GUICtrlCreateLabel("Counter", 24, 99, 41, 17) $label3 = GUICtrlCreateLabel("Length", 173, 98, 37, 17) $txtOutput = GUICtrlCreateEdit("", 38, 213, 326, 201) $btnGen = GUICtrlCreateButton("Run Command", 149, 154, 80, 25) GUICtrlSetOnEvent(-1, "btnGenClick") GUISetState(@SW_SHOW) #EndRegion ### END Koda GUI section ### local $key = "" local $length = 8 local $count = 5 Local $cmd = Run("cmd.exe", @ScriptDir, @SW_SHOWNORMAL, $STDIN_CHILD + $STDOUT_CHILD) While 1 Sleep(100) WEnd Func btnGenOTPClick() $length = GUICtrlRead($txtLength) $count = GUICtrlRead($txtCounter) $Seed = GUICtrlRead($txtSeed) $otp = runCommand($Seed,$length,$count) GUICtrlSetData($txtOutput,$otp & @CRLF, "|") EndFunc Func btnRandomSeedClick() GUICtrlSetData($txtKey,genKey(40)) EndFunc func runCommand($s,$l,$c) ;build the command $command = "oathgen.exe -c " & $c & " -l " & $l &" -s " & $s & "& @CRLF ConsoleWrite("COMMAND: " & $command & @CRLF) ; debug check ;send command to the console StdinWrite($cmd, $command) $line = "" ;loops and gathers all the command output While True $line &= StdoutRead($cmd) ConsoleWrite($line) ;debug If @error Then ExitLoop ;to exit the while loop we look for the blank command prompt, the trimming is needed to get a clean match. If StringRight($line,StringLen(@ScriptDir) + 1) = @ScriptDir & ">" Then ExitLoop Sleep(30) WEnd ;i do some parsing here of the output then return it , that all works fine EndFunc

      Basically where i'm calling StdinWrite($cmd, $command) i can see that $command is what i want it to be,  but the first time it runs nothing is sent and the second time you press the button the command is correct but it actually sends the *old* command to standard in . This is only when running in a GUI though. if you lop out the runCommand function and run it just in a script it works as expected
    • kcvinu
      By kcvinu
      Hi all,
      Last day i was tried some inter process communication code but not succeeded, as it is a big deal for me. Then i tried with ConsoleWriteError function. It worked for me. Here is my code. I am seeking suggestions to improve. 
      But my doubt is that; I can't understand the idea of StdinWrite function completely. This is what i think. Please correct me if i am wrong.
      1. Make a child program for collect and send the data.
      2. Make a parent program and run the child within it and receive data from the child program.
      So, i made the sample program with this scenario. But, when i looked StdinWrite function, it needs a process ID as a parameter. I thought that StdinWrite is used in the child program to send data to parent program. Am i right ?. 
      My code
      Child program for collecting and sending data.
      Local $compName = @ComputerName ConsoleWriteError($compName) Exit Parent program for receiving data
      #include <AutoItConstants.au3> Local $process = Run(@ScriptDir & "\stdWrite.exe","",Default, $STDERR_CHILD ) Local $Data While 1 $Data &= StderrRead($process) If $Data <> "" Then ExitLoop WEnd MsgBox(0, "Std Test Data", $Data)  
    • mojomatt
      By mojomatt
      Question regarding StdinWrite  and Stdoutread…
      Here’s my setup…
      I’m opening a powershell session using the AutoIT Run function with the $STDIN_CHILD  and $STDOUT_CHILD optional flags.
      I’m then sending a command using the STDINWRITE function to connect to a remote server which is a long distance away and it takes a “long” time to return results.
      I’m then trying to read the STDOUT stream.
      After this there are other STDINWRITE statements that I need to make so the STREAM hasn’t closed at this point.
      My question is this…How do I know for sure when all the data has been returned from the remote server after my first STDINWRITE?
      Things I’ve tried and don’t work…
      Use a loop which waits for @error to be non-zero.  This doesn’t work because as long as the STREAM is open then @error never is set to non-zero.
      Use a loop which waits for @extended to be 0.  This doesn’t work because the remote server sends data back in bursts and I don’t know how much time will pass in-between bursts.
      The only thing that is somewhat reliable is utilizing very long sleep statements – like 5 minutes – which really slows down my script and doesn’t ensure success either.
      What I really need is a way to tell that control has been returned to the command prompt.  What I mean by this is if you open a command prompt and issue the command DIR C:\ /s which lists all the files and folders on your C: drive you know you can’t issue any more commands until you get a blinking cursor at the command prompt again.  How do I programatically know when I get a blinking cursor again?
      Sample code that isn’t 100% foolproof…
      This code uses the logic looking for an @error to be non-zero...
      #include <constants.au3> Dim $StrOutput $StrPowerShellPID = Run('C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe  -command - ',@SystemDir, @SW_HIDE, $STDIN_CHILD + $STDERR_MERGED) StdinWrite($StrPowerShellPID, "Dir C:\" & @CRLF) ;this is just an example command ; I can't close the STDOUT stream at this point because I need to issue additional commands into the existing STDIN stream later in the script While 1 ;Wait for EOF as indicated by @error being non-zero     $StrOutput &= StdoutRead($StrPowerShellPID)     If @error <> 0 Then ExitLoop     sleep (500) WEnd Consolewrite($StrOutput) The above code doesn't work because @error will never be set to non-zero since the stream isn't closed.  I can't close the stream because I need to issue additional commands into it.
       
       
      This code uses logic looking for an @extended value of 0 indicating no data was returned to the stream...
      #include <constants.au3> Dim $StrOutput $StrPowerShellPID = Run('C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -command - ',@SystemDir, @SW_HIDE, $STDIN_CHILD + $STDERR_MERGED) StdinWrite($StrPowerShellPID, "Dir ""C:\program files\common files"" -recurse" & @CRLF) While 1 ;This loop detects when data starts to stream in and then exits the loop $StrOutput &= StdoutRead($StrPowerShellPID) If @extended > 0 Then ExitLoop sleep(1000) WEnd While 1 ;Now that data has begun to stream in I wait until I can't read any more data as indicated by 0 bytes read in the value of @extended $StrOutput &= StdoutRead($StrPowerShellPID) If @extended = 0 Then ExitLoop ;~ sleep (1000) ; this is the only way to get this specific sample to work but this won't work for me because I don't know the time in-between bursts of data coming into the STDIN stream WEnd ConsoleWrite($StrOutput) ;If you look at the output you'll see it's not complete because @extended was set to 0 during the second loop The above code doesn't always work because if the remote server doesn't send any data for several seconds, then the StdoutRead thinks it's done reading when in reality it may not be.