Jump to content
Sign in to follow this  
TheDcoder

Reading a child process's output is impossible without blocking and Windows won't return even after the child ends

Recommended Posts

Hi, I thought I would never post a C/WinAPI related question in this forum ever, but here we are after a few years and me having learnt enough of C to write a basic console program :)

My issue is that I am trying to read my child process's stdout output but ReadFile never returns if the child exits or if it is killed... very strange :unsure:, I have been trying to work my way around this. The options I can think of are:

  1. Create a new thread and check for existance of the process constantly while reading
  2. Somehow make the pipe asynchronous (overlapped) so that I can read it in a non-blocking manner
  3. Fix ReadFile to return when the process ends

Obviously I would prefer No. 3, I just want to make my program work. Here is my code if you guys want to take a look:

// No text highlighting for C/C++ but we have it for C#? Blasphemy!

bool allium_start(struct TorInstance *instance, char *config, allium_pipe *output_pipes) {
    char *cmd;
    
    // Figure out the command string for execution
    if (config) {
        char *parameters = " -f -";
        cmd = malloc(strlen(instance->tor_path) + strlen(parameters) + 1);
        if (!cmd) return false;
        strcpy(cmd, instance->tor_path);
        strcat(cmd, parameters);
    } else cmd = instance->tor_path;
    
    // Prepare startup info with appropriate information
    SecureZeroMemory(&instance->startup_info, sizeof instance->startup_info);
    instance->startup_info.dwFlags = STARTF_USESTDHANDLES;
    
    SECURITY_ATTRIBUTES pipe_secu_attribs = {sizeof(SECURITY_ATTRIBUTES), NULL, true};
    
    HANDLE pipes[2];
    if (output_pipes == NULL) {
        CreatePipe(&pipes[0], &pipes[1], &pipe_secu_attribs, 0);
        output_pipes = pipes;
    }
    instance->startup_info.hStdOutput = output_pipes[1];
    instance->startup_info.hStdError = output_pipes[1];
    instance->stdout_pipe = output_pipes[0]; // Stored for internal reference
    
    if (config) {
        // Reuse the pipes array to store standard input pipes
        CreatePipe(&pipes[0], &pipes[1], &pipe_secu_attribs, 0);
        instance->startup_info.hStdInput = pipes[0];
    }
    
    // Create the process
    bool success = CreateProcessA(
        NULL,
        cmd,
        NULL,
        NULL,
        config ? true : false,
        0,
        NULL,
        NULL,
        &instance->startup_info,
        SecureZeroMemory(&instance->process, sizeof instance->process)
    );
    
    // Free command string if needed
    if (config) free(cmd);
    
    // Write config to Tor's standard input
    unsigned long bytes_written;
    if (success) {
        WriteFile(pipes[1], config, strlen(config), &bytes_written, NULL);
        // Work around for simulating Ctrl + Z which sends the substitution character (ASCII 26),
        // this is needed in order for Tor to detect EOT/EOF while reading the config
        WriteFile(pipes[1], &(char){26}, 1, &bytes_written, NULL);
    }
    CloseHandle(pipes[1]);
    
    // Return on failure
    if (!success) return false;
}

char *allium_read_stdout_line(struct TorInstance *instance) {
    char *buffer = instance->buffer.data;
    
    // Check for valid buffer and allocate if needed
    if (instance->buffer.size == 0 || !buffer) {
        buffer = instance->buffer.data = malloc(instance->buffer.size = 80 + 1);
        if (!buffer) return NULL;
    }
    
    // Process the input
    unsigned int read_len = 0;
    while (true) {
        // Read data
        unsigned long bytes_read;
        if (ReadFile(instance->stdout_pipe, buffer, 1, &bytes_read, NULL) == false || bytes_read == 0) return NULL;
        
        // Check if we have reached end of line
        if (buffer[0] == '\n') break;
        
        // Proceed to the next character
        ++buffer; ++read_len;
        
        // Resize buffer if it is full
        if (read_len == instance->buffer.size) {
            char *new_buffer = malloc(instance->buffer.size += 50);
            if (new_buffer) memcpy(new_buffer, instance->buffer.data, read_len);
            free(instance->buffer.data);
            if (!new_buffer) return NULL;
            instance->buffer.data = new_buffer;
            buffer = instance->buffer.data + read_len;
        }
    }
    
    // Terminate the new line with null character and return
    // Special handling for Windows, terminate at CR if present
    buffer[read_len >= 2 && buffer[-1] == '\r' ? -1 : 0] = '\0';
}

The allium_start function creates the redirection pipes and the child process, the other allium_read_stdout_line function reads from the stdout pipe created by the first function, ReadFile in this function does not return when the child ends or gets killed.

I appriciate the help of the WinAPI gurus here, thanks in advance! :D

Share this post


Link to post
Share on other sites

@Earthshine Yup, was one of the first results that I got a few weeks back when investigating how output redirection works in POSIX.

Sadly it only talks about the POSIX functions to do this, Windows doesn't provide this functionality as far as I am aware. My code for POSIX works like a charm, the issue is only on Windows :(.

The ReadFile documentation mentions:

Quote

When a synchronous read operation reaches the end of a file, ReadFile returns TRUE and sets *lpNumberOfBytesRead to zero.

This does not happen when the process ends... it just never returns!

I wonder how the internal AutoIt functions like Run and StdoutRead work... I am especially keen on how StdoutRead can be non-blocking, does AutoIt use multi-threading to maintain to retrieve the output later? I might use this method, but I would like to avoid it if possible.

Share this post


Link to post
Share on other sites

It turns out that I lacked basic understanding of how handles work... and that they are duplicated when passed through CreateProcess, so you need to close your own copies of the handles in order for the pipe to properly close when the program exits. I also discovered that inheritence is a bit more complicated, as both handles (each for an end of the pipe) are marked as inheritable by the CreatePipe function during creation... this causes problems yet again by passively duplicating the handles which were not even passed to CreateProcess, you can prevent this by using the SetHandleInformation function. Here is the working code:

bool allium_start(struct TorInstance *instance, char *config, allium_pipe *output_pipes) {
    // Prepare startup info with appropriate information
    SecureZeroMemory(&instance->startup_info, sizeof instance->startup_info);
    instance->startup_info.dwFlags = STARTF_USESTDHANDLES;

    SECURITY_ATTRIBUTES pipe_secu_attribs = {sizeof(SECURITY_ATTRIBUTES), NULL, true};

    HANDLE pipes[2];
    if (output_pipes == NULL) {
        CreatePipe(&pipes[0], &pipes[1], &pipe_secu_attribs, 0);
        output_pipes = pipes;
    }
    SetHandleInformation(output_pipes[0], HANDLE_FLAG_INHERIT, 0);
    instance->startup_info.hStdOutput = output_pipes[1];
    instance->startup_info.hStdError = output_pipes[1];
    instance->stdout_pipe = output_pipes[0]; // Stored for internal reference

    // Create the process
    bool success = CreateProcessA(
        NULL,
        cmd,
        NULL,
        NULL,
        config ? true : false,
        0,
        NULL,
        NULL,
        &instance->startup_info,
        SecureZeroMemory(&instance->process, sizeof instance->process)
    );

    // Close the write end of our stdout handle
    CloseHandle(output_pipes[1]);

    // Return on failure
    if (!success) return false;
}

You can refer to the accepted answer in my StackOverflow question for more details, many thanks to Remy Lebeau and RbMm for answering my questions there.

Share this post


Link to post
Share on other sites
On 1/29/2019 at 8:50 AM, TheDcoder said:

Just posted this question over at StackOverflow in hopes of getting answers:  -snip- Hope this may help.

Thanks for sharing this link. I have found a couple of answers that are useful and could be implemented in work.

Edited by Jos
spam removed but we still expect that answer whenever you return

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By UEZ
      Here another approach to check if a script was already started using atoms and semaphores.
       
      Atom:
      #include <MsgBoxConstants.au3> Global $iSingleton = Singleton() If Not $iSingleton Then Exit MsgBox($MB_TOPMOST, "Singleton Test", "Process is already running!") EndIf MsgBox($MB_TOPMOST, "Singleton Test", "Singleton atom initialized: " & $iSingleton) Singleton_Delete($iSingleton) ; #FUNCTION# ==================================================================================================================== ; Name ..........: Singleton ; Description ...: Checks if the script has been started already. ; Syntax ........: Singleton([$sOccurrenceName = @ScriptFullPath]) ; Parameters ....: $sOccurrenceName - [optional] a string value. Default is @ScriptFullPath. ; Return values .: If the function succeeds, the return value is the newly created atom or 0 else error is set and false is returned. ; Author ........: UEZ ; Modified ......: ; Remarks .......: If Singleton finds the atom it will return 0 and the atom token will be set to extended macro. It can be used to get the atom string using _WinAPI_AtomGlobalGetName. ; Related .......: ; Link ..........: https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-globalfindatomw ; https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-globaladdatomw ; Example .......: No ; =============================================================================================================================== Func Singleton($sOccurrenceName = @ScriptFullPath) Local $iFind = _WinAPI_AtomGlobalFind($sOccurrenceName) If @error Then Return SetError(1, 0, False) If $iFind Then Return SetExtended($iFind, 0) Local $iAtom = _WinAPI_AtomGlobalAdd($sOccurrenceName) If @error Then Return SetError(2, 0, False) Return $iAtom EndFunc ;==>Singleton ; #FUNCTION# ==================================================================================================================== ; Name ..........: Singleton_Delete ; Description ...: Deletes the atom generated by the first started script. ; Syntax ........: Singleton_Delete($iAtom) ; Parameters ....: $iAtom - an integer value which was generated by Singleton ; Return values .: True if successful else false. ; Author ........: UEZ ; Modified ......: ; Remarks .......: Don't forget to call Singleton_Delete before first started script ends. ; Related .......: ; Link ..........: https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-globaldeleteatom ; Example .......: No ; =============================================================================================================================== Func Singleton_Delete($iAtom) _WinAPI_AtomGlobalDelete($iAtom) If @error Then Return SetError(1, 0, False) Return True EndFunc ;==>Singleton_Delete ;internal functions Func _WinAPI_AtomGlobalFind($sAtomString) Local $aReturn = DllCall("kernel32.dll", "short", "GlobalFindAtomW", "wstr", $sAtomString) If @error Then Return SetError(1, 0, -1) Return $aReturn[0] EndFunc ;==>_WinAPI_AtomGlobalFind Func _WinAPI_AtomGlobalAdd($sAtomString) Local $aReturn = DllCall("kernel32.dll", "short", "GlobalAddAtomW", "wstr", $sAtomString) If @error Then Return SetError(1, 0, -1) Return $aReturn[0] EndFunc ;==>_WinAPI_AtomGlobalAdd Func _WinAPI_AtomGlobalDelete($nAtom) Local $aReturn = DllCall("kernel32.dll", "short", "GlobalDeleteAtom", "short", $nAtom) If @error Then Return SetError(1, 0, -1) Return $aReturn[0] = 0 EndFunc ;==>_WinAPI_AtomGlobalDelete Func _WinAPI_AtomGlobalGetName($nAtom, $iBufferSize = 512) Local $tBufferAtom = DllStructCreate("wchar name[" & $iBufferSize & "]") Local $aReturn = DllCall("kernel32.dll", "uint", "GlobalGetAtomNameW", "short", $nAtom, "struct*", $tBufferAtom, "int", $iBufferSize) If @error Or Not $aReturn[0] Then Return SetError(1, 0, -1) Return $tBufferAtom.name EndFunc ;==>_WinAPI_AtomGlobalGetName  
      Semaphore:
      #include <MsgBoxConstants.au3> #include <WinAPIError.au3> Global $iSingleton = Singleton("&]8h/x87</htFV4-K*&.b.w~") If Not $iSingleton Then Exit MsgBox($MB_TOPMOST, "Singleton Test", "Process is already running!") EndIf MsgBox($MB_TOPMOST, "Singleton Test", "Singleton Semaphore initialized: " & $iSingleton) ; #FUNCTION# ==================================================================================================================== ; Name ..........: Singleton ; Description ...: Checks if the script has been started already. ; Syntax ........: Singleton($sOccurrenceName) ; Parameters ....: $sOccurrenceName - a string value which will be used to create the semaphore handle. ; Return values .: True if Singleton started the first time. False if script was already started ; Author ........: UEZ ; Modified ......: ; Remarks .......: The system closes the handle automatically when the process terminates. The semaphore object is destroyed when its last handle has been closed. ; Related .......: ; Link ..........: https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createsemaphorea ; Example .......: No ; =============================================================================================================================== Func Singleton($sOccurrenceName) If StringLen($sOccurrenceName) > 260 Then $sOccurrenceName = StringLeft($sOccurrenceName, 260) Local $aReturn = DllCall("kernel32.dll", "handle", "CreateSemaphoreA", "ptr", Null, "long", 0, "long", 1, "str", $sOccurrenceName) If @error Or Not $aReturn[0] Then Return SetError(1, 0, -1) Return SetExtended($aReturn[0], $aReturn[0] And _WinAPI_GetLastErrorMessage() = "The operation completed successfully.") EndFunc ;==>Singleton  
      Just start the script twice to see if it works.
      The disadvantage of using atoms is that atoms have a memory that means when your app is crashing or you forgot to delete the atom then the atom does still have the $sOccurrenceName saved and thus Singleton will not work if you use the same same value for $sOccurrenceName.
      With semaphore you don't have this issue.
       
      Thanks to jj2007 and SARG.
    • By BlueSkyMemory
      Hello guys! I'm a rookie in AutoIt lol.
      I've tried to looking up in MSDN and the UDFs, but it can only get the GUID of a usual partition and with the GUID to control it. Now I have no ways😥.
      Thanks a lot for your help!
    • By emendelson
      Using some very ingenious scripts that I found on this forum, I've put together a special-purpose folder watcher that watches a specified folder for printer output files and then either send them to a printer or converts them to a PDF and prints or opens the PDF. The page about this utility is here:
      http://www.columbia.edu/~em36/printfileprinter.html
      The one serious problem that it has is that it seems not to detect a new file if there are more than two or three files already in the watched folder. I can't figure out what I'm doing wrong, and will be grateful for any help. Here is the relevant part of the script. I've left out the functions that test whether the file is in use or not, and that send the raw data to the printer or create a PDF, etc. I hope there's enough code here to make sense of it, and will be very grateful for any help. Again, the problem is that the script doesn't detect newly-created files in watched folders with more than a very few files already in it.
      My totally ignorant guess is that the problem is in the line $iID = _WinAPI_WaitForMultipleObjects(2, $paObj, 0, 0) - but I don't know how to change it and of course I'm only guessing whether it's relevant or not. Many thanks
       
      Global $g_ahObj[2] $g_ahObj[0] = _WinAPI_FindFirstChangeNotification($watchPath, $FILE_NOTIFY_CHANGE_FILE_NAME) $g_ahObj[1] = _WinAPI_FindFirstChangeNotification($watchPath, $FILE_NOTIFY_CHANGE_DIR_NAME) If (Not $g_ahObj[0]) Or (Not $g_ahObj[1]) Then MsgBox(BitOR($MB_ICONERROR, $MB_SYSTEMMODAL), 'Error', 'Unable to create change notification.') Exit EndIf Local $tObjs = DllStructCreate('ptr;ptr') Local $paObj = DllStructGetPtr($tObjs) For $i = 0 To 1 DllStructSetData($tObjs, $i + 1, $g_ahObj[$i]) Next Local $iID While 1 Sleep(100) $select = 0 $print = 0 $format = "" Local $tempPDF $tempPDF = 0 $iID = _WinAPI_WaitForMultipleObjects(2, $paObj, 0, 0) Switch $iID Case 0 ; WAIT_OBJECT_0 ; ConsoleWrite('A file was created, renamed, or deleted in the directory.' & @CRLF) Local $hSearch = FileFindFirstFile($watchPath & "\*") Local $sFileName = "", $iResult = 0 Local $sFilePath = "" While 1 $sFileName = FileFindNextFile($hSearch) ; If there is no more file matching the search. If @error Then ExitLoop $sFilePath = $watchPath & "\" & $sFileName Local $fileUsed $fileUsed = 0 $fileUsed = _FileIsUsed($sFilePath) If $fileUsed = 1 Then While 1 Sleep(100) $fileUsed = _FileIsUsed($sFilePath) If $fileUsed = 0 Then ExitLoop WEnd EndIf If StringInStr($sFileName, "raw") Then If StringInStr($sFileName, "select") Then $select = 1 PrintRawFile($sFilePath, $select) Else If StringInStr($sFileName, "lg.") Then $pageSize = "legal" ElseIf StringInStr($sFileName, "a4.") Then $pageSize = "a4" ElseIf StringInStr($sFileName, "us.") Then $pageSize = "letter" EndIf $pdfTemp = 0 If StringLower(StringLeft($sFileName, 7) = "pdftemp") Then $pdfTemp = 1 If StringInStr($sFileName, ".pcl") Then $format = "pcl" $print = 1 If StringLeft($sFileName, 6) = "select" Then $select = 1 $print = 1 ElseIf StringLeft($sFileName, 3) = "pdf" Then $select = 0 $print = 0 EndIf MakePDF($sFilePath, $format, $print, $select, $pageSize, $pdfTemp) ElseIf StringInStr($sFileName, ".ps") Then $format = "ps" $print = 1 If StringLeft($sFileName, 6) = "select" Then $select = 1 $print = 1 ElseIf StringLeft($sFileName, 3) = "pdf" Then $select = 0 $print = 0 EndIf MakePDF($sFilePath, $format, $print, $select, $pageSize, $pdfTemp) ElseIf StringInStr($sFileName, ".esc") Then Sleep(200) $format = "epson" $print = 1 If StringLeft($sFileName, 6) = "select" Then $select = 1 $print = 1 ElseIf StringLeft($sFileName, 3) = "pdf" Then $select = 0 $print = 0 EndIf MakePDF($sFilePath, $format, $print, $select, $pageSize, $pdfTemp) ElseIf StringInStr($sFileName, ".prn") Then $format = GetFileFormat($sFilePath) $print = 1 If StringLeft($sFileName, 6) = "select" Then $select = 1 $print = 1 ElseIf StringLeft($sFileName, 3) = "pdf" Then $select = 0 $print = 0 EndIf ConsoleWrite("PRN test format: " & $format & @CRLF) MakePDF($sFilePath, $format, $print, $select, $pageSize, $pdfTemp) EndIf EndIf ; Display the file name. ; $iResult = MsgBox(BitOR($MB_SYSTEMMODAL, $MB_OKCANCEL), "", $watchPath & "\" & $sFileName) ;~ RunWait(@ComSpec & " /c notepad.exe " & $watchPath & "\" & $sFileName) ;~ FileDelete($watchPath & "\" & $sFileName) If $iResult <> $IDOK Then ExitLoop ; If the user clicks on the cancel/close button. WEnd ; Close the search handle. FileClose($hSearch) Case 1 ; WAIT_OBJECT_0 + 1 ; ConsoleWrite('A directory was created, renamed, or deleted.' & @CRLF) Case Else ContinueLoop EndSwitch If Not _WinAPI_FindNextChangeNotification($g_ahObj[$iID]) Then MsgBox(BitOR($MB_ICONERROR, $MB_SYSTEMMODAL), 'Error', 'Unexpected error.') Exit EndIf WEnd  
    • By xYuri
      This simple dllcall gives me error 5, access denied,
      Func _WinAPI_VkKeyScan($__key) _WinAPI_SetLastError(0) $res = DllCall('User32.dll', 'SHORT', 'VkKeyScan', 'CHAR', $__key) _xConsole('res: '&$res) $_LastErr = _WinAPI_GetLastError() If $_LastErr <> 0 Then _xConsole('Err: {' & $_LastErr & '}> ' & _WinAPI_GetLastErrorMessage()) Return $res EndFunc Am i doing something wrong?
      Also tried VkKeyScanA and W
      Edit:
      I want to send `:` via PostMessage() WM_KEYDOWN
    • By Sori
      Not going to dump the code down because it's essentially a keylogger.
      <snip>
×
×
  • Create New...