Jump to content
Sign in to follow this  
ChristianDB

Problems capturing console output from command line process

Recommended Posts

ChristianDB

Hi everyone,

I'm presently working on automating a command line process. We usually launch it using a batch file but for now I'm just trying to call the .exe directly without its command line arguments to test some AutoIt features. The process is completely command-line based and runs in a cmd.exe window. Normally, the process starts, asks a question to the user, waits for a character input, goes on, asks another question, etc. I obviously want to answer automatically those questions but I also want to capture the console output and hopefully store it to a file. 

Automating user input wasn't a problem, but for now I was unable to gather anything my process throws at the console. So I tried the following very simple code. The "peek" flag of the StdoutRead function set to true is not what I would want on the long-run but for debugging purposes I keep it on. 
 
#include <Constants.au3>

Local $consoleOutputHandle = Run(@ComSpec & " C:path_to_process\process.exe", @SystemDir, @SW_MAXIMIZE, $STDERR_CHILD + $STDOUT_CHILD)
Local $consoleContent

For $i = 1 To 5
    $consoleContent &= StdoutRead($consoleOutputHandle, true, false)
Sleep(500)
Next

FileWrite("test.txt", $consoleContent)

 

Then I see a cmd.exe window pop for a fraction of a second. I see nothing in it so I'm brought to believe that the console output does get intercepted by AutoIt. In "text.txt", I see the following lines:

 

Microsoft Windows [Version 6.1.7601]

Copyright © 2009 Microsoft Corporation.  All rights reserved.
 
C:Windowssystem32>Microsoft Windows [Version 6.1.7601]
Copyright © 2009 Microsoft Corporation.  All rights reserved.
 
C:Windowssystem32>Microsoft Windows [Version 6.1.7601]
Copyright © 2009 Microsoft Corporation.  All rights reserved.
 
C:Windowssystem32>Microsoft Windows [Version 6.1.7601]
Copyright © 2009 Microsoft Corporation.  All rights reserved.
 
C:Windowssystem32>

 

As you can notice, I can capture what cmd.exe sends when starting but nothing from my process. 

And If I try calling my process without the @ComSpec macro, like so,

#include <Constants.au3>

Local $consoleOutputHandle = Run("C:path_to_process\process.exe", @SystemDir, @SW_MAXIMIZE, $STDERR_CHILD + $STDOUT_CHILD)
Local $consoleContent

For $i = 1 To 5
    $consoleContent &= StdoutRead($consoleOutputHandle, true, false)
    Sleep(500)
Next

FileWrite("test.txt", $consoleContent)

a cmd.exe pops up and stays there. Nothing is printed in it which also brings me to believe the console output is "stored" somewhere. But the "test.txt" file remains empty. 

When removing the $STDERR_CHILD + $STDOUT_CHILD optional flags, my process does start and run as usual in the command window.

I mean this is quite simple code that I'm trying to run there. So there must be something I'm missing. 

.

Any help would be greatly appreciated!

edit : FWIW I may as well add that I'm on a 64 bit version of windows trying to run a 64 bit process. For other automation utilities I know this could be a problem.

Edited by ChristianDB

Share this post


Link to post
Share on other sites
BrewManNH

It is probably because you're only looking at StdOut 5 times and it probably has more information in it than 5 lines. Do you need to interact with this console application, or are you just trying to read information from it?


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
ChristianDB

Thanks for your reply. It does have more information in it than 5 lines, but I would expect StdoutRead to at least get what is stored in the output buffer when I call it, be it 1, 2 or 10 lines? And yes I do need to interact with the application. It's just that I removed this part of the code to concentrate on making stdoutread work. 

Share this post


Link to post
Share on other sites
BrewManNH

You're only giving it 2.5 seconds to read the output, does it take longer than that to run?


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
ChristianDB

The process prints its first lines almost instantly so I figured I didn't have to put long sleep delays before calling StdoutRead. However after reading your post I realized that StdoutRead works only if called after my console application returns and closes. The reason I couldn't see anything in my test.txt file is I let the AutoIt script execute completely (including the write the FileWrite function call) before closing my console application. 

Now is there anyway to capture console output and react consequently before the console application returns?

Share this post


Link to post
Share on other sites
BrewManNH

Somewhere in the depths of the Example Scripts section there's a console UDF floating around.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
Chimp

Hi ChristianDB

trying with this:

#include <Constants.au3>

Local $consoleOutputHandle = Run(@ComSpec & " /c  C:path_to_process\process.exe", @SystemDir, @SW_SHOW, $STDERR_CHILD + $STDOUT_CHILD)
Local $consoleContent

For $i = 1 To 5
    ; $consoleContent &= StdoutRead($consoleOutputHandle, false, false)
    ConsoleWrite(StdoutRead($consoleOutputHandle, False, False))
    Sleep(500)
Next

; FileWrite("test.txt", $consoleContent)

do you see the output in the SciTE console?

Edited by PincoPanco

small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Share this post


Link to post
Share on other sites
ChristianDB

Thank you for the reply. I only see the output in SciTE if the process completes execution and returns before 2.5 seconds (delay until which the AutoIt script returns). To do that I had to blindly press keys because StdoutRead intercepts output and leave the console window empty. My goal is really to capture console output before ,my console app returns, so that I can react accordingly in AutoIt and send the proper keys.

Edited by ChristianDB

Share this post


Link to post
Share on other sites
MHz

Perhaps this example may show you how to interact with a console process. The cmd script prompts some questions and the AutoIt script replies.

#include <Constants.au3>

Global $data, $pid, $testfile

If Not FileExists('test.cmd') Then
    $testfile = 'test.cmd'
    FileWrite($testfile, _
        '@echo off' & @CRLF & _ 
        'echo Please enter the password' & @CRLF & _ 
        'set /p password=' & @CRLF & _ 
        'echo Do you want to archive the results' & @CRLF & _ 
        'set /p archive=' & @CRLF & _ 
        'echo password = %password%' & @CRLF & _ 
        'echo archive = %archive%' & @CRLF & _ 
        'echo Thankyou and have a nice day' _
    )
EndIf

$pid = Run("cmd /c test.cmd", @ScriptDir, @SW_HIDE, $STDIN_CHILD + $STDOUT_CHILD)
While ProcessExists($pid)
    ; clear $data so the next prompt can be new without concern of old data
    $data = ''
    ; test @extended if StdInWrite is open as StdOutRead will not set @error 
    ; test @error if StdInWrite has closed as StdOutRead will set @error
    Do
        Sleep(10)
        $data &= StdOutRead($pid)
        If @error Then ExitLoop 2
    Until Not @extended
    ; check for the prompts and answer with StdInWrite
    Select
        Case StringInStr($data, 'Please enter the password')
            MsgBox(0, "Debug", 'StdOut: ' & $data & @CRLF & 'StdIn: strawberry')
            StdInWrite($pid, 'strawberry')
            ;
        Case StringInStr($data, 'Do you want to archive the results')
            MsgBox(0, "Debug", 'StdOut: ' & $data & @CRLF & 'StdIn: yes')
            StdInWrite($pid, 'yes')
            ; close StdinWrite
            StdInWrite($pid)
    EndSelect
WEnd

; show final message
If $data Then MsgBox(0, "Debug", 'StdOut: ' & $data)

; cleanup
If $testfile Then FileDelete($testfile)

:)

Share this post


Link to post
Share on other sites
Chimp

in MHz's example  is used the /c parameter in the run statement , in this way  the command process will close when the executed program terminates. To read the output of executed program you have to read StdOut untill you get @error. This error indicate that the StdOut can not read more data because the command process do not exist anymore, So you have to interact with the command window in a loop untill @error.

Another way to interact with the command process is to use the /k parameter  in the run command instead of the /c. so that the command prompt remains always active and do not close after execution of your program.

in this way you have all the time to interact with the command window, read what's going on via StdOut and send your answers/commands via StdIn

In the while of the execution of the dos command, your script can do other jobs, and time to time  you check the dos output and interact with it.

>in this post, for an example, there is a test I posted to experiment with.

Edited by PincoPanco

small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Share this post


Link to post
Share on other sites
ChristianDB

Ok so I tried Mhz code after modifying it a bit like so :

; Script Start - Add your code below here

#include <Constants.au3>

Global $data, $pid, $testfile

$pid = Run(@ComSpec & " /k C:\path_to_process\process.exe", "C:\path_to_process", @SW_MAXIMIZE, $STDIN_CHILD + $STDOUT_CHILD)
While ProcessExists($pid)
    ; clear $data so the next prompt can be new without concern of old data
    $data = ''
    ; test @extended if StdInWrite is open as StdOutRead will not set @error 
    ; test @error if StdInWrite has closed as StdOutRead will set @error
    Do
        Sleep(10)
        $data &= StdOutRead($pid)
        If @error Then ExitLoop 2
    Until Not @extended

    ConsoleWrite($data)
    
    FileWrite("test.txt", $data)
WEnd

; show final message
If $data Then MsgBox(0, "Debug", 'StdOut: ' & $data)

As you can see I removed the part of the code responsible for interacting with the app. All I kept is the code responsible for showing what's in the output buffer. Unfortunately I get the same behavior as before. Nothing appears in the SciTE console until the app exits. Then everything appears all at once. 

Share this post


Link to post
Share on other sites
BrewManNH

It's not written to show the information as it's received, it's written to only show it after the app has closed. You'd need to put the ConsoleWrite inside the Do loop.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
ChristianDB

It's not written to show the information as it's received, it's written to only show it after the app has closed. You'd need to put the ConsoleWrite inside the Do loop.

 

This didn't change anything :(.

Share this post


Link to post
Share on other sites
Chimp

what happens with this?

#include <Constants.au3>
Local $Output

; here we start a hidden DOS prompt to be used when needed (it remains alive becouse of /K parameter)
Global $prompt = Run(@ComSpec & " /K", "", @SW_HIDE, $STDIN_CHILD + $STDOUT_CHILD + $STDERR_CHILD + $STDERR_MERGED)
Do ; wait untill prompt is ready
    Sleep(100)
Until StringInStr(StdoutRead($prompt), ">")

; launch of program followed by echo of computername at the end of execution
StdinWrite($prompt, "C:path_to_process\process.exe & echo %computername%" & @CRLF)

Do ; this should output to the consol the program's output untill computername is detected (at the end of execution)
    $Output = StdoutRead($prompt)
    ConsoleWrite($Output)
    If StringInStr($Output, @ComputerName) Then
        ConsoleWrite(@CRLF & "--- Finished ---" & @CRLF)
        ; MsgBox(0, "", 'DETECTED end string ' &@ComputerName&' in dos output', 5)
        ExitLoop
    EndIf
Until 1 > 1
ProcessClose($prompt)
Edited by PincoPanco

small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Share this post


Link to post
Share on other sites
ChristianDB

I get the same behavior. I changed the @SW_HIDE flag to @SW_MAXIMIZE in order to see a windows console to be able to send key inputs and thus make the app complete execution. As with our previous attempts, the console window and the ScITE console remain empty until the process completes execution. 

This is weird. Could there be a particularity in how the console app is built (in the code or something) so that the console output behaves in this way?

Share this post


Link to post
Share on other sites
ChristianDB

Let's bump this once?

Share this post


Link to post
Share on other sites
ChristianDB

Ok here is an update. I've finally made a step forward in this issue. In the console app I want to test/automate, I added a for loop that shoots hundreds of lines of text and I was finally able to see them in the SciTE console! Under a certain number of lines, I can't see anything in the SciTE console until the app exits. It seems that the intercepted strings get stored into a buffer before being readable by StdoutRead. The buffer is probably flushed either when full or when the app exits. Now are you guys aware of any way to get this text as soon as it's sent by the console app? Like forcing a buffer flush much more frequently for example? It really is the part that's missing to complete the puzzle because the app we want to test maybe writes 25 lines of text at most, not enough to fill the buffer.

Share this post


Link to post
Share on other sites
BrewManNH

It would be helpful to see how you've changed the script so that we're not guessing what you did or are doing in it.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
ChristianDB

I'm presently working with PincoPanco's example :

#include <Constants.au3>
Local $Output

; here we start a hidden DOS prompt to be used when needed (it remains alive becouse of /K parameter)
Global $prompt = Run(@ComSpec & " /K", "", @SW_MAXIMIZE, $STDIN_CHILD + $STDOUT_CHILD + $STDERR_CHILD + $STDERR_MERGED)
Do ; wait untill prompt is ready
    Sleep(100)
Until StringInStr(StdoutRead($prompt), ">")

; launch of program followed by echo of computername at the end of execution
StdinWrite($prompt, "path_to_process\process.exe & echo %computername%" & @CRLF)

Do ; this should output to the consol the program's output untill computername is detected (at the end of execution)
    $Output = StdoutRead($prompt)
    ConsoleWrite($Output)
    If StringInStr($Output, @ComputerName) Then
        ConsoleWrite(@CRLF & "--- Finished ---" & @CRLF)
        ; MsgBox(0, "", 'DETECTED end string ' &@ComputerName&' in dos output', 5)
        ExitLoop
    EndIf
Until 1 > 1
ProcessClose($prompt)

But the code I showed in post 11 gives the same result. In both cases I can't get the output of the app before it exits if it sends less than ~75 lines of text.

Share this post


Link to post
Share on other sites
ChristianDB

Problem solved. Or should I say "bypassed". I added the following line in the code of the console app : 

setvbuf(stdout, 0, _IONBF, 0); (http://www.cplusplus.com/reference/cstdio/setvbuf/)

It's just unfortunate that I had to modify the code of the app. 

Maybe some details about stream buffering should be added on the page about StdoutRead in the AutoIt doc?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×