StdoutRead not retrieving what i need

[JackDinn]

Ok so im still battling with trying to create a decent upload speed tester. I didn't get any replies here so i slogged on with trying different FTP methods :-/

I have tried using the built in microsoft FTP commands but it just has to many limitations , mainly it cant connect in passive mode but there are other problems with this method to.

So iv moved on to trying a little command line exe from ncFtp (im using ncFTPput.exe).

So far this looks like it mught just do the job. It certainly "max's out" very fast upload connections (can match filezilla's speed) and i believe it will be able to connect in passive mode but im having difficulty retrieving the stdout when it is doing it's upload ? The command window shows the throughput whilst its uploading (which is really handy if i can get it) but using the normal stdoutread does not get the text that is shown in the console window.

this code below should show something ? it shows the error message if you use the wrong password etc. Just it dont show the throughput info i need as in pic above.

$tcrtout = ""
$iPID = Run('ncftpput.exe -p password -u greg -C newlap "C:Program FilesJDASTUPtestfile.bin" test/test.upt', @TempDir, @SW_SHOW, 0x2)
While 1
    GUIGetMsg()
    $tcrtout &= StdoutRead($iPID)
    If @error Then ExitLoop
    If $tcrtout <> "" Then ;$tcrcatch Then
        ConsoleWrite($tcrtout & @LF)
    EndIf
WEnd
Exit

I have looked at Screen_Scrape but its over my head and has a couple of errors like " _WinAPI_MakeDWord(): undefined function." but mainly im not sure i can see how to alter it to work for ncFTPput instead of 7-Zip.

I have attached a copy of ncFTPput.exe just in case anyone would like to help out.

http://jackdinn.co.uk/dump/ncftpput.rar

Cheers.

Edited May 23, 2012 by JackDinn

[lowray1975]

The built-in _FTP_Connect function has support for passive mode. You also might want to look at _FTP_ProgressUpload and (maybe) _FTP_SetStatusCallback for progress information during the transfer.

[JackDinn]

The built-in _FTP_Connect function has support for passive mode.

I didn't say it didn't , i said the built in microsoft ftp command line ftp does not !

Many FTP clients do not support PASV transfers. For example, the standard http://FTP.EXE built into Windows does not support this feature. The user can still do a "quote PASV", which will tell the server to enter PASV mode, but the client still will not work in this mode. (This is a common problem, users enter "quote PASV" but things still don't work).

You also might want to look at _FTP_ProgressUpload

yea _FTP_ProgressUpload was the first thing i started with , some months ago now, it just splits up the file into parts and then uses 'InternetWriteFile' to write each of these "chunks" , i do use this method but people with high upload speeds report that both _FTP_FilePut and 'InternetWriteFile' methods will NOT go above 3 to 4 Mb/s.

i dont know about _FTP_SetStatusCallback but its not the throughput measurement thats a problem is just that i have to first find an FTP method that will go as fast as filezilla (or other client) when tested on fiber connections , so far everyone with a fast upload speed is reporting that the methods you mention are not achieving more than 5Mb/s at the most !

I suspect it something to do with the winapi as when i get them to test with a small code i did using the microsoft ftp commands they all report a full speed upload.

Edited May 23, 2012 by JackDinn

[lowray1975]

I didn't realize there was a throughput limitation on the built-in commands. With that said I decided to investigate your code.

If I had to guess, ncFTPput doesn't write to STDOUT. Change your Run line to this:

$iPID = Run('ncftpput.exe -d stdout -p password -u greg -C newlap "C:Program FilesJDASTUPtestfile.bin" test/test.upt', @TempDir, @SW_SHOW, 0x4)

Keeping the opt_flag set to 0x2 seemed to create an endless loop.

(or I could be dead wrong)

[lowray1975]

I believe this is what you're looking for:

$iPID = Run('ncftpput.exe -v -p password -u greg -C newlap "C:Program FilesJDASTUPtestfile.bin" test/test.upt', @TempDir, @SW_SHOW, 0x2)

Edited May 23, 2012 by lowray1975

[JackDinn]

well your first one did give some more output,

LibNcFTP 3.2.5 (January 17, 2011) compiled for Windows
220: FileZilla Server version 0.9.41 beta
     written by Tim Kosse (Tim.Kosse@gmx.de)
     Please visit http://sourceforge.net/projects/filezilla/
Connected to newlap.
Cmd: USER greg
331: Password required for greg
Cmd: PASS xxxxxxxx
230: Logged on
Cmd: PWD
257: "/" is current directory.
Logged in to newlap as greg.
Cmd: FEAT
211: Features:
      MDTM
      REST STREAM
      SIZE
      MLST type*;size*;modify*;
      MLSD
      UTF8
      CLNT
      MFMT
     End
Cmd: HELP SITE
214: Command SITE is supported by FileZilla Server
Cmd: CLNT NcFTPPut 3.2.5 Windows
200: Don't care
Cmd: TYPE I
200: Type set to I
Cmd: PASV
227: Entering Passive Mode (192,168,0,3,11,55)
Cmd: STOR test/test.upt
150: Connection accepted

just it stops outputting after it makes the connection, there is no throughput info.

The second is just blank.

But thx for the suggestions though

[water]

How about this:

#include <Constants.au3>
Local $iPID = Run(@ComSpec & ' /c ncftpput.exe -p password -u greg -C newlap "C:Program FilesJDASTUPtestfile.bin" test/test.upt', @TempDir, @SW_SHOW, $STDOUT_CHILD)
Local $sLine = ""
While 1
    $sLine &= StdoutRead($iPID)
    If @error Then ExitLoop
WEnd
MsgBox(0, "STDOUT read:", $sLine)

[lowray1975]

Very strange. Here's the code I'm using:

$tcrtout = ""
$iPID = Run("ncftpput.exe -v -p " & $Password & " -u " & $Username & " -C " & $SiteName & " """ & @ScriptDir & "ncftpput.rar"" /subdomains/photos/httpdocs/ncftpput.rar", @TempDir, @SW_SHOW, 0x2)
While 1
    GUIGetMsg()
    $tcrtout &= StdoutRead($iPID, True)
    If @error Then ExitLoop
    If $tcrtout <> "" Then ;$tcrcatch Then
        ConsoleWrite($tcrtout & @LF)
    EndIf
WEnd
Exit

Here's the output:

>Running:(3.3.8.0):C:Program FilesAutoIt3autoit3.exe "hnc-officeusersjcookMy Documents_ProgrammingAutoITTestingftp shitftpdownload.au3"  
...usersjcookMy Documents_ProgrammingAutoITTestingftp shitncftpput.rar:
...estingftp shitncftpput.rar:   ETA:   0:00   32.00/ 95.79 kB  0.00 B/s
...estingftp shitncftpput.rar:   ETA:   0:00   95.79/ 95.79 kB  437.89 kB/s
...estingftp shitncftpput.rar:   ETA:   0:00   95.79/ 95.79 kB  266.54 kB/s
..._ProgrammingAutoITTestingftp shitncftpput.rar:   95.79 kB  266.54 kB/s
..._ProgrammingAutoITTestingftp shitncftpput.rar:   95.79 kB  266.54 kB/s
+>11:14:08 AutoIT3.exe ended.rc:0
>Exit code: 0   Time: 3.320

It's the -v that made the difference.

Edited May 23, 2012 by lowray1975

[JackDinn]

@water

I need the output in a stream while it is uploading , i believe your way would only give the output after the upload had finished (or at least only the once at the end), thx though.

@lowray1975

well you got what im looking for there just wondering why im not getting that , are you on win7 ? im on xp atm, i shall just go over and try it on win7 (although i dont see how that would make a dif)

BTW , I did copy your entire posted code and replace with my info.

But im keeping my fingerers crossed here that were close

[lowray1975]

XP SP3 x32 here!

[water]

@water

I need the output in a stream while it is uploading , i believe your way would only give the output after the upload had finished (or at least only the once at the end), thx though.

I just write the result to a MsgBox when all output lines have been processed. if you do it in the loop you can get the numbers during upload.

[JackDinn]

Yea , were getting something here :-

using lowray1975 last method ;-

...m FilesJDASTUPtestfile.bin: ETA: 0:00 0.03/ 97.66 MB 0.00 B/s

...m FilesJDASTUPtestfile.bin: ETA: 2:25 0.44/ 97.66 MB 688.82 kB/s

...m FilesJDASTUPtestfile.bin: ETA: 2:22 1.13/ 97.66 MB 695.55 kB/s

...m FilesJDASTUPtestfile.bin: ETA: 2:25 1.75/ 97.66 MB 679.63 kB/s

...m FilesJDASTUPtestfile.bin: ETA: 2:23 2.44/ 97.66 MB 683.21 kB/s

...m FilesJDASTUPtestfile.bin: ETA: 2:28 2.97/ 97.66 MB 653.30 kB/s

...m FilesJDASTUPtestfile.bin: ETA: 2:27 3.59/ 97.66 MB 654.34 kB/s

...m FilesJDASTUPtestfile.bin: ETA: 2:25 4.28/ 97.66 MB 657.57 kB/s

only thing to figure now is how to get it in real time, as it was working before its just that it only outputs this info after the end of the upload , but darn close ,, excellent stuff.

p.s. using

$iPID = Run('ncftpput.exe -v -p password -u greg -C newlap "C:Program FilesJDASTUPtestfile.bin" /test/test.upt', @TempDir, @SW_SHOW,0x2)

Edited May 23, 2012 by JackDinn

[lowray1975]

@water

What I found is that the standard method of retrieving STDOUT within AutoIt didn't work unless the -v was present. According to the ncftpput documentation it does write to TTY, which I thought was STDOUT.in the Windows world.

[water]

So your problem is solved now?

[JackDinn]

yea i read about the TTY but had no idea what it was ^^.

But im still wondering how to get the output in realtime as even with the

$tcrtout = StdoutRead($iPID,true)

ConsoleWrite($tcrtout & @LF)

in the loop it still dont show until after the end of the upload ?

[JackDinn]

just seems that $tcrtout is empty until the end of the upload , whereas when you just use the consol window it shows this info "as you go" in real time, so i dont understand how $tcrtout can be empty until the upload has finnished at which point it contains everything from while it was doing the upload ?

if you se what i mean :-/

[water]

Does removing "True" from line

$tcrtout = StdoutRead($iPID,true)

make a difference?

[JackDinn]

Nop , unfortunately it dont, the output is still blank until the upload has finished and then it show all the info

its sooo close, but still so far away , if only i could just get that throughput info as its doing the upload i would get the drinks in all around

[water]

This returns the result of the ping commands while ping is running (Windows 7 64 bit).

#include <Constants.au3>
Local $iPID = Run(@ComSpec & ' /c ping -n 10 localhost', @SystemDir, @SW_SHOW, $STDOUT_CHILD)
Local $sLine = ""
While 1
    $sLine = StdoutRead($iPID)
    If @error Then ExitLoop
    If StringStripWS($sLine, 3) <> "" Then ConsoleWrite($sLine & @CRLF)
WEnd

[lowray1975]

I just tested the help example for the StdoutRead function and it appears that certain outputted data is "groups". I just confused myself.

#include <Constants.au3>

$net = Run(@ComSpec & " /c dir c:temp", @SystemDir, @SW_HIDE, $STDERR_CHILD + $STDOUT_CHILD)

While 1
    $line = StdoutRead($net)
    If @error Then ExitLoop
    If $line <> "" Then ConsoleWrite("STDOUT read:" & $line & " -- " & @Sec & "." & @MSEC & @CRLF)
Wend

Exit

I then changed the directory to c:windows and there were many more "groups" of outputted data.