Progress line display while copying a directory in Command Line

[smartkey]

Hi All,

        I am new to AutoIT. Currently I am trying to write a program to copy a directory(Including sub directories recursively).

       The executable is from command line. Executable File compiled out of DirectCopy.au3 file name is DirectCopy.exe. 

       Copy function is working excellent, based on the arguments provided in the command line followed by DirectCopy.exe. 

        I am trying to get a progress bar/line in cmd window to show while DirectCopy.exe command is running.

       I have tried and browsed many posts to achieve printing progress line on the command line during folder copy.(not progress bar in GUI, this code is i saw in many ways)

      Please help/guide me!!!!!!

Command Line is

C:\>DirectCopy.exe /c c:\temp D:\temp_files

Code is as follows:

Local $temp=0
If UBound($CmdLine) > 1 Then
    if $CmdLine[1] = "/c" Then
        if UBound($CmdLine) > 3 Then
            if DirExist($CmdLine[2]) Then
                ConsoleWrite("Source Directory is Invalid" & @LF)
                ConsoleWrite("Tip: Folder name should not contain Space(s)" & @LF)
            Else
                if DirExist($CmdLine[3]) Then
                    ConsoleWrite("Destination Directory is Invalid" & @LF)
                    ConsoleWrite("Tip: Folder name should not contain Space(s)" & @LF)
                Else
                    $temp=DirCopy($CmdLine[2],$CmdLine[3],1)
                    If $temp = 0 Then
                        ConsoleWriteError("Something went wrong... Error occurred while trying to copy..." & @LF)
                    EndIf
                EndIf
            EndIf
        Else
            ConsoleWrite("Invalid/Insufficient no. of aruguments" & @LF)
        EndIf
    EndIf
EndIf
 

      

[Earthshine]

why not use robocopy built into windows? it's robust copy, very tenacious, let it display to user. you can call that in your script

 

https://www.computerhope.com/robocopy.htm

 

https://ss64.com/nt/robocopy.html

 

I use ROBOCOPY to push my builds to qa and production. ultra reliable and sweet. even with network problems. tenacious. you can customize the logging output to what you want. either way, doing a progress bar on the command line is kind of silly.

Edited February 2, 2018 by Earthshine

[smartkey]

Hi EarthShine,

     Thanks for your help. Further I would require your help in resolving the problem. I have taken the code related to robocopy from the link mentioned below. 

 Further I have modified as below, by adding messages for debugging.

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_Change2CUI=y
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****
;----------------------------------------------------------------------
#include <Constants.au3> ; STDIN & STDOUT
;Global Const $STDIN_CHILD              = 1
;Global Const $STDOUT_CHILD             = 2
;Global Const $STDERR_CHILD             = 4
;Global Const $STDERR_MERGED            = 8
;Global Const $STDIO_INHERIT_PARENT     = 0x10
;Global Const $RUN_CREATE_NEW_CONSOLE   = 0x00010000
;===============================================================================
;
; Function Name:    _robocopy($source, $target, $stitches = "mirror", $excludeFiles = "", $excludeFolders = "", $logFile = "", $statusBox = "", $progressBar = "")
; Description::     Runs the robocopy command
; Parameter(s):     $source     -   Source directory
;                   $target     -   Target Directory
;                   $stitches   -   Standard Robocopy switches, predefined meta commands or combination of both
;                               -   "Meta" or "Std Switches" or [meta][Space][Std switches] (eg mirror /r:0 /w:0)
;                               -   Mirror - Clone source to target (includeing subdirectories)
;                               -   Add - Add new/updates files but don't delete anything
;                               -   Copy - Same as Add
;                               -   CopySub - Same as add but also copies all subdirectories
;                               -   Move - Move files to destination
;                               -   MoveSub - Same as move but also copies all subdirectories
;                   $includeFiles   -   A list of files to include (Comma delimited)
;                                   -   OR An array of files
;                   $excludeFiles   -   A list of files to exclude (Comma delimited)
;                                   -   OR An array of files
;                   $excludeFolders -   A list of folders to exclude (Comma delimited)
;                                   -   OR An array of files
;                   $logFile        -   Filename for log file
;                                       If File starts with + then append to log
;                   $outputFunction -   Function name to pass text to from robocopy
;                                   -   Only full lines will be passed to this function
;                                   -   This will be run by execute command in the form {Function name}($text)
; Requirement(s):  Robocopy is installed on the computer or in the same directoy as script
; Return Value(s): Success: ???
;                   Error: 0 and @error =
;                   = 1 - Robocopy.exe can't be found
;                   = 2 - Source doesn't Exist
;                   = 3 - Destination can't be created (eg maybe read only media)
;                   = 4 - Log File can't be created
;                   = 5 - Robocopy Cant' be run
; Author(s):       John Morrison
;
; Basis:
; Thanks:   PsaltyDS http://www.autoitscript.com/forum/index.php?showtopic=98602&view=findpost&p=720326 _ProcessGetExitCode()
;===============================================================================

_robocopy("C:\Temp","D:\Temp_Files","CopySub", "", "" , "", "Copytest.txt","")
ConsoleWrite("Done")

;Func _robocopy($source, $target, $switches = "mirror", $includeFiles = "", $excludeFiles = "", $excludeFolders = "", $logFile = "", $outputFunction = "")
Func _robocopy($source, $target, $switches, $includeFiles = "", $excludeFiles = "", $excludeFolders = "", $logFile = "Copytest.txt", $outputFunction = "")
    #cs
    ConsoleWrite("All Switches" & @CR & _
            "$source = <" & $source & ">" & @CR & _
            "$target = <" & $target & ">" & @CR & _
            "$switches = <" & $switches & ">" & @CR & _
            "$includeFiles = <" & $includeFiles & ">" & @CR & _
            "$excludeFolders = <" & $excludeFolders & ">" & @CR & _
            "$logFile = <" & $logFile & ">" & @CR & _
            "$outputFunction = <" & $outputFunction & ">" & @CR)
    #ce
ConsoleWrite("Done1")
    ; Check if Robocopy can be executed
    Local $roboCopyCMD = ""
    While $roboCopyCMD = ""
        If FileExists(@SystemDir & "\robocopy.exe") Then
            $roboCopyCMD = @SystemDir & "\robocopy.exe "
            ConsoleWrite("testing1" &@LF)
        ElseIf FileExists(@ScriptDir & "\robocopy.exe") Then
            $roboCopyCMD = @ScriptDir & "\robocopy.exe "
            ConsoleWrite("testing2" &@LF)
        ElseIf FileExists(@ScriptDir & "\getrobocopy.exe") Then
            ;Download Robocopy
            If RunWait(@ScriptDir & "\getrobocopy.exe", @TempDir) <> 0 Then
                ConsoleWrite("testing3" &@LF)
                Return SetError(1, "", 0) ; Can't find Robocopy
            EndIf
        Else
            Return SetError(1, "", 0) ; Can't find Robocopy
        EndIf
    WEnd
ConsoleWrite("Done2") 
    $workingdir = $source
    ; Check source exists
    If StringRight($source, 1) = "\" Then
        ; remove trailing slash \
        $source = StringLeft($source, StringLen($source) - 1)
    EndIf
    If Not FileExists($source) Then
        ; source doesn't exist
        Return SetError(2, "", 0) ; Source doesn't Exist
    EndIf
    $source = _RCProcessFilelist($source)

    ; Check Target reachable
    If StringRight($target, 1) = "\" Then
        ; remove trailing slash \
        $target = StringLeft($target, StringLen($target) - 1)
    EndIf
    If Not FileExists($target) Then
        If Not DirCreate($target) Then
            Return SetError(3, "", 0) ; Destination can't be created (eg maybe read only media)
        EndIf
    EndIf
    $target = _RCProcessFilelist($target)
ConsoleWrite("Done3")
    ;Workout robocopy switches from Meta
    While 1
        ;Loop until all metas are removed
        Select
            ;Case _RCcheckMeta("sub", "/E", $switches)
            Case _RCcheckMeta("mirror", "/MIR ", $switches)
            Case _RCcheckMeta("Add", "", $switches)
            Case _RCcheckMeta("Copy", "", $switches)
            Case _RCcheckMeta("CopySub", "/E", $switches)
            Case _RCcheckMeta("Move", "/MOVE", $switches)
            Case _RCcheckMeta("MoveSub", "/MOVE /E", $switches)
            Case Else
                ;MsgBox(0, "pause", "META EKSE")
                ExitLoop
        EndSelect
        ;MsgBox(0, "pause", "META")
    WEnd

    ;Include files
    If $includeFiles = "" Then $includeFiles = "*.*"
    $switches = _RCProcessFilelist($includeFiles) & " " & StringStripWS($switches, 1)

    ;Exclude files/folders
    If $excludeFiles <> "" Then
        $switches &= " /XF " & _RCProcessFilelist($excludeFiles)
    EndIf
    If $excludeFolders <> "" Then
        ; Add target to 'excluded files' in case tragete is inside source (eg source=c: target=c:\backup)
        $switches &= " /XD " & _RCProcessFilelist($excludeFolders) & " " & _RCProcessFilelist($target)
    EndIf
ConsoleWrite(@LF & "Done4" & $switches & " LOGFILENAME "& $logFile & @LF)
    ; LOG
    If $logFile <> "" Then
        ConsoleWrite("Done4.0 " & $logFile & @LF)
        If StringLeft($logFile, 1) = "+" Then
            $logFile = StringMid($logFile, 2, 255)
            $switches &= ' /log+:"' & $logFile & '"'
        Else
            $switches &= ' /log:"' & $logFile & '"'
        EndIf

        ;$hfile = FileOpen($logFile, 1)
        ;FileChangeDir(@ScriptDir)
        $logFile=@ScriptDir & "\" & $logFile
        ConsoleWrite("Done4.1 " & @LF & $logFile & @LF)
        $hfile = FileOpen($logFile, 2)
        If $hfile = -1 Then
            ConsoleWrite("Done5 " & $switches & @LF)
            Return SetError(4, "", 0) ;Can't append/create Log file error
            ConsoleWrite("Done6 " & $switches & " working " & @LF)
        EndIf
        FileClose($hfile)
    EndIf

#cs
    ;Add extra switches if nessacary
    If Not StringInStr($switches, "/r:") Then
        ; retry count
        $switches &= " /r:0"
    EndIf
    If Not StringInStr($switches, "/w:") Then
        ; Wait time
        $switches &= " /w:0"
    EndIf
    If Not StringInStr($switches, "/ZB") Then
        ; Tries to copy files in restartable mode, but if that fails with an “Access Denied” error, switches automatically to Backup mode.
        ;TODO - MAY drop this out as slows copy ???
        $switches &= " /ZB"
    EndIf
    If Not StringInStr($switches, "/FFT") Then
        ; Assume FAT File Times (2-second granularity). Useful for copying to third-party systems that declare a volume to be NTFS but only implement file times with a 2-second granularity..
        $switches &= " /FFT"
    EndIf
    If Not StringInStr($switches, "/COPY:") Then
        ; Copies Data, Attributes, Timestamps only (SEE
        $switches &= " /COPY:DAT"
    EndIf
    ; Wasn't as useful as it looked-  removed
    ;If Not StringInStr($switches, "/ETA") Then
    ;   ; Shows estimated time of completion for copied files.
    ;   $switches &= " /ETA"
    ;EndIf
    If StringInStr($switches, "/log") Then
        ; Displays output in the console window, in addition to directing it to the log file specified by /LOG or /LOG+.
        ; Needed for status and progress displays
        $switches &= " /TEE"
    EndIf
    ; DEGUB TODO - remove commenting after testing
    ;If $progressBar = "" Then
    ;   ; no progress bar so "suppress the display of progress information"
    ;   $switches &= " /NP "
    ;EndIf

    MsgBox(0, "ROBOCOPY", _
            "$source = <" & $source & ">" & @CR & _
            "$target = <" & $target & ">" & @CR & _
            "$switches = <" & $switches & ">" & @CR & _
            "$roboCopyCMD = <" & $roboCopyCMD & " " & $source & " " & $target & " " & $switches & ">" & @CR & _
            "")
            #ce
    ConsoleWrite($roboCopyCMD & " " & $source & " " & $target & " " & $switches & @CR)

    ;Exit

    ; run robocopy
    Local $robocopyPID = Run('"' & $roboCopyCMD & '" ' & $source & " " & $target & " " & $switches, $workingdir, @SW_HIDE, $STDIN_CHILD + $STDOUT_CHILD)
    If $robocopyPID = 0 Then
        Return SetError(5, "", 0) ; Robocopy failed to run
    Else
        Local $hrobocopy = _ProcessGetHandle($robocopyPID)
        ConsoleWrite("Error Throwing!!!!" & @LF)
        Sleep(5000)
        ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $hrobocopy = ' & $hrobocopy & @crlf & '>Error code: ' & @error & @crlf) ;### Debug Console
        If @error Then MsgBox(0,"Error _ProcessGetHandle", "@error = " & @error)

        Local $cmdOut, $cmdoutlast, $buffer, $loopCount, $buffer2, $sPosit
        ;Local $cmdOutErr = 1 ; so loop is run at least once
        While 1 ; ProcessExists($robocopyPID); Or ($cmdOutErr = 0) ; $cmdOurErr - used to wait untill ALL data is extracted from command line

            $cmdOut = StdoutRead($robocopyPID)
            ;$cmdOutErr = @error
            If @error Then ExitLoop

            If $cmdOut <> $cmdoutlast Then
                $cmdoutlast = $cmdOut
                $fnewdata = True
                $buffer &= $cmdOut ; Stores all data for processing

                ;interpret $BUFFER data
                ;extract and send to $statusFunc() the buffer string up to and including the last @CRLF
                $posit = StringInStr($buffer, @CRLF, 0, -1) + 2
                If Not @error Then
                    $buffer2 = StringStripWS(StringLeft($buffer, $posit), 2)
                    If $buffer2 <> "" Then
                        ;ConsoleWrite($posit & "|" & StringLen($buffer) & "<<" & $buffer2 & ">>" & Asc(StringMid($buffer, $posit, 1)) & "|" & $buffer & "|" & @CR)
                        Execute($outputFunction & '("' & $buffer2 & '")')
                    EndIf
                    $buffer = StringMid($buffer, $posit, 1000)
                    ;ConsoleWrite("<<<" & StringMid($buffer, $posit, 1000) & ">>>" & @CR)

                EndIf
            EndIf ; $cmdOut
            ;endif ; $outputFunction

            Sleep(20)
        WEnd

        Local $iReturnCode = _ProcessGetExitCode($hrobocopy)
        ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $iReturnCode = ' & $iReturnCode & @crlf & '>Error code: ' & @error & @crlf) ;### Debug Console
        If @error Then MsgBox(0,"Error _ProcessGetExitCode", "@error = " & @error)
        #cs
            BITMAP
            Value   Meaning If Set
            16  Serious error. Robocopy did not copy any files. This is either a usage error or an error due to insufficient access privileges on the source or destination directories.
            8   Some files or directories could not be copied (copy errors occurred and the retry limit was exceeded). Check these errors further.
            4   Some Mismatched files or directories were detected. Examine the output log. Housekeeping is probably necessary.
            2   Some Extra files or directories were detected. Examine the output log. Some housekeeping may be needed.
            1   One or more files were copied successfully (that is, new files have arrived).
            0   No errors occurred, and no copying was done. The source and destination directory trees are completely synchronized.
        #ce
        _ProcessCloseHandle($hrobocopy)
        If @error Then MsgBox(0,"Error _ProcessCloseHandle", "@error = " & @error)

        Return $iReturnCode
    EndIf

EndFunc   ;==>_robocopy

Func _RCcheckMeta($meta, $metaSwitch, ByRef $switches)
    ;check if meta is the first part of $switches or whole of switches
    ; if so replace with Robocopy switches for that meta
    $meta = StringLower($meta)
    If StringInStr($switches, $meta & " ") = 1 Or StringLower($switches) = $meta Then
        $switches = StringStripWS(StringStripWS($metaSwitch & " " & StringStripWS(StringMid($switches, StringLen($meta) + 1, 100), 1), 1), 2)
        Return True
    Else
        Return False
    EndIf
EndFunc   ;==>_RCcheckMeta

Func _RCProcessFilelist($fileList)
    ; $includeFiles = either a comma delimited string or Array of strings
    Local $sReturn = ""

    If Not IsArray($fileList) Then
        ;Standard string so convert to array
        $fileList = StringSplit($fileList, ",", 2)
    EndIf

    For $item = 0 To UBound($fileList) - 1
        $fileList[$item] = StringStripWS(StringStripWS($fileList[$item], 1), 2)
        If StringLeft($fileList[$item], 1) <> '"' Then ;Search for inverted commas as first character
            $fileList[$item] = '"' & $fileList[$item]
        EndIf
        If StringRight($fileList[$item], 1) <> '"' Then ;Search for inverted commas as first character
            $fileList[$item] &= '"'
        EndIf
        $sReturn &= " " & $fileList[$item]
    Next
    Return StringStripWS($sReturn, 1) ; Strip leading space
EndFunc   ;==>_RCProcessFilelist

; Return handle of given PID
Func _ProcessGetHandle($iPID)
    Local Const $PROCESS_QUERY_INFORMATION = 0x0400
    Local $avRET = DllCall("kernel32.dll", "ptr", "OpenProcess", "int", $PROCESS_QUERY_INFORMATION, "int", 0, "int", $iPID)
    If @error Then
        Return SetError(1, 0, 0)
    Else
        Return $avRET[0]
    EndIf
EndFunc   ;==>_ProcessGetHandle

; Close process handle
Func _ProcessCloseHandle($hProc)
    Local $avRET = DllCall("kernel32.dll", "int", "CloseHandle", "ptr", $hProc)
    If @error Then
        Return SetError(1, 0, 0)
    Else
        Return 1
    EndIf
EndFunc   ;==>_ProcessCloseHandle

; Get process exit code from handle
Func _ProcessGetExitCode($hProc)
    Local $t_ExitCode = DllStructCreate("int")
    Local $avRET = DllCall("kernel32.dll", "int", "GetExitCodeProcess", "ptr", $hProc, "ptr", DllStructGetPtr($t_ExitCode))
    If @error Then
        Return SetError(1, 0, 0)
    Else
        Return DllStructGetData($t_ExitCode, 1)
    EndIf
EndFunc   ;==>_ProcessGetExitCode

 

Concern Here is: 

On executing the exe file, files and sub folders are getting copied successfully, but I am getting the following error(Plz exclude my debug msgs).

D:\testAutoInstall\23-Jan-2018>fastcopy2.exe
Done1testing1
Done2Done3
Done4"*.*" /E LOGFILENAME Copytest.txt
Done4.0 Copytest.txt
Done4.1
D:\testAutoInstall\23-Jan-2018\Copytest.txt
Error Throwing!!!!4\robocopy.exe  "C:\Temp" "D:\Temp_Files" "*.*" /E /log:"Copytest.txt"
@@ Debug(-1) : $hrobocopy = 0x000002DC
>Error code: 0
@@ Debug(-1) : $iReturnCode = 3
>Error code: 0
Done
D:\testAutoInstall\23-Jan-2018>

 

[Earthshine]

Ok. I’ll test and see. Describe what’s not working

Edited February 6, 2018 by Earthshine

[Earthshine]

Why the process get handle stuff that’s included in auto IT why did you write this stuff

[smartkey]

Hi,

Thanks for helping me. I have also tried executing by commenting that code in else statement where Get handle stuff is written.

By that I understand that this code was written to make the cursor wait till Copy Operation completes and then get back to the command prompt.

Because on commenting the content in Else condition part (i.e., from Local $hrobocopy = _ProcessGetHandle($robocopyPID) line to Return $iReturnCode), the control is returns to command line immediately though the file copy is in progress.

Today, I have modified by adding the below lines in place of the commented lines, so that it will show the progress in dots on DOS console while completing and this is working.

        Local $hrobocopy = ProcessExists("robocopy.exe")
        If $hrobocopy <> 0 Then
            ConsoleWrite("Copy in progress.")
        EndIf
        While $hrobocopy <> 0
            ConsoleWrite(".")
            $hrobocopy = ProcessExists("robocopy.exe")
        WEnd

I could able to achieve by doing this, but two things.

1. Even I did not get the logic as you asked why get handler used in the post where some of the people have certified.  Not sure may be i am not having that depth of understanding. 

2. Keeping above point a side, can we print the progress in % instead of dot. If there is a way can you please guide me.

 

 

fastcopy2.au3