Recursion Through Entire Drive - stops

[TechCoder]

using code from http://www.autoitscript.com/wiki/Recursion (with minor mods to show progress/errors)

expandcollapse popup

Local $folder = FileSelectFolder("select folder", "")
Local $size = DirGetSize($folder, 1)
ConsoleWrite("size of the directory is " & $size[0] & @CRLF & "file count is " & $size[1] & @CRLF & "directories is " & $size[2] & @CRLF)
Global $counter
ListFiles_Iterative($folder)
Func ListFiles_Iterative($sSourceFolder)

    Local $sFile

    ; Force a trailing \
    If StringRight($sSourceFolder, 1) <> "\" Then $sSourceFolder &= "\"
    ; Create an array to hold the folders to be searched
    Local $aFolderList[10] = [1, $sSourceFolder]

    ; Search within listed folders until all have been searched
    While $aFolderList[0] > 0

        ; Get path of folder to search
        Local $sSearchPath = $aFolderList[$aFolderList[0]]
        ; Remove folder from list
        $aFolderList[0] -= 1

        ; Start the search
        Local $hSearch = FileFindFirstFile($sSearchPath & "*.*")
        ; If failure then return
        If $hSearch = -1 Then Return

        ; Now run through the contents of the folder
        While 1
            ; Get next match
            $sFile = FileFindNextFile($hSearch)
            ; If no more files then close search handle and return
            If @error Then
                ConsoleWrite("Error = " & @error & "Extended = " & @extended & @CRLF)
                ExitLoop
            EndIf
            ; If a folder then add to array to be searched
            If @extended Then

                ; #######################################

                ; Increase folder count
                $aFolderList[0] += 1
                ; Double array size if too small (fewer ReDim needed)
                If UBound($aFolderList) <= $aFolderList[0] Then ReDim $aFolderList[UBound($aFolderList) * 2]
                ; Add folder
                $aFolderList[$aFolderList[0]] = $sSearchPath & $sFile & "\"

                ; #######################################

            Else
                ; If a file then write path and name
                ConsoleWrite("Found: " & $sSearchPath & $sFile & @CRLF)
                $counter += 1
            EndIf
        WEnd
        ConsoleWrite("EXITLOOP Occurred on : " & $sSearchPath & $sFile & @CRLF)
        ; Close search handle
        FileClose($hSearch)
        ConsoleWrite("Value of  $aFolderList[0] is now " & $aFolderList[0] & @CRLF)
    WEnd
    ConsoleWrite("Total count = " & $counter & @CRLF)

EndFunc   ;==>ListFiles_Iterative

results when selecting C:

expandcollapse popup

----------------------------------------------------------------------------------------------------------------
size of the directory is 407299410161
file count is 820104
directories is 102331
Found: C:\bootmgr
Found: C:\BOOTSECT.BAK
Found: C:\dell.sdr
Found: C:\eula.1028.txt
Found: C:\eula.1031.txt
Found: C:\eula.1033.txt
Found: C:\eula.1036.txt
Found: C:\eula.1040.txt
Found: C:\eula.1041.txt
Found: C:\eula.1042.txt
Found: C:\eula.1049.txt
Found: C:\eula.2052.txt
Found: C:\eula.3082.txt
Found: C:\globdata.ini
Found: C:\hiberfil.sys
Found: C:\hpWebHelper.log
Found: C:\install.exe
Found: C:\install.ini
Found: C:\install.res.1028.dll
Found: C:\install.res.1031.dll
Found: C:\install.res.1033.dll
Found: C:\install.res.1036.dll
Found: C:\install.res.1040.dll
Found: C:\install.res.1041.dll
Found: C:\install.res.1042.dll
Found: C:\install.res.1049.dll
Found: C:\install.res.2052.dll
Found: C:\install.res.3082.dll
Found: C:\InstallHelper.log
Found: C:\log.txt
Found: C:\pagefile.sys
Found: C:\pcdr.ini
Found: C:\prefs.js
Found: C:\THE ONE.mov
Found: C:\ut.bat
Found: C:\ut9x.bat
Found: C:\vcredist.bmp
Found: C:\VC_RED.cab
Found: C:\VC_RED.MSI
Found: C:\YServer.txt
Error = 1Extended = 0
EXITLOOP Occurred on : C:\
Value of  $aFolderList[0] is now 40
Found: C:\Word2003RTFSpec\Word2003RTFSpec.doc
Error = 1Extended = 0
EXITLOOP Occurred on : C:\Word2003RTFSpec\
Value of  $aFolderList[0] is now 39
Found: C:\WMdownloads\Thumbs.db
Error = 1Extended = 0
EXITLOOP Occurred on : C:\WMdownloads\
Value of  $aFolderList[0] is now 38
Found: C:\Windows\bfsvc.exe
Found: C:\Windows\bootstat.dat
Found: C:\Windows\Capsule.dll
Found: C:\Windows\comsetup.log
Found: C:\Windows\csup.txt
Found: C:\Windows\DELL_LANGCODE.ini
Found: C:\Windows\DELL_OSTYPE.ini
Found: C:\Windows\DirectX.log
Found: C:\Windows\DPINST.LOG
Found: C:\Windows\DtcInstall.log
Found: C:\Windows\EvtMessage.dll
Found: C:\Windows\explorer.exe
Found: C:\Windows\Factory.xml
Found: C:\Windows\fveupdate.exe
Found: C:\Windows\HelpPane.exe
Found: C:\Windows\hh.exe
Found: C:\Windows\HomeBasic.xml
Found: C:\Windows\HomePremium.xml
Found: C:\Windows\IE10_main.log
Found: C:\Windows\IE11_main.log
Found: C:\Windows\IE9_main.log
Found: C:\Windows\InfoBOM.xml
Found: C:\Windows\LaunApp.exe
Found: C:\Windows\LaunApp.ini
Found: C:\Windows\launApp.log
Found: C:\Windows\Logo.reg
Found: C:\Windows\mib.bin
Found: C:\Windows\msdfmap.ini
Found: C:\Windows\msxml4-KB954430-enu.LOG
Found: C:\Windows\msxml4-KB973688-enu.LOG
Found: C:\Windows\NLSDownlevelMapping.log
Found: C:\Windows\notepad.exe
Found: C:\Windows\PFRO.log
Found: C:\Windows\PLaunch.exe
Found: C:\Windows\PLaunch.log
Found: C:\Windows\Prelaunch.ini
Found: C:\Windows\regedit.exe
Found: C:\Windows\RPSETUP.EXE.LOG
Found: C:\Windows\setLOGO.cmd
Found: C:\Windows\setupact.log
Found: C:\Windows\setuperr.log
Found: C:\Windows\splwow64.exe
Found: C:\Windows\Starter.xml
Found: C:\Windows\system.ini
Found: C:\Windows\TSSysprep.log
Found: C:\Windows\twain.dll
Found: C:\Windows\twain_32.dll
Found: C:\Windows\twunk_16.exe
Found: C:\Windows\twunk_32.exe
Found: C:\Windows\version
Found: C:\Windows\win.ini
Found: C:\Windows\WindowsShell.Manifest
Found: C:\Windows\WindowsUpdate.log
Found: C:\Windows\winhlp32.exe
Found: C:\Windows\WisHWDest.ini
Found: C:\Windows\WisLangCode.ini
Found: C:\Windows\WisPriority.ini
Found: C:\Windows\WisSysInfo.ini
Found: C:\Windows\WLXPGSS.SCR
Found: C:\Windows\WMSysPr9.prx
Found: C:\Windows\write.exe
Found: C:\Windows\XSitePro2 Uninstaller.exe
Found: C:\Windows\XSitePro2.5 Resource Pack Uninstaller.exe
Error = 1Extended = 0
EXITLOOP Occurred on : C:\Windows\
Value of  $aFolderList[0] is now 96
Found: C:\Windows\WisTools\bcdinfo2.txt
Found: C:\Windows\WisTools\DellLastPE.log
Found: C:\Windows\WisTools\DellModule.log
Found: C:\Windows\WisTools\DSL.tag
Found: C:\Windows\WisTools\Eddy.ini
Found: C:\Windows\WisTools\EDocNames.ini
Found: C:\Windows\WisTools\finalP1.txt
Found: C:\Windows\WisTools\Interop.IWshRuntimeLibrary.dll
Found: C:\Windows\WisTools\MakeEdocsShortcut.exe
Found: C:\Windows\WisTools\MakeEdocsShortcut.log
Found: C:\Windows\WisTools\NewEnv.bat
Found: C:\Windows\WisTools\RESULT_LOG.log
Found: C:\Windows\WisTools\RosaCapsule.log
Found: C:\Windows\WisTools\SetModel.cmd
Found: C:\Windows\WisTools\SetModel.reg
Found: C:\Windows\WisTools\temp.txt
Found: C:\Windows\WisTools\WCrPcs64.exe
Found: C:\Windows\WisTools\WisCrPcs.exe
Found: C:\Windows\WisTools\WisINI.exe
Found: C:\Windows\WisTools\WisIni2Bat.exe
Found: C:\Windows\WisTools\WisRun.cmd
Error = 1Extended = 0
EXITLOOP Occurred on : C:\Windows\WisTools\
Value of  $aFolderList[0] is now 106
Found: C:\Windows\WisTools\Tool\SLEEP32.EXE
Found: C:\Windows\WisTools\Tool\SLEEP64.EXE
Found: C:\Windows\WisTools\Tool\WisCrPcs32.exe
Found: C:\Windows\WisTools\Tool\WisCrPcs64.exe
Found: C:\Windows\WisTools\Tool\WisIni2Bat32.exe
Found: C:\Windows\WisTools\Tool\WisIni2Bat64.exe
Found: C:\Windows\WisTools\Tool\WisINI32.exe
Found: C:\Windows\WisTools\Tool\WisINI64.exe
Found: C:\Windows\WisTools\Tool\WisXMLEdit.log
Found: C:\Windows\WisTools\Tool\WisXMLEdit32.exe
Found: C:\Windows\WisTools\Tool\WisXMLEdit64.exe
Error = 1Extended = 0
EXITLOOP Occurred on : C:\Windows\WisTools\Tool\
Value of  $aFolderList[0] is now 105
Error = 1Extended = 0
EXITLOOP Occurred on : C:\Windows\WisTools\Recovery\
Value of  $aFolderList[0] is now 105
+>15:58:28 AutoIt3.exe ended.rc:0
>Exit code: 0    Time: 14.557
 
----------------------------------------------------------------------------------------------------------------

Running Windows 7 Home 

 

This "STOP" occurs on various directories, though not on the largest on the drive - with 30,000+ files.

results from this subdirectory (without all the files...)

size of the directory is 93284469557

file count is 31769

directories is 156

 

I expect it has something to do with MS "protecting you from yourself", however, I can find no way to further break this down to find where it might be stopping the code.

 

Is there another way to recurse through the entire drive?

 

I don't really 'need' the full drive search, personally, though some of my users may take the 'lazy' way when selecting the folder and I want to be ready for an answer - either having it work, or knowing the limitations so I can document (or code) around it.

Edited January 20, 2014 by Melba23
Added tags

[jdelaney]

Try something like this instead...much more simple/easy to follow...add in additional data collection as you require.

#include <File.au3>
RecursiveDir("c:\copy2")
Func RecursiveDir($path)
    $aFiles = _FileListToArray($path,"*",1)
    For $i = 1 To UBound($aFiles)-1
        ConsoleWrite($path & "\" & $aFiles[$i] & @CRLF)
    Next

    $aFolders = _FileListToArray($path,"*",2)
    For $i = 1 To UBound($aFolders)-1
        RecursiveDir($path & "\" & $aFolders[$i])
    Next
EndFunc

Obviously, change the inital path to grab more/fewer dirs.

Edited January 20, 2014 by jdelaney

[Melba23]

jdelaney,

I wonder sometimes why we bother to add new functions.

_FileListToArrayRec does it all for you.

M23

[jdelaney]

Oops, all ignorant once

[TechCoder]

Doing a search in my help file revealed no topic, so I did a search online for _FileListToArrayRec

found http://www.autoitscript.com/autoit3/docs/libfunctions/_FileListToArrayRec.htm

#include <File.au3>
#include <Array.au3> ; Only required to display the arrays
#include <MsgBoxConstants.au3>
    ; A sorted list of all files and folders in the AutoIt installation
    Local $aArray = _FileListToArrayRec("c:/", "*", 0, 1, 1)
    ConsoleWrite("Error: " & @error & " - " & " Extended: " & @extended & @CRLF)
    _ArrayDisplay($aArray, "Sorted tree")

results in 

ERROR: can't open include file <MsgBoxConstants.au3>.

did a search on this error, found

'?do=embed' frameborder='0' data-embedContent>>

(a nice bit of code to check versions by @Exit told me I did not have the latest {though I only got AutoIt a couple months ago} - didn't think about a new release - now I have this program I can at least check before asking - would be nice to have that program 'somewhere' - like in the SciTE editor.....)

from that, I found

http://www.autoitscript.com/site/autoit/downloads and got 

http://www.autoitscript.com/cgi-bin/getfile.pl?autoit3/autoit-v3-setup.exe

and upgraded.

ran the program - it works quickly in the AutoIt directory, though choosing C: (as in my initial request) returns 'nothing' (at least, it has been over 10 minutes and no output..........)

#include <File.au3>
#include <Array.au3> ; Only required to display the arrays
#include <MsgBoxConstants.au3>
    Local $sAutoItDir = StringLeft(@AutoItExe, StringInStr(@AutoItExe, "\", Default, -1))
    If StringRight($sAutoItDir, 5) = "beta\" Then
        $sAutoItDir = StringTrimRight($sAutoItDir, 5)
    EndIf
    ConsoleWrite($sAutoItDir & @CRLF)
       ; A sorted list of all files and folders in the AutoIt installation
    Local $aArray = _FileListToArrayRec($sAutoItDir, "*", 0, 1, 1)
    ConsoleWrite("Error: " & @error & " - " & " Extended: " & @extended & @CRLF)
    _ArrayDisplay($aArray, "Sorted tree")

; use exactly the 'working' code from above, though allow selection of the folder
    Local $folder = FileSelectFolder("select folder","")
    $aArray = _FileListToArrayRec($folder, "*", 0, 1, 1)
    ConsoleWrite("Error: " & @error & " - " & " Extended: " & @extended & @CRLF)
    _ArrayDisplay($aArray, "Sorted tree")

[AZJIO]

Is there another way to recurse through the entire drive?

 

 

'?do=embed' frameborder='0' data-embedContent>>

[TechCoder]

 

'?do=embed' frameborder='0' data-embedContent>>

спасибо - что функция работает - и очень быстро

Great feature - and super-fast.  It read all the drive in just over 18 seconds.  I will be including this to handle the search I need.  And, it didn't require the upgrade on AutoIt (perhaps good for someone that has a lot of scripts that might need rework before upgrading - this looks to be a good way to go).

огромное спасибо (Russian is my second 'spoken' language - I read it well enough, too, though typing it has never been a practice)

[AZJIO]

super-fast

Note that the first launch builds a buffer access to recent files. The second test of the same folder is always faster.

Acknowledge this difference to test the algorithm, and not the speed of the hard disk.

Do a search for 2 times for every script