Sign in to follow this  
Followers 0
TechCoder

Recursion Through Entire Drive - stops

8 posts in this topic

#1 ·  Posted (edited)

using code from http://www.autoitscript.com/wiki/Recursion (with minor mods to show progress/errors)

Local $folder = FileSelectFolder("select folder", "")
Local $size = DirGetSize($folder, 1)
ConsoleWrite("size of the directory is " & $size[0] & @CRLF & "file count is " & $size[1] & @CRLF & "directories is " & $size[2] & @CRLF)
Global $counter
ListFiles_Iterative($folder)
Func ListFiles_Iterative($sSourceFolder)

    Local $sFile

    ; Force a trailing \
    If StringRight($sSourceFolder, 1) <> "\" Then $sSourceFolder &= "\"
    ; Create an array to hold the folders to be searched
    Local $aFolderList[10] = [1, $sSourceFolder]

    ; Search within listed folders until all have been searched
    While $aFolderList[0] > 0

        ; Get path of folder to search
        Local $sSearchPath = $aFolderList[$aFolderList[0]]
        ; Remove folder from list
        $aFolderList[0] -= 1

        ; Start the search
        Local $hSearch = FileFindFirstFile($sSearchPath & "*.*")
        ; If failure then return
        If $hSearch = -1 Then Return

        ; Now run through the contents of the folder
        While 1
            ; Get next match
            $sFile = FileFindNextFile($hSearch)
            ; If no more files then close search handle and return
            If @error Then
                ConsoleWrite("Error = " & @error & "Extended = " & @extended & @CRLF)
                ExitLoop
            EndIf
            ; If a folder then add to array to be searched
            If @extended Then

                ; #######################################

                ; Increase folder count
                $aFolderList[0] += 1
                ; Double array size if too small (fewer ReDim needed)
                If UBound($aFolderList) <= $aFolderList[0] Then ReDim $aFolderList[UBound($aFolderList) * 2]
                ; Add folder
                $aFolderList[$aFolderList[0]] = $sSearchPath & $sFile & "\"

                ; #######################################

            Else
                ; If a file then write path and name
                ConsoleWrite("Found: " & $sSearchPath & $sFile & @CRLF)
                $counter += 1
            EndIf
        WEnd
        ConsoleWrite("EXITLOOP Occurred on : " & $sSearchPath & $sFile & @CRLF)
        ; Close search handle
        FileClose($hSearch)
        ConsoleWrite("Value of  $aFolderList[0] is now " & $aFolderList[0] & @CRLF)
    WEnd
    ConsoleWrite("Total count = " & $counter & @CRLF)

EndFunc   ;==>ListFiles_Iterative
results when selecting C:

----------------------------------------------------------------------------------------------------------------
size of the directory is 407299410161
file count is 820104
directories is 102331
Found: C:\bootmgr
Found: C:\BOOTSECT.BAK
Found: C:\dell.sdr
Found: C:\eula.1028.txt
Found: C:\eula.1031.txt
Found: C:\eula.1033.txt
Found: C:\eula.1036.txt
Found: C:\eula.1040.txt
Found: C:\eula.1041.txt
Found: C:\eula.1042.txt
Found: C:\eula.1049.txt
Found: C:\eula.2052.txt
Found: C:\eula.3082.txt
Found: C:\globdata.ini
Found: C:\hiberfil.sys
Found: C:\hpWebHelper.log
Found: C:\install.exe
Found: C:\install.ini
Found: C:\install.res.1028.dll
Found: C:\install.res.1031.dll
Found: C:\install.res.1033.dll
Found: C:\install.res.1036.dll
Found: C:\install.res.1040.dll
Found: C:\install.res.1041.dll
Found: C:\install.res.1042.dll
Found: C:\install.res.1049.dll
Found: C:\install.res.2052.dll
Found: C:\install.res.3082.dll
Found: C:\InstallHelper.log
Found: C:\log.txt
Found: C:\pagefile.sys
Found: C:\pcdr.ini
Found: C:\prefs.js
Found: C:\THE ONE.mov
Found: C:\ut.bat
Found: C:\ut9x.bat
Found: C:\vcredist.bmp
Found: C:\VC_RED.cab
Found: C:\VC_RED.MSI
Found: C:\YServer.txt
Error = 1Extended = 0
EXITLOOP Occurred on : C:\
Value of  $aFolderList[0] is now 40
Found: C:\Word2003RTFSpec\Word2003RTFSpec.doc
Error = 1Extended = 0
EXITLOOP Occurred on : C:\Word2003RTFSpec\
Value of  $aFolderList[0] is now 39
Found: C:\WMdownloads\Thumbs.db
Error = 1Extended = 0
EXITLOOP Occurred on : C:\WMdownloads\
Value of  $aFolderList[0] is now 38
Found: C:\Windows\bfsvc.exe
Found: C:\Windows\bootstat.dat
Found: C:\Windows\Capsule.dll
Found: C:\Windows\comsetup.log
Found: C:\Windows\csup.txt
Found: C:\Windows\DELL_LANGCODE.ini
Found: C:\Windows\DELL_OSTYPE.ini
Found: C:\Windows\DirectX.log
Found: C:\Windows\DPINST.LOG
Found: C:\Windows\DtcInstall.log
Found: C:\Windows\EvtMessage.dll
Found: C:\Windows\explorer.exe
Found: C:\Windows\Factory.xml
Found: C:\Windows\fveupdate.exe
Found: C:\Windows\HelpPane.exe
Found: C:\Windows\hh.exe
Found: C:\Windows\HomeBasic.xml
Found: C:\Windows\HomePremium.xml
Found: C:\Windows\IE10_main.log
Found: C:\Windows\IE11_main.log
Found: C:\Windows\IE9_main.log
Found: C:\Windows\InfoBOM.xml
Found: C:\Windows\LaunApp.exe
Found: C:\Windows\LaunApp.ini
Found: C:\Windows\launApp.log
Found: C:\Windows\Logo.reg
Found: C:\Windows\mib.bin
Found: C:\Windows\msdfmap.ini
Found: C:\Windows\msxml4-KB954430-enu.LOG
Found: C:\Windows\msxml4-KB973688-enu.LOG
Found: C:\Windows\NLSDownlevelMapping.log
Found: C:\Windows\notepad.exe
Found: C:\Windows\PFRO.log
Found: C:\Windows\PLaunch.exe
Found: C:\Windows\PLaunch.log
Found: C:\Windows\Prelaunch.ini
Found: C:\Windows\regedit.exe
Found: C:\Windows\RPSETUP.EXE.LOG
Found: C:\Windows\setLOGO.cmd
Found: C:\Windows\setupact.log
Found: C:\Windows\setuperr.log
Found: C:\Windows\splwow64.exe
Found: C:\Windows\Starter.xml
Found: C:\Windows\system.ini
Found: C:\Windows\TSSysprep.log
Found: C:\Windows\twain.dll
Found: C:\Windows\twain_32.dll
Found: C:\Windows\twunk_16.exe
Found: C:\Windows\twunk_32.exe
Found: C:\Windows\version
Found: C:\Windows\win.ini
Found: C:\Windows\WindowsShell.Manifest
Found: C:\Windows\WindowsUpdate.log
Found: C:\Windows\winhlp32.exe
Found: C:\Windows\WisHWDest.ini
Found: C:\Windows\WisLangCode.ini
Found: C:\Windows\WisPriority.ini
Found: C:\Windows\WisSysInfo.ini
Found: C:\Windows\WLXPGSS.SCR
Found: C:\Windows\WMSysPr9.prx
Found: C:\Windows\write.exe
Found: C:\Windows\XSitePro2 Uninstaller.exe
Found: C:\Windows\XSitePro2.5 Resource Pack Uninstaller.exe
Error = 1Extended = 0
EXITLOOP Occurred on : C:\Windows\
Value of  $aFolderList[0] is now 96
Found: C:\Windows\WisTools\bcdinfo2.txt
Found: C:\Windows\WisTools\DellLastPE.log
Found: C:\Windows\WisTools\DellModule.log
Found: C:\Windows\WisTools\DSL.tag
Found: C:\Windows\WisTools\Eddy.ini
Found: C:\Windows\WisTools\EDocNames.ini
Found: C:\Windows\WisTools\finalP1.txt
Found: C:\Windows\WisTools\Interop.IWshRuntimeLibrary.dll
Found: C:\Windows\WisTools\MakeEdocsShortcut.exe
Found: C:\Windows\WisTools\MakeEdocsShortcut.log
Found: C:\Windows\WisTools\NewEnv.bat
Found: C:\Windows\WisTools\RESULT_LOG.log
Found: C:\Windows\WisTools\RosaCapsule.log
Found: C:\Windows\WisTools\SetModel.cmd
Found: C:\Windows\WisTools\SetModel.reg
Found: C:\Windows\WisTools\temp.txt
Found: C:\Windows\WisTools\WCrPcs64.exe
Found: C:\Windows\WisTools\WisCrPcs.exe
Found: C:\Windows\WisTools\WisINI.exe
Found: C:\Windows\WisTools\WisIni2Bat.exe
Found: C:\Windows\WisTools\WisRun.cmd
Error = 1Extended = 0
EXITLOOP Occurred on : C:\Windows\WisTools\
Value of  $aFolderList[0] is now 106
Found: C:\Windows\WisTools\Tool\SLEEP32.EXE
Found: C:\Windows\WisTools\Tool\SLEEP64.EXE
Found: C:\Windows\WisTools\Tool\WisCrPcs32.exe
Found: C:\Windows\WisTools\Tool\WisCrPcs64.exe
Found: C:\Windows\WisTools\Tool\WisIni2Bat32.exe
Found: C:\Windows\WisTools\Tool\WisIni2Bat64.exe
Found: C:\Windows\WisTools\Tool\WisINI32.exe
Found: C:\Windows\WisTools\Tool\WisINI64.exe
Found: C:\Windows\WisTools\Tool\WisXMLEdit.log
Found: C:\Windows\WisTools\Tool\WisXMLEdit32.exe
Found: C:\Windows\WisTools\Tool\WisXMLEdit64.exe
Error = 1Extended = 0
EXITLOOP Occurred on : C:\Windows\WisTools\Tool\
Value of  $aFolderList[0] is now 105
Error = 1Extended = 0
EXITLOOP Occurred on : C:\Windows\WisTools\Recovery\
Value of  $aFolderList[0] is now 105
+>15:58:28 AutoIt3.exe ended.rc:0
>Exit code: 0    Time: 14.557
 
----------------------------------------------------------------------------------------------------------------
Running Windows 7 Home 

 

This "STOP" occurs on various directories, though not on the largest on the drive - with 30,000+ files.

results from this subdirectory (without all the files...)

size of the directory is 93284469557

file count is 31769

directories is 156

 

I expect it has something to do with MS "protecting you from yourself", however, I can find no way to further break this down to find where it might be stopping the code.

 

Is there another way to recurse through the entire drive?

 

I don't really 'need' the full drive search, personally, though some of my users may take the 'lazy' way when selecting the folder and I want to be ready for an answer - either having it work, or knowing the limitations so I can document (or code) around it.

Edited by Melba23
Added tags

Share this post


Link to post
Share on other sites



#2 ·  Posted (edited)

Try something like this instead...much more simple/easy to follow...add in additional data collection as you require.

#include <File.au3>
RecursiveDir("c:\copy2")
Func RecursiveDir($path)
    $aFiles = _FileListToArray($path,"*",1)
    For $i = 1 To UBound($aFiles)-1
        ConsoleWrite($path & "\" & $aFiles[$i] & @CRLF)
    Next

    $aFolders = _FileListToArray($path,"*",2)
    For $i = 1 To UBound($aFolders)-1
        RecursiveDir($path & "\" & $aFolders[$i])
    Next
EndFunc

Obviously, change the inital path to grab more/fewer dirs.

Edited by jdelaney

IEbyXPATH-Grab IE DOM objects by XPATH IEscriptRecord-Makings of an IE script recorder ExcelFromXML-Create Excel docs without excel installed GetAllWindowControls-Output all control data on a given window.

Share this post


Link to post
Share on other sites

jdelaney,

I wonder sometimes why we bother to add new functions. :whistle:

_FileListToArrayRec does it all for you. ;)

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

Doing a search in my help file revealed no topic, so I did a search online for _FileListToArrayRec

found http://www.autoitscript.com/autoit3/docs/libfunctions/_FileListToArrayRec.htm

#include <File.au3>
#include <Array.au3> ; Only required to display the arrays
#include <MsgBoxConstants.au3>
    ; A sorted list of all files and folders in the AutoIt installation
    Local $aArray = _FileListToArrayRec("c:/", "*", 0, 1, 1)
    ConsoleWrite("Error: " & @error & " - " & " Extended: " & @extended & @CRLF)
    _ArrayDisplay($aArray, "Sorted tree")

results in 

ERROR: can't open include file <MsgBoxConstants.au3>.

did a search on this error, found

'?do=embed' frameborder='0' data-embedContent>>

(a nice bit of code to check versions by @Exit told me I did not have the latest {though I only got AutoIt a couple months ago} - didn't think about a new release - now I have this program I can at least check before asking - would be nice to have that program 'somewhere' - like in the SciTE editor.....)

from that, I found

http://www.autoitscript.com/site/autoit/downloads and got 

http://www.autoitscript.com/cgi-bin/getfile.pl?autoit3/autoit-v3-setup.exe

and upgraded.

ran the program - it works quickly in the AutoIt directory, though choosing C: (as in my initial request) returns 'nothing' (at least, it has been over 10 minutes and no output..........)

#include <File.au3>
#include <Array.au3> ; Only required to display the arrays
#include <MsgBoxConstants.au3>
    Local $sAutoItDir = StringLeft(@AutoItExe, StringInStr(@AutoItExe, "\", Default, -1))
    If StringRight($sAutoItDir, 5) = "beta\" Then
        $sAutoItDir = StringTrimRight($sAutoItDir, 5)
    EndIf
    ConsoleWrite($sAutoItDir & @CRLF)
       ; A sorted list of all files and folders in the AutoIt installation
    Local $aArray = _FileListToArrayRec($sAutoItDir, "*", 0, 1, 1)
    ConsoleWrite("Error: " & @error & " - " & " Extended: " & @extended & @CRLF)
    _ArrayDisplay($aArray, "Sorted tree")

; use exactly the 'working' code from above, though allow selection of the folder
    Local $folder = FileSelectFolder("select folder","")
    $aArray = _FileListToArrayRec($folder, "*", 0, 1, 1)
    ConsoleWrite("Error: " & @error & " - " & " Extended: " & @extended & @CRLF)
    _ArrayDisplay($aArray, "Sorted tree")

Share this post


Link to post
Share on other sites

спасибо - что функция работает - и очень быстро

Great feature - and super-fast.  It read all the drive in just over 18 seconds.  I will be including this to handle the search I need.  And, it didn't require the upgrade on AutoIt (perhaps good for someone that has a lot of scripts that might need rework before upgrading - this looks to be a good way to go).

огромное спасибо (Russian is my second 'spoken' language - I read it well enough, too, though typing it has never been a practice)

Share this post


Link to post
Share on other sites

super-fast

Note that the first launch builds a buffer access to recent files. The second test of the same folder is always faster.

Acknowledge this difference to test the algorithm, and not the speed of the hard disk.

Do a search for 2 times for every script

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0