XinYoung Posted January 24, 2017 Posted January 24, 2017 Hello all! First post here... I've searched existing threads for something that can solve my problem but I've found none. Description: First you must know that I have a folder full of .txt files. All of the files have a date somewhere in their name (always formatted YYYY-MM-DD). When the script runs, a GUI asks the user to select a BEGINNING DATE and an END DATE. I want to build an array that gets the files with their two dates and everything in between. Right now I have... _FileListToArrayRec($myPath, "*" & GUICtrlRead($begDate) & "*.txt;*" & GUICtrlRead($endDate) & "*.txt", 1) But, of course, this only gets the files that have the specified dates. How do I include every file in between the two selected dates? I tried to use _DateDiff to find out how many days are in between the selected dates, so I have a count, but I don't know what to do with it! Am I going about this the wrong way? Any help would be greatly appreciated!
Moderators Melba23 Posted January 24, 2017 Moderators Posted January 24, 2017 (edited) XinYoung, Welcome to the AutoIt forums. You will need to list all the files and then work through the array removing those which do not fall between the 2 dates. let me try and work up an example for you before dinner. M23 Edit: And just in time as I have thi s moment been called to the table: expandcollapse popup#include <Array.au3> #include <Date.au3> ; simulate reading in the files Global $aFileList[11] = [10] For $i = 1 To 10 $iRandomYear = Random(2000, 2016, 1) $iRandomMonth = Random(1, 12, 1) $iRandomDay = Random(1, 28, 1) ; I know some months have 29/30/31 days but this is just an example and this way all dates will be valid $aFileList[$i] = "BlahBlah" & $iRandomYear & "-" & StringFormat("%02i", $iRandomMonth) & "-" & StringFormat("%02i", $iRandomDay) & "blahblah.txt" Next ; And here is what we get _ArrayDisplay($aFileList, "All Files", Default, 8) ; Now set Start and end dates $iStartdate = "2005-01-01" $iEndDate = "2009-12-31" ; Now loop through the array (from teh bottom up) and delte any files which do not fall within those dates For $i = $aFileList[0] To 1 Step -1 ; Extract the date $sFileDate = StringRegExpReplace($aFileList[$i], "^.*(\d{4}-\d{2}-\d{2}).*$", "$1") ; Convert date to correct format $sFileDate = StringReplace($sFileDate, "-", "/") ; Check on or later than start date or on or earlier than end date If _DateDiff("D", $iStartdate, $sFileDate) < 0 Or _DateDiff("D", $sFileDate, $iEndDate) < 0 Then ; Remove from array _ArrayDelete($aFileList, $i) EndIf Next ; And we are left with this _ArrayDisplay($aFileList, "In-date Files", Default, 8) Please ask if you have any questions. M23 Edited January 24, 2017 by Melba23 Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind Open spoiler to see my UDFs: Spoiler ArrayMultiColSort ---- Sort arrays on multiple columnsChooseFileFolder ---- Single and multiple selections from specified path treeview listingDate_Time_Convert -- Easily convert date/time formats, including the language usedExtMsgBox --------- A highly customisable replacement for MsgBoxGUIExtender -------- Extend and retract multiple sections within a GUIGUIFrame ---------- Subdivide GUIs into many adjustable framesGUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView itemsGUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeViewMarquee ----------- Scrolling tickertape GUIsNoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxesNotify ------------- Small notifications on the edge of the displayScrollbars ----------Automatically sized scrollbars with a single commandStringSize ---------- Automatically size controls to fit textToast -------------- Small GUIs which pop out of the notification area
XinYoung Posted January 25, 2017 Author Posted January 25, 2017 Hi Melba, thanks for your input. It works, but since this method arrays every file in the folder first, performance is suffering. The folder it's looking in has hundred and hundreds of text files. I need to array the files that meet the search criteria right from the start. But your code gave me an idea... check it out. ;use the dashes in my GUICtrlCreateDate to separate YYYY, MM, and DD. $aStartDate = StringSplit(GUICtrlRead($begDate), "-") $aEndDate = StringSplit(GUICtrlRead($endDate), "-") ;Find the numeric value of the start and end dates, then calculate the difference Global $iStartDate = _DateToDayValue($aStartDate[1], $aStartDate[2], $aStartDate[3]) Global $iEndDate = _DateToDayValue($aEndDate[1], $aEndDate[2], $aEndDate[3]) Global $idayCount = $iEndDate - $iStartDate ;Create a count to find the all dates in between the Start and End dates For $i = 0 To $idayCount Step 1 Global $Y, $M, $D $StartDate = _DayValueToDate($iStartDate, $Y, $M, $D) $iStartDate += 1 ;ConsoleWrite($iStartDate & "(" & $Y & "-" & $M & "-" & $D & ")" & @CRLF) ;Go to the folder and find the files that match the date string Global $aFileList = _FileListToArray( $myPath, "*" & $Y & "-" & $M & "-" & $D & "*.TXT", 0, True) Next Thank you!
Moderators Melba23 Posted January 25, 2017 Moderators Posted January 25, 2017 (edited) XinYoung, You do realise that this searches through the entire list of files how ever many times there are days between the start and end dates? I fail to see how it can be faster than a single read through the files and then looping through the resulting array. However, if you say it works for you..... M23 Edit: Interesting - I created up a folder of 1000 files and ran some different methods against it: Method 1: What I suggested - read the files once and loop through the array deleing as files are checked Method 2: The same, but making a single range string of the index values to delete and then doing the deletion in one go Method 3: Your suggested way of searching for each day in turn expandcollapse popup#include <Date.au3> #include <File.au3> #include <Array.au3> #include <FileConstants.au3> ; Set Start and end dates $iStartdate = "2005-01-01" $iEndDate = "2009-12-31" ; Method 1 - Read all files and loop through deleting as we go ################################################################################################# $nBegin = TimerInit() ; List Files $aFileList = _FileListToArray(@ScriptDir & "\test", "*.txt", $FLTA_FILES, True) ;ConsoleWrite("Files Read: " & TimerDiff($nBegin) & @CRLF) ; Now loop through the array (from the bottom up) and delte any files which do not fall within those dates For $i = $aFileList[0] To 1 Step -1 ; Extract the date $sFileDate = StringRegExpReplace($aFileList[$i], "^.*(\d{4}-\d{2}-\d{2}).*$", "$1") ; Convert date to correct format $sFileDate = StringReplace($sFileDate, "-", "/") ; Check on or later than start date or on or earlier than end date If _DateDiff("D", $iStartdate, $sFileDate) < 0 Or _DateDiff("D", $sFileDate, $iEndDate) < 0 Then ; Remove from array _ArrayDelete($aFileList, $i) ; Lower count $aFileList[0] -= 1 EndIf Next ConsoleWrite("Files checked: " & TimerDiff($nBegin) & @CRLF) ; And we are left with this _ArrayDisplay($aFileList, "In-date Files 1", Default, 8) ; Method 2 - Read all files and loop through but only deleting once at the end ################################################################################# $nBegin = TimerInit() ; List Files $aFileList = _FileListToArray(@ScriptDir & "\test", "*.txt", $FLTA_FILES, True) ;ConsoleWrite("Files Read: " & TimerDiff($nBegin) & @CRLF) $sIndexToDelete = "" ; Now loop through the array and list the index number of any files which do not fall within those dates For $i = 1 To $aFileList[0] ; Extract the date $sFileDate = StringRegExpReplace($aFileList[$i], "^.*(\d{4}-\d{2}-\d{2}).*$", "$1") ; Convert date to correct format $sFileDate = StringReplace($sFileDate, "-", "/") ; Check on or later than start date or on or earlier than end date If _DateDiff("D", $iStartdate, $sFileDate) < 0 Or _DateDiff("D", $sFileDate, $iEndDate) < 0 Then ; Add to list $sIndexToDelete &= $i & ";" ; Lower count $aFileList[0] -= 1 EndIf Next ; Remove final ; $sIndexToDelete = StringTrimRight($sIndexToDelete, 1) ;ConsoleWrite($sIndexToDelete & @CRLF) ; Now delete the files _ArrayDelete($aFileList, $sIndexToDelete) ConsoleWrite("Files checked: " & TimerDiff($nBegin) & @CRLF) ; And we are left with this _ArrayDisplay($aFileList, "In-date Files 2", Default, 8) ; Method 3 - Search for each date ####################################################################################################### Global $aFileList[1]= [0] $aStartDate = StringSplit($iStartdate, "-") $aEndDate = StringSplit($iEndDate, "-") ;Find the numeric value of the start and end dates, then calculate the difference Global $iStartDate = _DateToDayValue($aStartDate[1], $aStartDate[2], $aStartDate[3]) Global $iEndDate = _DateToDayValue($aEndDate[1], $aEndDate[2], $aEndDate[3]) Global $iDayCount = $iEndDate - $iStartDate Global $Y, $M, $D $nBegin = TimerInit() ;Create a count to find the all dates in between the Start and End dates For $i = 0 To $iDayCount Step 1 $StartDate = _DayValueToDate($iStartDate, $Y, $M, $D) $iStartDate += 1 ;Go to the folder and find the files that match the date string $aDayFileList = _FileListToArray(@ScriptDir & "\test", "*" & $Y & "-" & $M & "-" & $D & "*.TXT", $FLTA_FILES, True) If Not @error Then _ArrayConcatenate($aFileList, $aDayFileList, 1) $aFileList[0] += (UBound($aDayFileList) - 1) EndIf Next ConsoleWrite("Array filled: " & TimerDiff($nBegin) & @CRLF) _ArrayDisplay($aFileList, "In-date files 3", Default, 8) The results: Files checked: 5404.87346516393 Files checked: 1899.13545458063 Array filled: 5315.51370544365 Just goes to show how slow the ReDim function is! Deleting from the array as you go or concatenating the arrays for each date uses a lot of ReDim calls - best to use just the one as in method 2. M23 Edited January 25, 2017 by Melba23 Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind Open spoiler to see my UDFs: Spoiler ArrayMultiColSort ---- Sort arrays on multiple columnsChooseFileFolder ---- Single and multiple selections from specified path treeview listingDate_Time_Convert -- Easily convert date/time formats, including the language usedExtMsgBox --------- A highly customisable replacement for MsgBoxGUIExtender -------- Extend and retract multiple sections within a GUIGUIFrame ---------- Subdivide GUIs into many adjustable framesGUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView itemsGUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeViewMarquee ----------- Scrolling tickertape GUIsNoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxesNotify ------------- Small notifications on the edge of the displayScrollbars ----------Automatically sized scrollbars with a single commandStringSize ---------- Automatically size controls to fit textToast -------------- Small GUIs which pop out of the notification area
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now