XinYoung

_FileListToArrayRec using a range ???

4 posts in this topic

Hello all! First post here... I've searched existing threads for something that can solve my problem but I've found none.

Description:

First you must know that I have a folder full of .txt files. All of the files have a date somewhere in their name (always formatted YYYY-MM-DD). When the script runs, a GUI asks the user to select a BEGINNING DATE and an END DATE. I want to build an array that gets the files with their two dates and everything in between.

Right now I have...

_FileListToArrayRec($myPath, "*" & GUICtrlRead($begDate) & "*.txt;*" & GUICtrlRead($endDate) & "*.txt", 1)

But, of course, this only gets the files that have the specified dates. How do I include every file in between the two selected dates?

I tried to use _DateDiff to find out how many days are in between the selected dates, so I have a count, but I don't know what to do with it! Am I going about this the wrong way? Any help would be greatly appreciated!

Share this post


Link to post
Share on other sites



#2 ·  Posted (edited)

XinYoung,

Welcome to the AutoIt forums.

You will need to list all the files and then work through the array removing those which do not fall between the 2 dates. let me try and work up an example for you before dinner.

M23

Edit: And just in time as I have thi s moment been called to the table:

#include <Array.au3>
#include <Date.au3>

; simulate reading in the files
Global $aFileList[11] = [10]
For $i = 1 To 10

    $iRandomYear = Random(2000, 2016, 1)
    $iRandomMonth = Random(1, 12, 1)
    $iRandomDay = Random(1, 28, 1) ; I know some months have 29/30/31 days but this is just an example and this way all dates will be valid

    $aFileList[$i] = "BlahBlah" & $iRandomYear & "-" & StringFormat("%02i", $iRandomMonth) & "-" & StringFormat("%02i", $iRandomDay) & "blahblah.txt"

Next
; And here is what we get
_ArrayDisplay($aFileList, "All Files", Default, 8)

; Now set Start and end dates
$iStartdate = "2005-01-01"
$iEndDate = "2009-12-31"

; Now loop through the array (from teh bottom up) and delte any files which do not fall within those dates
For $i = $aFileList[0] To 1 Step -1

    ; Extract the date
    $sFileDate = StringRegExpReplace($aFileList[$i], "^.*(\d{4}-\d{2}-\d{2}).*$", "$1")

    ; Convert date to correct format
    $sFileDate = StringReplace($sFileDate, "-", "/")

    ; Check on or later than start date or on or earlier than end date
    If _DateDiff("D", $iStartdate, $sFileDate) < 0 Or _DateDiff("D", $sFileDate, $iEndDate) < 0 Then
        ; Remove from array
        _ArrayDelete($aFileList, $i)
    EndIf

Next

; And we are left with this
_ArrayDisplay($aFileList, "In-date Files", Default, 8)

Please ask if you have any questions.

M23

Edited by Melba23

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

Hi Melba, thanks for your input.

It works, but since this method arrays every file in the folder first, performance is suffering. The folder it's looking in has hundred and hundreds of text files. I need to array the files that meet the search criteria right from the start.

But your code gave me an idea... check it out.

 

                ;use the dashes in my GUICtrlCreateDate to separate YYYY, MM, and DD.

                $aStartDate = StringSplit(GUICtrlRead($begDate), "-")
                $aEndDate = StringSplit(GUICtrlRead($endDate), "-")

                ;Find the numeric value of the start and end dates, then calculate the difference
                Global $iStartDate = _DateToDayValue($aStartDate[1], $aStartDate[2], $aStartDate[3])
                Global $iEndDate = _DateToDayValue($aEndDate[1], $aEndDate[2], $aEndDate[3])
                Global $idayCount = $iEndDate - $iStartDate

                ;Create a count to find the all dates in between the Start and End dates
                For $i = 0 To $idayCount Step 1
                    Global $Y, $M, $D
                    $StartDate = _DayValueToDate($iStartDate, $Y, $M, $D)
                    $iStartDate += 1
                    ;ConsoleWrite($iStartDate & "(" & $Y & "-" & $M & "-" & $D & ")" & @CRLF)

                    ;Go to the folder and find the files that match the date string
                    Global $aFileList = _FileListToArray( $myPath, "*" & $Y & "-" & $M & "-" & $D & "*.TXT", 0, True)

                Next

 

Thank you!

Share this post


Link to post
Share on other sites

#4 ·  Posted (edited)

XinYoung,

You do realise that this searches through the entire list of files how ever many times there are days between the start and end dates? I fail to see how it can be faster than a single read through the files and then looping through the resulting array. However, if you say it works for you.....

M23

Edit:

Interesting - I created up a folder of 1000 files and ran some different methods against it:

Method 1: What I suggested - read the files once and loop through the array deleing as files are checked

Method 2: The same, but making a single range string of the index values to delete and then doing the deletion in one go

Method 3: Your suggested way of searching for each day in turn

#include <Date.au3>
#include <File.au3>
#include <Array.au3>
#include <FileConstants.au3>

; Set Start and end dates
$iStartdate = "2005-01-01"
$iEndDate = "2009-12-31"

; Method 1 - Read all files and loop through deleting as we go #################################################################################################

$nBegin = TimerInit()

; List Files
$aFileList = _FileListToArray(@ScriptDir & "\test", "*.txt", $FLTA_FILES, True)

;ConsoleWrite("Files Read: " & TimerDiff($nBegin) & @CRLF)

; Now loop through the array (from the bottom up) and delte any files which do not fall within those dates
For $i = $aFileList[0] To 1 Step -1

    ; Extract the date
    $sFileDate = StringRegExpReplace($aFileList[$i], "^.*(\d{4}-\d{2}-\d{2}).*$", "$1")

    ; Convert date to correct format
    $sFileDate = StringReplace($sFileDate, "-", "/")

    ; Check on or later than start date or on or earlier than end date
    If _DateDiff("D", $iStartdate, $sFileDate) < 0 Or _DateDiff("D", $sFileDate, $iEndDate) < 0 Then
        ; Remove from array
        _ArrayDelete($aFileList, $i)
        ; Lower count
        $aFileList[0] -= 1
    EndIf

Next

ConsoleWrite("Files checked: " & TimerDiff($nBegin) & @CRLF)

; And we are left with this
_ArrayDisplay($aFileList, "In-date Files 1", Default, 8)

; Method 2 - Read all files and loop through but only deleting once at the end #################################################################################

$nBegin = TimerInit()

; List Files
$aFileList = _FileListToArray(@ScriptDir & "\test", "*.txt", $FLTA_FILES, True)

;ConsoleWrite("Files Read: " & TimerDiff($nBegin) & @CRLF)

$sIndexToDelete = ""

; Now loop through the array and list the index number of any files which do not fall within those dates
For $i = 1 To $aFileList[0]

    ; Extract the date
    $sFileDate = StringRegExpReplace($aFileList[$i], "^.*(\d{4}-\d{2}-\d{2}).*$", "$1")

    ; Convert date to correct format
    $sFileDate = StringReplace($sFileDate, "-", "/")

    ; Check on or later than start date or on or earlier than end date
    If _DateDiff("D", $iStartdate, $sFileDate) < 0 Or _DateDiff("D", $sFileDate, $iEndDate) < 0 Then
        ; Add to list
        $sIndexToDelete &= $i & ";"
        ; Lower count
        $aFileList[0] -= 1
    EndIf

Next

; Remove final ;
$sIndexToDelete = StringTrimRight($sIndexToDelete, 1)

;ConsoleWrite($sIndexToDelete & @CRLF)

; Now delete the files
_ArrayDelete($aFileList, $sIndexToDelete)

ConsoleWrite("Files checked: " & TimerDiff($nBegin) & @CRLF)

; And we are left with this
_ArrayDisplay($aFileList, "In-date Files 2", Default, 8)

; Method 3 - Search for each date #######################################################################################################

Global $aFileList[1]= [0]

$aStartDate = StringSplit($iStartdate, "-")
$aEndDate = StringSplit($iEndDate, "-")

;Find the numeric value of the start and end dates, then calculate the difference
 Global $iStartDate = _DateToDayValue($aStartDate[1], $aStartDate[2], $aStartDate[3])
 Global $iEndDate = _DateToDayValue($aEndDate[1], $aEndDate[2], $aEndDate[3])
 Global $iDayCount = $iEndDate - $iStartDate
 Global $Y, $M, $D

$nBegin = TimerInit()

;Create a count to find the all dates in between the Start and End dates
 For $i = 0 To $iDayCount Step 1

    $StartDate = _DayValueToDate($iStartDate, $Y, $M, $D)
    $iStartDate += 1

    ;Go to the folder and find the files that match the date string
    $aDayFileList = _FileListToArray(@ScriptDir & "\test", "*" & $Y & "-" & $M & "-" & $D & "*.TXT", $FLTA_FILES, True)

    If Not @error Then

        _ArrayConcatenate($aFileList, $aDayFileList, 1)
        $aFileList[0] += (UBound($aDayFileList) - 1)

    EndIf

Next

ConsoleWrite("Array filled: " & TimerDiff($nBegin) & @CRLF)

_ArrayDisplay($aFileList, "In-date files 3", Default, 8)

The results:

Files checked: 5404.87346516393
Files checked: 1899.13545458063
Array filled: 5315.51370544365

Just goes to show how slow the ReDim function is! Deleting from the array as you go or concatenating the arrays for each date uses a lot of ReDim calls - best to use just the one as in method 2.

M23

Edited by Melba23

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now