Jump to content

Retrieve file list based on file modified time


Recommended Posts

Hi,

I am trying to get a list of files from just one directory but have it sorted based on the Modified Time. The reason for this is that the files have names based on dates but the month portion is three characters so they look like

2020May13 

2020May14

The problem is that the default sorting by windows is by name so

2020Dec01 comes before 2020Mar01

The files are created on the date/time in the file name so the Modified Time is in the correct order. If I use Windows Explorer to display the files and click on the Modified time in the column header the files are sorted based on Modified Time however the _FileListToArray gets the files in a different order - I am not sure what order that is because it isn't just alphabetical.

Anybody ever needed this or am I going to have to write a UDF to try to sort them. BTW I have changed file names like this to change the months to digits but I don't want to do that in this instance.

Thanks

George

Link to post
Share on other sites
24 minutes ago, GeorgeP said:

Anybody ever needed this

Yes.

Edited by mLipok

Signature beginning:
Please remember: "AutoIt"..... *  Wondering who uses AutoIt and what it can be used for ? * Forum Rules *
ADO.au3 UDF * POP3.au3 UDF * XML.au3 UDF * for other useful stuff click the following button:

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST APIErrorLog.au3 UDF - A logging Library * Include Dependency Tree (Tool for analyzing script relations) *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 UDF * ADO.au3 UDF SMTP Mailer UDF * Dual Monitor resolution detection * * 2GUI on Dual Monitor System * _SciLexer.au3 UDF * SciTE - Lexer for console pane

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Good coding practices in AutoIt * 

OpenOffice/LibreOffice/XLS Related: WriterDemo.au3 * XLS/MDB from scratch with ADOX

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * IE in TaskSchedulerIE Embedded Control Versioning (use IE9+ and HTML5 in a GUI) * PDF Related:How to get reference to PDF object embeded in IE *

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

I also encourage you to check awesome @trancexx code:  * Create COM objects from modules without any demand on user to register anything. * Another COM object registering stuffOnHungApp handlerAvoid "AutoIt Error" message box in unknown errors  * HTML editor

winhttp.au3 related : * https://www.autoitscript.com/forum/topic/206771-winhttpau3-download-problem-youre-speaking-plain-http-to-an-ssl-enabled-server-port/

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

Signature last update: 2021-10-18

Link to post
Share on other sites

I might do something like this:

1) Retrieve file listing in Array
2) Append new column
3) Enumerate array and collect modified times
4) Sort array by modified times column
5) Remove modified times column from array

#include <Array.au3>
#include <File.au3>

$sSourceDir = @ScriptDir

;Collect file list
Local $aFileList = _FileListToArray($sSourceDir)
;Get file list count
Local $iCount = $aFileList[0]
;Remove file list count from array
_ArrayDelete($aFileList,0)
;Insert nw column to hold modified times
_ArrayColInsert($aFileList,1)
;Enumerate modified time
For $iFile = 0 to $iCount -1
    $aFileList[$iFile][1] = FileGetTime($sSourceDir & "\" & $aFileList[$iFile][0],$FT_MODIFIED,$FT_STRING)
Next
;Sort array by modified times
_ArraySort($aFileList,0,0,0,1)
;Remove modified times column from array
_ArrayColDelete($aFileList,1)


_ArrayDisplay($aFileList)

 

Of course, you could write a function to parse the folder names and convert them to a date, which you could then sort.  But if the modified times are accurate and should always be, this (above) might work.

Link to post
Share on other sites

Personally, I would rake through the array, changing the month string to it's numeric equivalent. Then sort the array and then rake through it again and change the month string back to its original format. Ie. 2020May13 becomes 202000513 then later becomes 2020May13 again.

As I was typing this @spudw2kposted a workable suggestion. Either way it's a programmatic solution needing to loop through the array.

Phil Seakins

Link to post
Share on other sites

Still using the quick Dir command for that kind of thing. Just change the path at your wish (variable $sPath)

#include <Array.au3>
#include <StringConstants.au3>

Local $sPath = "C:\"
Local $sCommand = " /c dir /a-d /b /od " ; a-d (no folders), b (file names), od (sort by date)
Local $iPid = Run(@ComSpec & $sCommand, $sPath, @SW_HIDE, $STDOUT_CHILD)
ProcessWaitClose($iPid)

Local $sList = StdoutRead($iPid)
$sList = StringTrimRight($sList, 2) ; delete last @CRLF
Local $aList = StringSplit($sList, @CRLF, BitOr($STR_ENTIRESPLIT, $STR_NOCOUNT))
_ArrayDisplay($aList, "Sorted by date")

 

Link to post
Share on other sites
8 hours ago, spudw2k said:

1) Retrieve file listing in Array

1. A small note (you probably just forgot ;) ) :
Use

#include <FileConstants.au3>
[...]
;Collect file list
Local $aFileList = _FileListToArray($sSourceDir, default, $FLTA_FILES)

instead of

;Collect file list
Local $aFileList = _FileListToArray($sSourceDir)

Otherwise the folders will also be listed.

2. Regarding LastModified date, I would like to quote a post  written by @seadoggie01  :

Quote

I will never ever, ever, ever, [...], ever believe the LastModified date. I have tried (and failed) to create a backup program that relied heavily on the last modified date, just to find out that Windows is terrible about updating it occasionally. I don't remember why, but I have a vague bitter feeling of distrust around that property now.

(I agree with that !)

As far as I understand it, the date within the file name is the controlling element. The problem is just the sorting, since the month name is given as a shortened string. I would therefore modify @spudw2k approach a bit. Instead of a date, replace the month part in the original filename with a numeric value (Jan=01 ... Dec=12) and insert it into the additional column.

The suggestions of @pseakins and @pixelsearch would also be a solution, of course :).

Edited by Musashi

Musashi-C64.png

"In the beginning the Universe was created. This has made a lot of people very angry and been widely regarded as a bad move."

Link to post
Share on other sites

@GeorgeP
One of many ways:

#include <Array.au3>
#include <Date.au3>
#include <File.au3>

Test()


Func Test()

    Local $strFilePath = @ScriptDir & "\TestDirectories\", _
          $arrPathList, _
          $strMonthName

    $arrPathList = _FileListToArray($strFilePath, "*", $FLTA_FOLDERS)
    If @error Then Return ConsoleWrite("_FileListToArray ERR: " & @error & @CRLF)

    For $i = 1 To $arrPathList[0] Step 1
        $strMonthName = StringRegExp($arrPathList[$i], '(?i)\d{4}([a-z]+)\d{2}', $STR_REGEXPARRAYMATCH)[0]
        $arrPathList[$i] = StringRegExpReplace($arrPathList[$i], '(?i)(\d{4})([a-z]+)(\d{2})', '${1}' & _MonthNameToNumber($strMonthName) & '${3}')
    Next

    _ArraySort($arrPathList)
    _ArrayDisplay($arrPathList)

EndFunc

Func _MonthNameToNumber($strMonthName)

    For $i = 1 To 12 Step 1
        If StringRegExp($strMonthName, '(?i)' & _DateToMonth($i, $DMW_LOCALE_SHORTNAME), $STR_REGEXPMATCH) Then Return ($i > 9 ? $i : '0' & $i)
    Next

EndFunc

:)

Edited by FrancescoDiMuro

Click here to see my signature:

Spoiler

ALWAYS GOOD TO READ:

 

Link to post
Share on other sites
13 hours ago, GeorgeP said:

so the Modified Time is in the correct order

Nooooooooooooooooooooooooooooooooooooooo! Run!

Seriously though, what happens when you suddenly need to change a file from last year because you made a mistake? Use the file name and convert it to sort your files :)

All my code provided is Public Domain... but it may not work. ;) Use it, change it, break it, whatever you want.

Spoiler

My Humble Contributions:
Personal Function Documentation - A personal HelpFile for your functions
Acro.au3 UDF - Automating Acrobat Pro
ToDo Finder - Find #ToDo: lines in your scripts

Link to post
Share on other sites
1 hour ago, seadoggie01 said:

Seriously though, what happens when you suddenly need to change a file from last year because you made a mistake?

Are you really serious, though?  
Unless I’m missing something, the OP said nothing about editing these files or making mistakes, or which order would be preferred if remodified, or even what the consequences would be if the order was inaccurate.

I rely on last modified date in many scripts, but have yet to notice a problem.

I did have a problem trying to use last accessed date, much like you said it was often not updated timely.  

Code hard, but don’t hard code...

Link to post
Share on other sites

Bad wording there, I meant more like, "I'll stop being funny now, [...]" and less that it was a grave matter.

Correct, OP didn't mention editing the files, but I like to look beyond the stated requirements to help avoid potential issues down the road. Squash any potential issues before they arise (provided they're simple enough). I'm saying, similar to us suggesting ControlClick instead of MouseClick, use the file name instead of modified date. Lastly, the comment might not apply to OP, but might apply to anyone reading this later :)

I can't speak to the specifics of my hatred of last modified date anymore as it stems from a project from 5+ years ago, but a general feeling of distrust still lingers 

All my code provided is Public Domain... but it may not work. ;) Use it, change it, break it, whatever you want.

Spoiler

My Humble Contributions:
Personal Function Documentation - A personal HelpFile for your functions
Acro.au3 UDF - Automating Acrobat Pro
ToDo Finder - Find #ToDo: lines in your scripts

Link to post
Share on other sites
7 hours ago, seadoggie01 said:

but I like to look beyond the stated requirements to help avoid potential issues down the road.

ok, fair enough.

7 hours ago, seadoggie01 said:

can't speak to the specifics of my hatred of last modified date anymore as it stems from a project from 5+ years ago, but a general feeling of distrust still lingers 

totally understand.  ngl, I also have accumulated many biases based on good or bad experiences, many without a rigorous examination.

in this particular case, I’m wondering if your experience was related to last modified date of folders, or just files like the OP.

there are many reports of frustration over the apparent lack of update to the LMD when a file is changed within a folder. however a folder LMD is updated only when it’s contents (meaning the list of files it contains), is altered.

 

Code hard, but don’t hard code...

Link to post
Share on other sites

I'm pretty sure it wasn't folders giving me an issue, I looped through each file to check if it had been modified and copied if it had. Early on, I certainly had the idea to check the folder's LMD, but found that completely failed. When running the LMD comparison on files, I pretty consistently received both false positives and negatives iirc. Later, I found that the LMD is also based on each device's internal time, crippling the effectiveness of the program when backing up to a network drive. (In my case, our router never had the time set and the LMDs were off by a few years). I'm still not entirely sure how a backup program could be implemented better without hashing everything.

All my code provided is Public Domain... but it may not work. ;) Use it, change it, break it, whatever you want.

Spoiler

My Humble Contributions:
Personal Function Documentation - A personal HelpFile for your functions
Acro.au3 UDF - Automating Acrobat Pro
ToDo Finder - Find #ToDo: lines in your scripts

Link to post
Share on other sites

Hi All,

I would like to thank everyone who has contributed to this topic, it has enlightened me in several ways - alternative ways to achieve similar things that I didn't think of.

I had toyed with the idea to get the file names into an array and rename the month portion to digits, re-sort the array and then change the digits back to alphas but thought there may be other ways. Thought it would take a while to code but turned out very simple. I was also working with over 51,000 files so I thought it would take a decent amount of time to process but turned out to be just a few seconds.

So good to have such great people offering advice.

Regards

George

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...