Jump to content
Sign in to follow this  
Ascend4nt

_FileFindEx - Get More from File/Folder Searches

Recommended Posts

_FileFindEx

Get More from File/Folder Searches

(formerly _WinAPI_FileFind)

Since it's always bugged me that the AutoIT implementation of 'FindFirstFile' and 'FindNextFile' only returned filenames and that extra calls had to be made to get file-size, attributes, short-names, and date/time of file creation,last-access, & last-modification which severely increased the amount of time it took to properly analyze the contents of a folder and it's files, I decided to create an alternative.

This uses the same Windows calls as AutoIT, except it returns all the information that it rightfully should for each file found - including:

  • File attributes (in numerical form, not a silly string format)
  • Creation Time
  • Last-Access Time
  • Last-Write Time
  • FileSize
  • Filename (obviously)
  • 8.3 short name (if it is 1. different from the regular Filename and 2. if short-names haven't been turned off
  • Reparse Point info (if available)
Now, the calling process is a little different, though for the most part not much is required to be altered in existing code. Basically, the attributes-check for folders is a numerical test, and when a folder is found, you *need* to test for '.' and '..' navigation (fake) folders. Also, the 'While' loop changes into a 'Do-Until' loop. Additionally, the first _FileFindExFirstFile() call returns a file, whereas FileFindFirstFile() doesn't (which never made sense to me).

To convert times into a readable format, you'll need to pass the array to the _FileFindExTimeConvert() function. Optionally, you can get the _WinTimeFunctions UDF and call those functions using array index constants:

$FFX_CREATION_TIME, $FFX_LAST_ACCESS_TIME, or $FFX_LAST_MODIFIED_TIME.

Note all _FileFindEx* calls are done using a special array, though 'ByRef' for quicker performance.

A nice application I found for this UDF was comparing files/folders - which is pretty easy using 64-bit filetime and file-size comparisons (no need for time conversion there).

The ZIP includes 4 files: the _FileFindEx UDF, FileFindExTest, the license agreement (same as below), and _LinkedOutputDisplay. Please note that _LinkedOutputDisplay on its own is a mess - but its included as-is to help see a side-by-side comparison of the output from FileFindExTest.

To get all the same information that _FileFindEx provides, the AutoIt functions below would need to be called:

  • FileFindFirst/NextFile
  • FileGetAttrib *** NOTE: This Fails to report on some attributes (Reparse Points, Sparse Files) ***
  • FileGetTime *3 (one for each time - Creation, Last-Access, Last-Modified)
  • FileGetSize
  • FileGetShortName (note: this provides a full path, rather than just a file name)
Please note that for a fair time comparison, a clean boot is needed for each test due to O/S buffering after a scan. Between boots, the order of function calls in 'FileFindExTest' would need to be swapped. In first-run tests, _FileFindEx has consistently been quicker when gathering more than basic filename info. However, running the UDF in 64-bit mode on Vista+ O/S's results in slower performance, hence this note:

*IMPORTANT* - Speed is severely affected on certain processors when the script is run in x64 mode. I therefore recommend running/compiling the code in both bit-modes beforehand to see what the speed difference is. On my machine I've found x86 is much faster, whereas x64 is much slower than AutoIt's search functions. Other's have so I'm guessing it must be how optimized the hardware architecture is at running x64 code.

Update Log:

  • 5/24/2011: Further speed optimization when using _FileFindExTimeConvert().
  • 5/23/2011: New name, faster execution!

    * Removed: _WinTimeFunctions UDF. No longer needed (see below).

    * Added: New array element for REPARSE Points that are located ($aFind[7]).

    * Added: Internal time-conversion function removes the need for _WinTime* calls. This results in a solid increase in performance.

    * SCRIPT-BREAKING CHANGES *: Function-names changed - A simple search&replace should work here.

  • 5/9/2010: New version with better 64-bit FileTime support, bundled separate example with output display, and the _WinTimeFunctions required module (+ example use). All code falls under the same License agreement! AutoIT v3.3.6+ required.
  • - Updated for AutoIT v3.3 support (or 3.2.12.1 UNICODE-only)

Download the ZIP Here

Ascend4nt's AutoIT Code License agreement:

While I provide this source code freely, if you do use the code in your projects, all I ask is that:

  • If you provide source, keep the header as I have put it, OR, if you expand it, then at least acknowledge me as the original author, and any other authors I credit
  • If the program is released, acknowledge me in your credits (it doesn't have to state which functions came from me, though again if the source is provided - see #1)
  • The source on it's own (as opposed to part of a project) can not be posted unless a link to the page(s) where the code were retrieved from is provided and a message stating that the latest updates will be available on the page(s) linked to.
  • Pieces of the code can however be discussed on the threads where Ascend4nt has posted the code without worrying about further linking.
Edited by Ascend4nt

Share this post


Link to post
Share on other sites

@ascendant

Great and useful job ! :P

Five stars from me :unsure:

Cheers, FireFox.


 

OS : Win XP SP2 (32 bits) / Win 7 SP1 (64 bits) / Win 8 (64 bits) | Autoit version: latest stable / beta.
Hardware : Intel(R) Core(TM) i5-2400 CPU @ 3.10Ghz / 8 GiB RAM DDR3.

My UDFs : Skype UDF | TrayIconEx UDF | GUI Panel UDF | Excel XML UDF | Is_Pressed_UDF

My Projects : YouTube Multi-downloader | FTP Easy-UP | Lock'n | WinKill | AVICapture | Skype TM | Tap Maker | ShellNew | Scriptner | Const Replacer | FT_Pocket | Chrome theme maker

My Examples : Capture toolIP Camera | Crosshair | Draw Captured Region | Picture Screensaver | Jscreenfix | Drivetemp | Picture viewer

My Snippets : Basic TCP | Systray_GetIconIndex | Intercept End task | Winpcap various | Advanced HotKeySet | Transparent Edit control

 

Share this post


Link to post
Share on other sites

Thanks guys =)

@ptrex - that really annoys me that they removed that, arghh.. I'm still clinging to 3.2.12.1, even though I haven't found myself coding for anything ANSI lately, and I keep modifying the 3.2.12.1 WinAPI functions to include error handling hehe. I guess I should just make the switch and never look back!

Win9x oh how I will miss you.. :P


Share this post


Link to post
Share on other sites

Heh, I'd be honored to have my code included in that, I must admit SMF's a pretty awesome looking tool. I've also been considering someday working with your ShellTriStateTreeView, another awesome program.


Share this post


Link to post
Share on other sites

Updated UDF to support AutoIT v3.3, or UNICODE v3.2.12.1, see 1st post for new code


Share this post


Link to post
Share on other sites

Wow... the speed stats are impressive.. at least for the first run

_WinAPI_FileFind Stats:
Total File count:2072, Total Folder count:52, Time elapsed:823.423088688646 ms
FileFindFile Stats:
Total File count:2072, Total Folder count:52, Time elapsed:2259.5382932747 ms

Second run and so on; I'm getting varied results. Most of the time FileFindFile() is faster your function. :)

_WinAPI_FileFind Stats:
Total File count:2072, Total Folder count:52, Time elapsed:1103.81850207219 ms
FileFindFile Stats:
Total File count:2072, Total Folder count:52, Time elapsed:671.125240777808 ms

a little better...

_WinAPI_FileFind Stats:
Total File count:2072, Total Folder count:52, Time elapsed:867.172224402822 ms
FileFindFile Stats:
Total File count:2072, Total Folder count:52, Time elapsed:688.528849336997 ms

Don't bother, It's inside your monitor!------GUISetOnEvent should behave more like HotKeySet()

Share this post


Link to post
Share on other sites

Second run and so on; I'm getting varied results. Most of the time FileFindFile() is faster your function. :)

That depends on what you expect to be the return :) , this one not only returns file location info, but at the same time file-attributes, size and times...

Share this post


Link to post
Share on other sites

Wow... the speed stats are impressive.. at least for the first run

...

Second run and so on; I'm getting varied results. Most of the time FileFindFile() is faster your function. :)

...

Kastout, I think I remarked that for getting the attributes and all, _WinAPI_FileFind will always be faster than FileFind.. functions. As for getting just pure filename info, your best bet is the built in FileFind.. functions.

Also, remember, when you are timing how long it takes to read files on a hard drive, the first search will always be the longest, and all subsequent calls will be much faster. That's due to how Windows will buffer the information (though how long the results stay in memory I couldn't say).

The best bet for a clear performance comparison is to reboot Windows between compares. (And don't run them one after the other, but one version on the first clean boot, the other on the 2nd boot)

Also make sure you don't have any other processes actively eating up processor time or accessing the hard drive excessively..


Share this post


Link to post
Share on other sites

Awesome function set, just needed a bump :) ...

Hey, thanks for promoting my UDF ;) You even added it to your sig! Wow, and who gave this UDF 5 stars? Nothing I ever did got stars!! haha.. i feel special now :)

By the way, I've rewritten this UDF so that it can indeed grab the FileTime as a 64-bit number (has to do with structure-alignment), for a little less code and ease of use (and the ability to do some neat manipulations).

Then I went further and made it possible to format the time string in a crapload of ways, to manipulate the FileTime itself (adding/subtracting minutes, hours, days, and so on), and do multiple conversions.

But I digress... all that stuff I'd rather not post here, because this as it stands does just enough and the programmer can choose to modify it at their own leisure.

Anyway, thanks again. Seeya around the forums!


Share this post


Link to post
Share on other sites

is the udf missing?

Yes, the member has left the building and he decided to take his code with him. >_<

This is what he has written in his profile about removing the code:

My apologies to the AutoIT community for removing my posted code. This was done because I knew I would be banned, and I'd rather not have the inability to access, update, and respond to anything related to MY hard work. I hope to move my code over to Google sites, but for now I'm just posting on my blog about the potential future.

For those that want to know about the banning, and my response to the person-responsible's incorrect assertions in this thread, you could visit this blog post. (He specifically said he did not want to discuss things further - which often means 'you'll be banned if you continue', so this is my only recourse)


If you learn from It, it's not a mistake

Share this post


Link to post
Share on other sites

Text and Link to code is now added. Sorry about any inconveniences :(


Share this post


Link to post
Share on other sites

After having this sitting on my PC for a while, I decided to upload the new version. It has the same functioning as it did before, but the FileTime's are now individual array items, and additionally the _WinTimeFunctions module is needed.

UPDATES:

- 5/9/2010: New version with better 64-bit FileTime support, bundled separate example with output display, and the _WinTimeFunctions required module (+ example use). All code falls under the same License agreement! AutoIT v3.3.6+ required.


Share this post


Link to post
Share on other sites

This UDF has been updated! Besides getting rid of the '_WinAPI_' prefix, I've worked on increasing performance with this release, as well as providing Reparse Point information. I'm dedicating this update to KaFu, maker of the fine SMF (Search My Files) program, which includes an older version of this UDF.

Updates:

5/23/2011: New name, faster execution!

* Removed: _WinTimeFunctions UDF. No longer needed (see below).

* Added: New array element for REPARSE Points that are located ($aFind[7]).

* Added: Internal time-conversion function removes the need for _WinTime* calls. This results in a solid increase in performance.

* SCRIPT-BREAKING CHANGES *: Function-names changed - A simple search&replace should work here.


Share this post


Link to post
Share on other sites

Share this post


Link to post
Share on other sites

Ahh! I just looked at your 1.5 source again and realized there will be more script-breaking changes then I listed, since you used an older version which still used 32-bit values instead of 64-bit ones (changed in last year's release). The array will now be indexed/accessed differently for you, so it'll be a little more work to get the indexes correct. Sorry about that, mate!


Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Scriptmaster
      <NEWBIE ALERT>
      Is it possible to unzip a zip file in Autoit without using a dll!
      My script downloads a zip file to a temporary directory (successfully) and it must be unzipped to (Or moved to after unzip) another a folder which just happens to be
      @scriptdir & "file.exe" Thanks in advance
    • By Inpho
      Hi All,
      I intend on keeping custom functions/UDFs (works in progress) here; if anyone wants to use any code, feel free.
      String functions:
      #AutoIt3Wrapper_AU3Check_Parameters=-d -w- 1 -w 2 -w 3 -w 4 -w 5 -w 6 #include-once ; #FUNCTION# ==================================================================================================================== ; Name ..........: _DateTimeGet ; Description ...: Returns the date and time formatted for use in sortable filenames, logs, listviews, etc. ; Syntax ........: _DateTimeGet(iType = 1[, $bHumanFormat = False]) ; Parameters ....: $iType - [optional] an integer value. Default is 1. ; 1 - Date and time in file-friendly format; 20190115_113756 ; 2 - Date in file-friendly format; 20190115 ; 3 - Time in file friendly format; 113756 ; $bHumanFormat - [optional] a boolean value. Default is False. ; True - Includes slashes in the date and colons in the time with a space inbetween ; False - No slashes or colons included with an underscore inbetween ; Return values .: Success - String ; Failure - Sets @error to non-zero and returns an empty string ; Author ........: Sam Coates ; =============================================================================================================================== Func _DateTimeGet($iType = 1, $bHumanFormat = False) If $iType < 1 Or $iType > 3 Then Return (SetError(-1, 0, "")) ;; Param1: ;; 1 = Date and time in file friendly format: 20190115_113756 ;; 2 = Date in file friendly format: 20190115 ;; 3 = Time in file friendly format: 113756 ;; Param2: ;; True = Use human-readable format: 15/01/2019 11:37:56 Local $sTime = @HOUR & ":" & @MIN & ":" & @SEC Local $sDate = @MDAY & "/" & @MON & "/" & @YEAR If $iType = 1 Then If $bHumanFormat = False Then $sTime = StringReplace($sTime, ":", "") $sDate = StringReplace($sDate, "/", "") $sDate = StringTrimLeft($sDate, 4) & StringMid($sDate, 3, 2) & StringLeft($sDate, 2) Return ($sDate & "_" & $sTime) Else Return ($sDate & " " & $sTime) EndIf ElseIf $iType = 2 Then If $bHumanFormat = False Then $sDate = StringReplace($sDate, "/", "") $sDate = StringTrimLeft($sDate, 4) & StringMid($sDate, 3, 2) & StringLeft($sDate, 2) EndIf Return ($sDate) ElseIf $iType = 3 Then If $bHumanFormat = False Then $sTime = StringReplace($sTime, "/", "") EndIf Return ($sTime) EndIf EndFunc ;==>_DateTimeGet ; #FUNCTION# ==================================================================================================================== ; Name ..........: _FileToFileExtension ; Description ...: Returns a file extension from a filename/FQPN (Fully Qualified Path Name) ; Syntax ........: _FileToFileExtension($sPath) ; Parameters ....: $sPath - a string value. ; Return values .: Success - String ; Failure - Empty string as returned from StringTrimLeft() ; Author ........: Sam Coates ; =============================================================================================================================== Func _FileToFileExtension($sPath) Return (StringTrimLeft($sPath, StringInStr($sPath, ".", 0, -1))) EndFunc ;==>_FileToFileExtension ; #FUNCTION# ==================================================================================================================== ; Name ..........: _FileToFileName ; Description ...: Returns a filename from a FQPN (Fully Qualified Path Name) ; Syntax ........: _FileToFileName($sPath[, $bIncludeExtension = True]) ; Parameters ....: $sPath - a string value. ; $bIncludeExtension - [optional] a boolean value. Default is True. ; Return values .: Success - String ; Failure - Empty string as returned from StringLeft() ; Author ........: Sam Coates ; =============================================================================================================================== Func _FileToFileName($sPath, $bIncludeExtension = True) Local $sReturn = StringTrimLeft($sPath, StringInStr($sPath, "\", 0, -1)) If $bIncludeExtension = False Then $sReturn = StringLeft($sReturn, StringInStr($sReturn, ".", 0, -1) - 1) Return ($sReturn) EndFunc ;==>_FileToFileName ; #FUNCTION# ==================================================================================================================== ; Name ..........: _FileToFilePath ; Description ...: Returns a folder path from a FQPN (Fully Qualified Path Name) ; Syntax ........: _FileToFilePath($sPath) ; Parameters ....: $sPath - a string value. ; Return values .: Success - String ; Failure - Empty string as returned from StringLeft() ; Author ........: Sam Coates ; =============================================================================================================================== Func _FileToFilePath($sPath) Return (StringLeft($sPath, StringInStr($sPath, "\", 0, -1) - 1)) EndFunc ;==>_FileToFilePath ; #FUNCTION# ==================================================================================================================== ; Name ..........: _StringLeft ; Description ...: Searches for a string inside a string, then removes everything on the right of that string ; Syntax ........: _StringLeft($sString, $sRemove[, $iCaseSense = 0, $iOccurrence = 1]) ; Parameters ....: $sString - a string value. The string to search inside. ; $sRemove - a string value. The string to search for. ; $iCaseSense - an integer value. Flag to indicate if the operations should be case sensitive. ; $iOccurrence - an integer value. Which occurrence of the substring to find in the string. Use a ; negative occurrence to search from the right side. ; Return values .: Success - String ; Failure - Empty string as returned from StringLeft() ; Author ........: Sam Coates ; =============================================================================================================================== Func _StringLeft($sString, $sRemove, $iCaseSense = 0, $iOccurrence = 1) Return (StringLeft($sString, StringInStr($sString, $sRemove, $iCaseSense, $iOccurrence) - 1)) EndFunc ;==>_StringLeft ; #FUNCTION# ==================================================================================================================== ; Name ..........: _StringRandom ; Description ...: Returns a string of random characters ; Syntax ........: _StringRandom($iAmount[, $iType = 1]) ; Parameters ....: $iAmount - an integer value. Length of returned string ; $iType - [optional] an integer value. Default is 1. ; 1 - Return digits (0-9) ; 2 - Return hexadecimal (0-9, A - F) ; 3 - Return Alphanumeric upper (0-9, A - Z) ; 4 - Return Alphanumeric (0-9, A - Z, a - z) ; 5 - Return Alpha upper (A - Z) ; 6 - Return Alpha (A - Z, a - z) ; Return values .: Success - String ; Failure - Empty string and @error flag as follows: ; @error : 1 - $iAmount is not a positive integer ; 2 - $iType is out of bounds ; Author ........: Sam Coates ; =============================================================================================================================== Func _StringRandom($iAmount, $iType = 1) If $iAmount < 1 Or IsInt($iAmount) = 0 Then Return (SetError(-1, 0, "")) Local $sString = "" Local $iRandomLow = 1, $iRandomHigh = 62 #Tidy_Off Local Static $aCharId[63] = [0, Chr(48), Chr(49), Chr(50), Chr(51), Chr(52), Chr(53), Chr(54), Chr(55), Chr(56), Chr(57), Chr(65), Chr(66), Chr(67), _ Chr(68), Chr(69), Chr(70), Chr(71), Chr(72), Chr(73), Chr(74), Chr(75), Chr(76), Chr(77), Chr(78), Chr(79), Chr(80), _ Chr(81), Chr(82), Chr(83), Chr(84), Chr(85), Chr(86), Chr(87), Chr(88), Chr(89), Chr(90), Chr(97), Chr(98), Chr(99), _ Chr(100), Chr(101), Chr(102), Chr(103), Chr(104), Chr(105), Chr(106), Chr(107), Chr(108), Chr(109), Chr(110), Chr(111), _ Chr(112), Chr(113), Chr(114), Chr(115), Chr(116), Chr(117), Chr(118), Chr(119), Chr(120), Chr(121), Chr(122)] #Tidy_On If $iType = 1 Then ;; digits: 1 - 10 $iRandomHigh = 10 ElseIf $iType = 2 Then ;; hexadecimal: 1 - 16 $iRandomHigh = 16 ElseIf $iType = 3 Then ;; alnumupper: 1 - 36 $iRandomHigh = 36 ElseIf $iType = 4 Then ;; alnum: 1 - 62 $iRandomHigh = 62 ElseIf $iType = 5 Then ;; alphaupper: 11 - 36 $iRandomLow = 11 $iRandomHigh = 36 ElseIf $iType = 6 Then ;; alpha: 11 = 62 $iRandomLow = 11 $iRandomHigh = 62 Else Return (SetError(-2, 0, "")) EndIf For $i = 1 To $iAmount $sString &= $aCharId[Random($iRandomLow, $iRandomHigh, 1)] ;; append string with corresponding random character from ascii array Next Return ($sString) EndFunc ;==>_StringRandom ; #FUNCTION# ==================================================================================================================== ; Name ..........: _StringTrimLeft ; Description ...: Searches for a string inside a string, then removes everything on the left of that string ; Syntax ........: _StringTrimLeft($sString, $sRemove[, $iCaseSense = 0, $iOccurrence = 1]) ; Parameters ....: $sString - a string value. The string to search inside. ; $sRemove - a string value. The string to search for. ; $iCaseSense - an integer value. Flag to indicate if the operations should be case sensitive. ; $iOccurrence - an integer value. Which occurrence of the substring to find in the string. Use a ; negative occurrence to search from the right side. ; Return values .: Success - String ; Failure - Empty string as returned from StringTrimLeft() ; Author ........: Sam Coates ; =============================================================================================================================== Func _StringTrimLeft($sString, $sRemove, $iCaseSense = 0, $iOccurrence = 1) Return (StringTrimLeft($sString, StringInStr($sString, $sRemove, $iCaseSense, $iOccurrence) + StringLen($sRemove) - 1)) EndFunc ;==>_StringTrimLeft Examples:
      ConsoleWrite(_StringRandom(100, 6) & @CRLF) ConsoleWrite(_StringTrimLeft("C:\Windows\System32\cmd.exe", "C:\Windows\System32\") & @CRLF) ConsoleWrite(_StringLeft("C:\Windows\System32\cmd.exe", "cmd.exe") & @CRLF) ConsoleWrite(_FileToFileName("C:\Windows\System32\cmd.exe") & @CRLF) ConsoleWrite(_FileToFilePath("C:\Windows\System32\cmd.exe") & @CRLF) ConsoleWrite(_FileToFileExtension("C:\Windows\System32\cmd.exe") & @CRLF) ConsoleWrite(_StringRandom(6, 4) & "-" & _StringRandom(4, 4) & "-" & _StringRandom(4, 4) & "-" & _StringRandom(4, 4) & "-" & _StringRandom(6, 4)& @CRLF)  
    • By nacerbaaziz
      hello sirs
      please i want to use the WinHttp to get the google drive file title using the api
      i searched in the forom but i didn't found any Google drive api UDF
      for that i liked to ask you for that
      i had read the google drive api documentation but i couldn't do it
      please help me
      here is the api doc
      note i've got the apikey and i have the file id
      what i want is to get the title of the file (the file name)
      because i want to download the files from the google drive using the autoit
      please help me for that
      am sorry because i didn't gave you any example or what i tried but all what tried was  failed
      thanks in advance
    • By dinotom
      Can files be read from the web or the cloud?
      I am trying to read this file but get the file open error.
      #include <FileConstants.au3> #include <MsgBoxConstants.au3> ;Assign the file path to a variable Local $sFilePath = "C:\Automation\test.txt" Local $sFilePathAzure ="https://batlgroupimages.blob.core.windows.net/files/test.txt" Local $nLineNumberToLookFor = 0 ;Open the file test.txt in append mode. ;If the folder C:\Automation does not exist, it will be created. Local $hFileOpen = FileOpen($sFilePathAzure, $FO_APPEND + $FO_CREATEPATH) ;Display a message box in case of any errors. If $hFileOpen = -1 Then MsgBox($MB_SYSTEMMODAL, "", "An error occurred while opening the file.") EndIf ;Set the file position to beginning for reading the data from the beginning of the file. FileSetPos($hFileOpen, 0, $FILE_BEGIN) ;Read the data from the file using the handle returned by FileOpen ;Local $sFileRead = FileRead($hFileOpen) ;Read the 2nd line of data from the file using the handle returned by FileOpen Local $sFileReadLine = FileReadLine ($hFileOpen,2) ;Close the handle returned by FileOpen. FileClose($hFileOpen)  
×
×
  • Create New...