_FileFindEx - Get More from File/Folder Searches

[KaFu]

No worries, my current Beta already uses the latest version , so no problem there.

[Ascend4nt]

Okay, was a bit unhappy with the optimizations, so made one more. To take it one step further, one can use the $FFX_ constants and call the internal function __FFEXWT_Convert()..

5/24/2011: Further speed optimization when using _FileFindExTimeConvert().

*Edit: Found out speed is severely affected when run in x64 mode! Therefore I recommend running/compiling this code in x86 (32-bit) mode for best performance!

Edited May 24, 2011 by Ascend4nt

[KaFu]

For performance reasons I would recommend to rewrite all

if ... then...

calls to

if ... then

...

endif

[Ascend4nt]

I remember threads on the performance variations on If/Then vs. If/Then/EndIf. The thing is, its not a universal truth that the latter is always faster. In fact, for cases where the 'If' statement turns out to be False, its actually faster to have only the If/Then variation when the result is typically false, as it is in parameter-checks and DLLCall error-checks. So in that way, my code is pretty much already optimized.

Run this to see what I mean:

Local $iTimer,$iVar=1

$iTimer=TimerInit()
For $i=1 To 500000
    If $iVar=0 Then $iVar=1
Next
ConsoleWrite("Time to execute:"&TimerDiff($iTimer)&" ms"&@LF)

$iTimer=TimerInit()
For $i=1 To 500000
    If $iVar=0 Then
        $iVar=1
    EndIf
Next
ConsoleWrite("Time to execute:"&TimerDiff($iTimer)&" ms"&@LF)

What gets me is how slow the 64-bit code is. I'm pretty sure it has to do with AutoIt's implementation of DLLCall() being unoptimized...

How many others get the same result (ie the 64-bit slowdown)?

*edit: code example

Edited May 24, 2011 by Ascend4nt

[KaFu]

Run this to see what I mean:

What confuses me most is that even if you set $iVar = 0 the first option is still faster ...

How many others get the same result (ie the 64-bit slowdown)?

Same here, your function running as 64-bit is much slower (which does not bother me as SMF needs to be compiled to 32bit due to the dlls anyhow ).

Edit:

Exchanging it in SMF took me 20 minutes... and it's definitely faster , awesome work m8 !

Edited May 24, 2011 by KaFu

[Ascend4nt]

What confuses me most is that even if you set $iVar = 0 the first option is still faster ...

Oh, if you do that, the statement ceases to be true on subsequent passes. The 'Then' part should be changed to '$iVar=0' instead of '$iVar=1'. After that, it'll show you that the 2nd option is faster for True 'If' statements.

Same here, your function running as 64-bit is much slower (which does not bother me as SMF needs to be compiled to 32bit due to the dlls anyhow ).

Edit:

Exchanging it in SMF took me 20 minutes... and it's definitely faster , awesome work m8 !

Great to hear! And thanks for the 64-bit check. That kinda makes me want to test other code in both bit modes for speed comparisons.

Edited May 24, 2011 by Ascend4nt

[guinness]

Tested on Windows 7 x64.

When using the x32 Version of AutoIt I found that _FileFindEx() was faster by an average of 150ms. But when run using the x64 Version of AutoIt the difference was huge. _FileFindEx() was again faster but the difference was 900ms between _FileFindEx() & the inbuilt versions!

Nice Function(s)!

[mailro]

Hi,

I tested the function on a win xp64 machine both scripts compiled as x86 and The autoit functions are faster . What I am doing wrong?

Below the scripts

$OrdersDir = "X:\ES\"
FileDelete($OrdersDir & "oif*.txt")
while 1
$search = FileFindFirstFile($OrdersDir & "oif*.txt")
While $search <> -1
  $file = FileFindNextFile($search)
  If @error Then
   FileClose($search)
   ExitLoop
  EndIf
  $Filesize = FileGetSize($OrdersDir & $file)
  If $Filesize > 0 Then
   _dbg("AutoIt: Order Received")
   $Order = StringSplit(FileRead($OrdersDir & $file), ";")
   FileMove($OrdersDir & $file, $OrdersDir & "Exec\*.txt", 1)
  EndIf
WEnd
sleep(5)
WEnd
Func _dbg($msg = "NotFound")
Global $Pass
$Pass += 1
$Pass = String($Pass)
$PID = String(@AutoItPID)
If $msg = "NotFound" Then
  DllCall("kernel32.dll", "none", "OutputDebugString", "str", $PID & " Pass: " & $Pass)
Else
  DllCall("kernel32.dll", "none", "OutputDebugString", "str", $msg)
EndIf
EndFunc   ;==>_dbg

#include <_FileFindEx.au3>
$OrdersDir = "X:\ES2\"
FileDelete($OrdersDir & "oif*.txt")
while 1
$search = _FileFindExFirstFile($OrdersDir & "oif*.txt")
While $search <> -1
  If $search[3] > 0 Then
   _dbg("AutoIt: EX Order Received")
   $Order = StringSplit(FileRead($OrdersDir & $search[0]), ";")
   FileMove($OrdersDir & $search[0], $OrdersDir & "Exec\*.txt", 1)
  Else
   ExitLoop
  EndIf
  If Not _FileFindExNextFile($search) then
   _FileFindExClose($search)
  ExitLoop
EndIf
WEnd
sleep(5)
WEnd
Func _dbg($msg = "NotFound")
Global $Pass
$Pass += 1
$Pass = String($Pass)
$PID = String(@AutoItPID)
If $msg = "NotFound" Then
  DllCall("kernel32.dll", "none", "OutputDebugString", "str", $PID & " Pass: " & $Pass)
Else
  DllCall("kernel32.dll", "none", "OutputDebugString", "str", $msg)
EndIf
EndFunc   ;==>_dbg

You take the diference of 2 time values to find the time needed each function to see the file and report the size.

#, Time , Process

1 0.00000000 [3284] AutoIt: Order Received

2 0.00055759 [3284] AutoIt: Order Received

3 0.00112891 [3284] AutoIt: Order Received

4 0.00162222 [3284] AutoIt: Order Received

5 0.00253099 [3284] AutoIt: Order Received

6 0.00291112 [3284] AutoIt: Order Received

7 0.00349047 [3284] AutoIt: Order Received

8 0.00385437 [3284] AutoIt: Order Received

9 0.00431364 [3284] AutoIt: Order Received

10 0.00485418 [3284] AutoIt: Order Received

1 0.00000000 [1660] AutoIt: EX Order Received

2 0.47575378 [1660] AutoIt: EX Order Received

3 0.47759247 [1660] AutoIt: EX Order Received

4 0.47883606 [1660] AutoIt: EX Order Received

5 0.47932434 [1660] AutoIt: EX Order Received

6 0.47982025 [1660] AutoIt: EX Order Received

7 0.48047638 [1660] AutoIt: EX Order Received

8 0.48097992 [1660] AutoIt: EX Order Received

9 0.48139954 [1660] AutoIt: EX Order Received

10 0.48213196 [1660] AutoIt: EX Order Received

Edited September 25, 2011 by mailro

[Ascend4nt]

I tested the function on a win xp64 machine both scripts compiled as x86 and The autoit functions are faster . What I am doing wrong?

I would often question myself - who in their right mind owns Windows XP 64. Now I have my answer

As far as _FileFindEx being slower, it depends on how many file attributes you are looking at - and whether or not you are 'physically' touching each file. In your case, you are reading the entire contents of every file, so there's no need for the functions in my UDF. You won't see a speed advantage. Also, no need to read the filesize. Simply do 'FileRead' and check the # of characters returned to see if it's < 1.

[mailro]

Hi,

Thanks for the tip.

[money]

I filed a feature request to return "extended information" as a optional parameter in the FileFindNextFile function, but it was quickly shot down by Valik.

http://www.autoitscript.com/trac/autoit/ticket/2045

Thanks for your function, this is much better than waiting for the devs to reconsider.

Edited November 1, 2011 by money

[Ascend4nt]

AutoIt wasn't intended to be used the way it is today.. a lot of features were taken out to simplify programming. FindFirst/Next returning a simple filename was probably the most people needed in the early stages. The rest of the information available is discarded by AutoIt, and, while your suggestion to add it in as extended attributes is a good idea, you'll always get shot down if you request something that can be done through other means. Plus, like Valik said in that ticket, speed is not a concern of the developers. Underscoring that fact is the noticeable speed drop in the latest betas with COM objects, something which has been noted by the developers, but is currently being ignored.

Additionally, you won't get much of anything else added or implemented in AutoIt since DLLCall opens up every nook and cranny of the Windows API. The most basic of things you may expect a language to have implemented won't be, precisely because of that ability to do it 'another way'. I would make a point to say however that AutoIt is currently failing - and in a pretty big way - at being true to its roots in Automation. To fix that would mean adding interfaces for IAccessible and IUIAutomation to the language. Of course, now that ObjCreateInterface() is a part of AutoIt, these important automation interfaces will likely go ignored and the burden will be put on the community to fix that part of AutoIt that no longer works. Might as well rename it 'It'...

oh gee, a soapbox has developed at my feet. I'll step off it now, thanks

[Cposture]

It's a very nice work.it's a more fast and useful function then filefindnextfile & filefindnextfile