73 replies to this topic

[guinness]

I took the idea from the function_RFLTA_ListToMask() by Melba23 and modified it a little. So all the credit should be focused towards those mentioned in the function header.

Any suggestions or problems post below.

Function:

; #FUNCTION# ====================================================================================================================
; Name ..........: _IsValidFileType
; Description ...: Checks if a filepath contains an approved filetype extension.
; Syntax ........: _IsValidFileType($sFilePath[, $sList = 'bat;cmd;exe'[, $iOpFast = 1]])
; Parameters ....: $sFilePath - File path of the file.
;                  $sList - [optional] A list of filetype extensions separated with ';' e.g. 'zip;rar;7zip;gzip'. Default is 'bat;cmd;exe'.
;                  $iOpFast - [optional] Either 0 (No) or 1 (True) to use the faster operation for StringReplace. Default is 1.
; Return values .: Success - Return 1 = Valid filetype or 0 = InValid filetype
; Author ........: guinness
; Modified ......:
; Remarks........; The code was initially based on the original function of _RFLTA_ListToMask() by Melba23, but overtime with input from
;                  AZJIO, Beege and Shaggi, I opted on using Robjong's suggestion [http://www.autoitscript.com/forum/topic/123674-isvalidtype/page__view__findpost__p__959470.]
; Related .......: _WinAPI_PathMatchSpec in WinAPIEx.au3 by Yashied.
; Link ..........:
; Example .......: Yes
; ===============================================================================================================================
Func _IsValidFileType($sFilePath, $sList = 'bat;cmd;exe', $iOpFast = 1)
    Return StringRegExp($sFilePath, '.(?i:Q' & StringReplace($sList, ';', 'E|Q', 0, $iOpFast * 2) & 'E)z')
EndFunc   ;==>_IsValidFileType

Example 1:

ConsoleWrite(_IsValidFileType('AutoIt3.cmd') & @CRLF) ; Returns 1 as .cmd has been approved as a valid filetype.
ConsoleWrite(_IsValidFileType('C:AutoIt3.exe') & @CRLF) ; Returns 1 as .exe has been approved as a valid filetype.
ConsoleWrite(_IsValidFileType('C:AutoIt3.zip') & @CRLF) ; Returns 0 as .zip hasn't been approved as a valid filetype.
ConsoleWrite(_IsValidFileType('C:AutoIt3.zip', 'zip;rar;7zip;gzip') & @CRLF) ; Returns 1 as .zip has been approved as a valid filetype.

Example 2:

AutoIt        expandcollapse  popup

#include <GUIConstantsEx.au3>
#include <StaticConstants.au3>
#include <WindowsConstants.au3>

Example()

Func Example()
    Local $hGUI = GUICreate('_IsValidFileType Simple Drop', 255, 155, Default, Default, BitXOR($GUI_SS_DEFAULT_GUI, $WS_MINIMIZEBOX), $WS_EX_ACCEPTFILES)
    Local $iLabel = GUICtrlCreateLabel('Drop Supported FileType (au3/exe/txt)', 2, 2, 250, 150, $SS_CENTER)
    GUICtrlSetState(-1, $GUI_DROPACCEPTED)
    GUISetState(@SW_SHOW, $hGUI)

    While 1
        Switch GUIGetMsg()
            Case $GUI_EVENT_CLOSE
                ExitLoop

            Case $GUI_EVENT_DROPPED
                If _IsValidFileType(@GUI_DragFile, 'au3;exe;txt') Then
                    GUICtrlSetData($iLabel, 'Dropped - ' & @GUI_DragFile)
                Else
                    GUICtrlSetData($iLabel, 'I said ''Supported FileType!''')
                EndIf

        EndSwitch
    WEnd
    GUIDelete($hGUI)
EndFunc   ;==>Example

For a more complex approach with an improvement in speed then have a look at Robjong's example in post #54. It should be noted that the speed difference isn't in seconds but milli/nanoseconds.

Edited by guinness, 23 September 2012 - 09:21 PM.

[Melba23]

guinness,

The hours were not spent just by me - I had a lot of help!

The function header says it all:

"SRE patterns developed from those posted by various forum members and Spiff59 in particular"

M23

[guinness]

OK, sorry about that. I updated the OP to reflect the countless hours of improvement

[guinness]

Update OP and added a new Example.

[guinness]

I updated the OP with a small bug fix, which was fixed in Melba's latest RecFileListToArray.

[Shaggi]

why all the regexp?

func isvalidtype($path, $aftypes = 0)
    if not $aftypes or not isarray($aftypes) then
        local $types[4] = ["pif","bat","exe","cmd"]
    else
        local $types = $aftypes
    endif
    local $ipos = stringinstr($path,".",default,-1), $stype
    if not $ipos then return false
    $stype = stringmid($path,$ipos+1)
    for $ext in $types
        if $stype = $ext then return true
    next
    return false
endfunc

[guinness]

Sorry? What's the problem with using SRE?

Edit: Also your function is badly formatted and will not pass Au3Check.

Edited by guinness, 31 January 2012 - 08:56 PM.

[Shaggi]

Sorry? What's the problem with using SRE?

besides being a lot slower, i think its overly complicated for such a small task. but most importantly, it embeds another language which most of autoit users dont understand and thus obfuscates the code. (kiss)

Edit: Also your function is badly formatted and will not pass Au3Check.

? it does. i dont see anything unusual besides lower case

[guinness]

? it does. i dont see anything unusual besides lower case

Run Au3Check and you'll see why. Hint: Declaring a variable twice.

besides being a lot slower, i think its overly complicated for such a small task. but most importantly, it embeds another language which most of autoit users dont understand and thus obfuscates the code. (kiss)

No comment as I want to keep this thread on topic.

[Shaggi]

Run Au3Check and you'll see why. Hint: Declaring a variable twice.

>Running AU3Check (1.54.19.0) from:C:\Program Files (x86)\AutoIt3

+>00:00:04 AU3Check ended.rc:0

you might have another version? but no variable is declared twice.

No comment as I want to keep this thread on topic.

cool, but you asked yourself - anyway, it was just an honest comment

[guinness]

if not $aftypes or not isarray($aftypes) then
local $types[4] = ["pif","bat","exe","cmd"]
else
local $types = $aftypes

Apologies Au3Check didn't show an error.

[AZJIO]

If to execute StringReplace($sList, ";", "|*."), symbol ";" any more won't be. The next line isn't meaningful
"s*;s*", ";"
";", "|"

-------------------
"|/|:|<|>||" = [/:<>|]

[guinness]

Good catch AZJIO thanks! And also for the other advice.

[guinness]

I've updated the original post with a mistake pointed out by AZJIO.

[guinness]

Actually a lot can be improved, give me 10 minutes.

[guinness]

Complete. I stripped the function back to the very basics. I've updated the original post.

[guinness]

Shaggi,

Maybe I'm wrong but _IsValidType seems a little quicker now.

AutoIt        expandcollapse  popup

Local $iTimer = TimerInit()
For $A = 1 To 5000
    isvalidtype(@ScriptFullPath)
Next
ConsoleWrite(TimerDiff($iTimer) / 5000 & @LF)

$iTimer = TimerInit()
For $A = 1 To 5000
    _IsValidType(@ScriptFullPath)
Next
ConsoleWrite(TimerDiff($iTimer) / 5000 & @LF)

Func isvalidtype($path, $aftypes = 0)
    If Not $aftypes Or Not IsArray($aftypes) Then
        Local $types[3] = ["bat", "exe", "cmd"]
    Else
        Local $types = $aftypes
    EndIf
    Local $ipos = StringInStr($path, ".", Default, -1), $stype
    If Not $ipos Then Return False
    $stype = StringMid($path, $ipos + 1)
    For $ext In $types
        If $stype = $ext Then Return True
    Next
    Return False
EndFunc   ;==>isvalidtype

Func _IsValidType($sFilePath, $sList = "bat;cmd;exe")
    If StringRegExp($sList, "[/:<>|]") Then
        Return SetError(1, 0, -1)
    EndIf
    Return StringRegExp($sFilePath, "(?i)^(" & ".*?." & StringReplace($sList, ";", "|.*?.") & ")z")
EndFunc   ;==>_IsValidType

[Shaggi]

Shaggi,

Maybe I'm wrong but _IsValidType seems a little quicker now.

AutoIt        expandcollapse  popup

Local $iTimer = TimerInit()
For $A = 1 To 5000
    isvalidtype(@ScriptFullPath)
Next
ConsoleWrite(TimerDiff($iTimer) / 5000 & @LF)

$iTimer = TimerInit()
For $A = 1 To 5000
    _IsValidType(@ScriptFullPath)
Next
ConsoleWrite(TimerDiff($iTimer) / 5000 & @LF)

Func isvalidtype($path, $aftypes = 0)
    If Not $aftypes Or Not IsArray($aftypes) Then
        Local $types[3] = ["bat", "exe", "cmd"]
    Else
        Local $types = $aftypes
    EndIf
    Local $ipos = StringInStr($path, ".", Default, -1), $stype
    If Not $ipos Then Return False
    $stype = StringMid($path, $ipos + 1)
    For $ext In $types
        If $stype = $ext Then Return True
    Next
    Return False
EndFunc   ;==>isvalidtype

Func _IsValidType($sFilePath, $sList = "bat;cmd;exe")
    If StringRegExp($sList, "[\\/:<>|]") Then
        Return SetError(1, 0, -1)
    EndIf
    Return StringRegExp($sFilePath, "(?i)^(" & ".*?\." & StringReplace($sList, ";", "|.*?\.") & ")\z")
EndFunc   ;==>_IsValidType

perhaps, i dont have the old function anymore? anyway, to test the two, here's a little script that show the difference:

AutoIt        expandcollapse  popup

SmallTest()
ScaledTest()

Func SmallTest()
    $types = "cmd;"
    $etypes = stringsplit($types,";",2)
    $path = "C:\"
    $ext = "wanted_file.err" ;extension is not in the list, so both algorithms have to search everything through
    $path &= $ext

    ;Calculate the overhead for the loop and a lot of func calls
    $testtime = timerinit()
    for $i = 0 to 0xFFFF
        testfunc($path,$types)
    Next
    $overhead = timerdiff($testtime)
    ConsoleWrite("Assumed overhead: "& $overhead & @lf)

    ;test of new _isvalidtype
    $starttime = timerinit()
    for $i = 0 to 0xFFFF
        _IsValidType($path,$etypes)
    next
    $test1 = timerdiff($starttime)

    ;test of isvalidtype
    $starttime = timerinit()
    for $i = 0 to 0xFFFF
        isvalidtype($path,$types)
    next
    $test2 = timerdiff($starttime)

    ConsoleWrite("(small) Test data: " & @lf & "_isvalidtype: " & $test1 - $overhead & @lf & "isvalidtype: " & $test2 - $overhead & @lf)
endfunc

Func ScaledTest()
    $types = "cmd;bat;tiff;jpg;con;dll;exe;bat;pif;txt;xml;doc;bmp;mp3;mov;png;flp;cmd;bat;tiff;jpg;con;dll;exe;bat;pif;txt;xml;doc;bmp;mp3;mov;png;flp;"
    $etypes = stringsplit($types,";",2)
    $path = "C:\random1\random2\random3\random4\random5\random6\random7\random8\random9\random10\random11\random12\random13\random14\random15\random16\random17\random18\random19\random20\random21\random22\random23\random24\random25\random26\random27\random28\random29\random30\"
    $ext = "wanted_file.err" ;extension is not in the list, so both algorithms have to search everything through
    $path &= $ext

    ;Calculate the overhead for the loop and a lot of func calls
    $testtime = timerinit()
    for $i = 0 to 0xFFFF
        testfunc($path,$etypes)
    Next
    $overhead = timerdiff($testtime)
    ConsoleWrite("Assumed overhead: "& $overhead & @lf)

    ;test of new _isvalidtype
    $starttime = timerinit()
    for $i = 0 to 0xFFFF
        _IsValidType($path,$types)
    next
    $test1 = timerdiff($starttime)

    ;test of isvalidtype
    $starttime = timerinit()
    for $i = 0 to 0xFFFF
        isvalidtype($path,$etypes)
    next
    $test2 = timerdiff($starttime)

    ConsoleWrite("(scaled) Test data: " & @lf & "_isvalidtype: " & $test1 - $overhead & @lf & "isvalidtype: " & $test2 - $overhead & @lf)
EndFunc




func isvalidtype(byref $path, $aftypes = 0)
    if not $aftypes or not isarray($aftypes) then
        local $types[3] = ["cmd","bat","exe"]
    else
        local $types = $aftypes
    endif
    local $ipos = stringinstr($path,".",0,-1), $stype
    if not $ipos then return false
    $stype = stringmid($path,$ipos+1)
    for $ext in $types
        if $ext = $stype then
            return true
        endif
    next
    return false
endfunc

Func _IsValidType($sFilePath, $sList = "bat;cmd;exe")
    If StringRegExp($sList, "[\\/:<>|]") Then
        Return SetError(1, 0, -1)
    EndIf
    Return StringRegExp($sFilePath, "(?i)^(" & ".*?\." & StringReplace($sList, ";", "|.*?\.") & ")\z")
EndFunc   ;==>_IsValidType

Func testfunc($a,$b)
EndFunc

results here:

Spoiler

Assumed overhead: 212.181195527824
(small) Test data:
_isvalidtype: 1261.08944816422
isvalidtype: 2009.5421428352
Assumed overhead: 219.468354359682
(scaled) Test data:
_isvalidtype: 25379.333455585
isvalidtype: 1597.608925734

What the test shows, is that yours excels in very small tests, however as the test size grows to medium-long filenames and types, it becomes 16x times slower.
this is because my function is designed to take near constant (actually less in test - but dont count on that) time O(1) for any request , because it searches backwards in the input string, and the only real overhead is the iteration and comparison of the array.
Yours however, seems to be more like linear time O(n), which is not bad, but if this algorithm is used in a recursively folder search with file extension masking, you will note the difference.

[guinness]

OK fair enough. Thanks for testing.

[Beege]

besides being a lot slower

What? Calling SRE a lot slower is fucking crazy. Theres a good reason pcre is included in a ton of languages out there. Its powerful and fast.

but most importantly, it embeds another language which most of autoit users dont understand and thus obfuscates the code.

Don't use the tools built into autoit because some people might not understand how they work? Thats weak. Especially coming from someone whos using hex values in there For loops.

I also want to point out in that perfomance test, the main slowdown is coming from StringReplace(). Not SRE. If you change the code to

Func _IsValidType(ByRef $sFilePath, $sList = "bat;cmd;exe")
    If StringRegExp($sList, "[\\/:<>|]") Then Return SetError(1, 0, -1)
    Return StringRegExp($sFilePath, "(?i)^(" & ".*?\." & StringReplace($sList, ";", "|.*?\.", 2) & ")\z")
EndFunc   ;==>_IsValidType

my results then become:

(scaled) Test data:
_isvalidtype: 3263.63947284885
isvalidtype: 1521.45284004219

Which is only 2x slower. And more so, in reality your function only gains micro seconds. In my opinion its not worth turning a 2 line function into 10 lines just to gain microseconds.

@@ Debug(62) : $test1 = 0.128203815394698
@@ Debug(70) : $test2 = 0.0692218683760509