[SOLVED] Is there any _ArrayFindAll replacement that support wildcard?

[michaelslamet]

Hi fellow members,

Currently i'm using _ArrayFindAll for searching into array until I realized now I need to using wildcard (*) as a search criteria and _ArrayFindAll doesn't support it

Is there any similiar function that do what _ArrayFindAll do but support wildcard as a search criteria?

Thanks

Edited January 7, 2013 by michaelslamet

[michaelslamet]

Or can we modify the original array.au3, maybe using RegEx to accomplish this?

Really have no idea, googling and searching through this forum bring nothing

; #FUNCTION# ====================================================================================================================
; Name...........: _ArrayFindAll
; Description ...: Find the indices of all ocurrences of a search query between two points in a 1D or 2D array using _ArraySearch().
; Syntax.........: _ArrayFindAll(Const ByRef $avArray, $vValue[, $iStart = 0[, $iEnd = 0[, $iCase = 0[, $iPartial = 0[, $iSubItem = 0]]]]])
; Parameters ....: $avArray  - The array to search
;                 $vValue   - What to search $avArray for
;                 $iStart   - [optional] Index of array to start searching at
;                 $iEnd  - [optional] Index of array to stop searching at
;                 $iCase    - [optional] If set to 1, search is case sensitive
;                 $iCompare - [optional] 0 AutoIt variables compare (default), "string" = 0, "" = 0  or "0" = 0 match
;                                        1 executes a partial search (StringInStr)
;                                        2 comparison match if variables have same type and same value
;                 $iSubItem - [optional] Sub-index to search on in 2D arrays
; Return values .: Success - An array of all index numbers in array containing $vValue
;                 Failure - -1, sets @error (see _ArraySearch() description for error codes)
; Author ........: GEOSoft, Ultima
; Modified.......:
; Remarks .......:
; Related .......: _ArrayBinarySearch, _ArraySearch
; Link ..........:
; Example .......: Yes
; ===============================================================================================================================
Func _ArrayFindAll(Const ByRef $avArray, $vValue, $iStart = 0, $iEnd = 0, $iCase = 0, $iCompare = 0, $iSubItem = 0)
    $iStart = _ArraySearch($avArray, $vValue, $iStart, $iEnd, $iCase, $iCompare, 1, $iSubItem)
    If @error Then Return SetError(@error, 0, -1)

    Local $iIndex = 0, $avResult[UBound($avArray)]
    Do
        $avResult[$iIndex] = $iStart
        $iIndex += 1
        $iStart = _ArraySearch($avArray, $vValue, $iStart + 1, $iEnd, $iCase, $iCompare, 1, $iSubItem)
    Until @error

    ReDim $avResult[$iIndex]
    Return $avResult
EndFunc   ;==>_ArrayFindAll

expandcollapse popup

; #FUNCTION# ====================================================================================================================
; Name...........: _ArraySearch
; Description ...: Finds an entry within a 1D or 2D array. Similar to _ArrayBinarySearch(), except that the array does not need to be sorted.
; Syntax.........: _ArraySearch(Const ByRef $avArray, $vValue[, $iStart = 0[, $iEnd = 0[, $iCase = 0[, $iPartial = 0[, $iForward = 1[, $iSubItem = -1]]]]]])
; Parameters ....: $avArray  - The array to search
;                 $vValue   - What to search $avArray for
;                 $iStart   - [optional] Index of array to start searching at
;                 $iEnd  - [optional] Index of array to stop searching at
;                 $iCase    - [optional] If set to 1, search is case sensitive
;                 $iCompare - [optional] 0 AutoIt variables compare (default), "string" = 0, "" = 0  or "0" = 0 match
;                                        1 executes a partial search (StringInStr)
;                                        2 comparison match if variables have same type and same value
;                 $iForward - [optional] If set to 0, searches the array from end to beginning (instead of beginning to end)
;                 $iSubItem - [optional] Sub-index to search on in 2D arrays
; Return values .: Success - The index that $vValue was found at
;                 Failure - -1, sets @error:
;                 |1 - $avArray is not an array
;                 |2 - $avArray is not a 1 or 2 dimensional array
;                 |4 - $iStart is greater than $iEnd
;                 |6 - $vValue was not found in array
;                 |7 - $avArray has too many dimensions
; Author ........: SolidSnake <MetalGX91 at GMail dot com>
; Modified.......: gcriaco <gcriaco at gmail dot com>, Ultima - 2D arrays supported, directional search, code cleanup, optimization
; Remarks .......: This function might be slower than _ArrayBinarySearch() but is useful when the array's order can't be altered.
; Related .......: _ArrayBinarySearch, _ArrayFindAll
; Link ..........:
; Example .......: Yes
; ===============================================================================================================================
Func _ArraySearch(Const ByRef $avArray, $vValue, $iStart = 0, $iEnd = 0, $iCase = 0, $iCompare = 0, $iForward = 1, $iSubItem = -1)
    If Not IsArray($avArray) Then Return SetError(1, 0, -1)
    If UBound($avArray, 0) > 2 Or UBound($avArray, 0) < 1 Then Return SetError(2, 0, -1)

    Local $iUBound = UBound($avArray) - 1

    ; Bounds checking
    If $iEnd < 1 Or $iEnd > $iUBound Then $iEnd = $iUBound
    If $iStart < 0 Then $iStart = 0
    If $iStart > $iEnd Then Return SetError(4, 0, -1)

    ; Direction (flip if $iForward = 0)
    Local $iStep = 1
    If Not $iForward Then
        Local $iTmp = $iStart
        $iStart = $iEnd
        $iEnd = $iTmp
        $iStep = -1
    EndIf

    ; same var Type of comparison
    Local $iCompType = False
    If $iCompare = 2 Then
        $iCompare = 0
        $iCompType = True
    EndIf

    ; Search
    Switch UBound($avArray, 0)
        Case 1 ; 1D array search
            If Not $iCompare Then
                If Not $iCase Then
                    For $i = $iStart To $iEnd Step $iStep
                        If $iCompType And VarGetType($avArray[$i]) <> VarGetType($vValue) Then ContinueLoop
                        If $avArray[$i] = $vValue Then Return $i
                    Next
                Else
                    For $i = $iStart To $iEnd Step $iStep
                        If $iCompType And VarGetType($avArray[$i]) <> VarGetType($vValue) Then ContinueLoop
                        If $avArray[$i] == $vValue Then Return $i
                    Next
                EndIf
            Else
                For $i = $iStart To $iEnd Step $iStep
                    If StringInStr($avArray[$i], $vValue, $iCase) > 0 Then Return $i
                Next
            EndIf
        Case 2 ; 2D array search
            Local $iUBoundSub = UBound($avArray, 2) - 1
            If $iSubItem > $iUBoundSub Then $iSubItem = $iUBoundSub
            If $iSubItem < 0 Then
                ; will search for all Col
                $iSubItem = 0
            Else
                $iUBoundSub = $iSubItem
            EndIf

            For $j = $iSubItem To $iUBoundSub
                If Not $iCompare Then
                    If Not $iCase Then
                        For $i = $iStart To $iEnd Step $iStep
                            If $iCompType And VarGetType($avArray[$i][$j]) <> VarGetType($vValue) Then ContinueLoop
                            If $avArray[$i][$j] = $vValue Then Return $i
                        Next
                    Else
                        For $i = $iStart To $iEnd Step $iStep
                            If $iCompType And VarGetType($avArray[$i][$j]) <> VarGetType($vValue) Then ContinueLoop
                            If $avArray[$i][$j] == $vValue Then Return $i
                        Next
                    EndIf
                Else
                    For $i = $iStart To $iEnd Step $iStep
                        If StringInStr($avArray[$i][$j], $vValue, $iCase) > 0 Then Return $i
                    Next
                EndIf
            Next
        Case Else
            Return SetError(7, 0, -1)
    EndSwitch

    Return SetError(6, 0, -1)
EndFunc   ;==>_ArraySearch

[jchd]

I never advise to modify standard includes. Rather copy the source function and rename it to something else, then change it to suit your needs.

But since you seem to be using SQLite already in your project(s) what I'd consider is storing the array(s) in temporary (or not) table(s) then using the power and speed of SQLite to search and access wanted data in very flexible ways: wildcards, casing, collation, complex sorting conditions, eventually FTS4 search, etc.

I've writen a little SQLite extension for full Unicode support of many string operations among other things, including a fuzzy search. The extension is freely downloadable here and comes with full C source and ready to use 32-bit DLL.

If you need guidance on how to use SQLite extension, look at

[PhoenixXL]

expandcollapse popup

#include <Array.au3>

Local $aTest_Array[3] = ['Testing`_Temp.jpg', 'JOHN.jpeg', 'Boshe.jpeg']
Local $Ret_Array
$Ret_Array = _ArrayFindAllEx($aTest_Array, '*_Temp.jp?g|*h?.jp?g', True, 'Test*.jp?g|b*.jpeg', False)
_ArrayDisplay($Ret_Array, @extended & ' Match')

#cs
Wildcards
*  - Zero or more character
+  - One or more character
?  - No or one character
#ce
; #FUNCTION# ====================================================================================================================
; Name ..........: _ArrayFindAllEx
; Description ...: Similar to _ArrayFindAll with Include, Exclude masks
; Syntax ........: _ArrayFindAllEx(Const Byref $aArray, $sIncludeMask[, $fIncSenstive = True[, $sExcludeMask = ''[,
;                  $fExcSenstive = True]]])
; Parameters ....: $aArray              - [in/out and const] The Array to search for the values.
;                  $sIncludeMask        - A string value.
;                  $fIncSenstive        - [optional] A boolean value. Default is True.
;                  $sExcludeMask        - [optional] A string value. Default is ''.
;                  $fExcSenstive        - [optional] A boolean value. Default is True.
; Return values .: Sucess - Returns the array of the Index of the found values
; - @extended is set to the number of items found
;    Failure  -  Returns '' and sets @error to 1
; Author ........: Phoenix XL
; Modified ......:
; Remarks .......:
; Related .......: _ArrayFindAll
; Link ..........:
; Example .......: Yes
; ===============================================================================================================================
Func _ArrayFindAllEx(ByRef Const $aArray, $sIncludeMask, $fIncSenstive = True, $sExcludeMask = '', $fExcSenstive = True)
;Set Sensitivity Values
If $fIncSenstive Then
$fIncSenstive = ''
Else
$fIncSenstive = '(?i)'
EndIf
If $fExcSenstive Then
$fExcSenstive = ''
Else
$fExcSenstive = '(?i)'
EndIf

;Make the Include Mask Pattern
$sIncludeMask = StringRegExpReplace($sIncludeMask, '(?s)([^\w|])', '[\1]')
$sIncludeMask = StringRegExpReplace($sIncludeMask, '(?s)(\[\?\])', '.?')
$sIncludeMask = StringRegExpReplace($sIncludeMask, '(?s)(\[\+\])', '.+')
$sIncludeMask = $fIncSenstive & '(?s)\A(' & StringRegExpReplace($sIncludeMask, '(?s)(\[\*\])', '.*') & ')\z'

Local $aRet[1]
;Debug Out Include Mask Patterns
ConsoleWrite($sIncludeMask & @CR)

;Get the to-include Strings
For $i = 0 To UBound($aArray) - 1
If StringRegExp($aArray[$i], $sIncludeMask) Then _ArrayAdd($aRet, $i)
Next
_ArrayDelete($aRet, 0)
If Not IsArray($aRet) Then Return 0
If Not $sExcludeMask Then Return SetExtended( UBound( $aRet ), $aRet )

;Make the Exclude Mask Pattern
$sExcludeMask = StringRegExpReplace($sExcludeMask, '(?s)([^\w|])', '[\1]')
$sExcludeMask = StringRegExpReplace($sExcludeMask, '(?s)(\[\?\])', '.?')
$sExcludeMask = StringRegExpReplace($sExcludeMask, '(?s)(\[\+\])', '.+')
$sExcludeMask = $fExcSenstive & '(?s)\A(' & StringRegExpReplace($sExcludeMask, '(?s)(\[\*\])', '.*') & ')\z'

;Debug Out Exclude Mask Patterns
ConsoleWrite($sExcludeMask & @CR)
Local $nDeleted = 0
;Delete the to-exclude strings
For $i = 0 To UBound($aRet) - 1
If StringRegExp($aArray[$aRet[$i - $nDeleted]], $sExcludeMask) Then $nDeleted += Number(_ArrayDelete($aRet, $i - $nDeleted) > 0)
Next
;Done
Return SetError(Not IsArray($aRet), UBound($aRet), $aRet)

EndFunc   ;==>_ArrayFindAllEx

Thumbs up if it helped Edited January 6, 2013 by PhoenixXL

[michaelslamet]

PhoenixXL, awesome, awesome, awesome!

This is exactly what I need! Thanks!

But it has a small glitch and I dont know how to fix it

Using this search criteria:

total*.exe

will also give result:

abctotal.exe

xxxxxxtotalxxxx.exe

which it shouldn't.

It should only give result like:

totalxxxxxx.exe

How to fix this?

[michaelslamet]

jchd, thanks a lot

Yes, I wont edit the standard include, I will save it as another file name

As for SQLite, i never try it before, currently database engine I "know" (well, I dont really know them) only dbase, foxpro and mysql. Thanks for remind me about SQLite. I read about it few times some time ago, but never really know how it different with mysql and in what situation it fit best. I just googling again and get some knowledge about it.

Currently I still dont need SQLite in my project (i think) but if the files and folders grow into too big list, i will use SQLite as you adviced.

For now I only need to read those files and save them to an array, then search into that array when needed, it will improve speed as currently I search those files to hdd everytime I need them.

Thanks again.

[PhoenixXL]

Fixed that bug, please recheck that post

[BrewManNH]

BTW, the _ArrayFindAll function allows you to do a partial string search with the $iCompare parameter set to 1, if you have John, Joe, Sam and Helen in your array and search for "Jo", you'll find John and Joe.

Edited January 5, 2013 by BrewManNH

[michaelslamet]

PhoenixXL, excellent! it works great!

I just realized it will be much better if it accept more than 1 search criteria and can accept exclude list criterias more than 1. Something like this:

$Ret_Array = _ArrayFindAllEx($aTest_Array, 'total*.txt|total*.exe|total*.jpg','*from-webaa*.jpg|*from-webbb*.jpg', False)

Means search in $aTest_Array something that match total*.txt and total*.exe and total*.jpg but do not want totalblablblafrom-webaablabla.jpg and do not want totalblablblafrom-webbbblabla.jpg

is it possible?

Edited January 6, 2013 by michaelslamet

[michaelslamet]

BTW, the _ArrayFindAll function allows you to do a partial string search with the $iCompare parameter set to 1, if you have John, Joe, Sam and Helen in your array and search for "Jo", you'll find John and Joe.

Hi BrewManNH,

Yes, I'm aware about it But with standard _ArrayFindAll function we cant search something like this: total*.txt (only total*.txt do not want total*.exe for example)

[PhoenixXL]

is it possible?

For Sure

Added that feature,

check the original post

Edited January 6, 2013 by PhoenixXL

[michaelslamet]

PhoenixXL, looks great! But I found this small bug:

$iIncSenstive and $iExcSenstive should be $fIncSenstive and $fExcSenstive

and

this code give no search result where is should give 1 search result (Boshe.jpeg)

#include <Array.au3>


Local $aTest_Array[3] = ['Testing`_Temp.jpg', 'john.jpeg', 'Boshe.jpeg']
Local $Ret_Array
_ArrayDisplay($aTest_Array)

;Case-InSensitive
$Ret_Array = _ArrayFindAllEx($aTest_Array, '*_Temp.jp?g|*h?.jp?g',  False, 'Test*.jp?g|jo*', False)
_ArrayDisplay($Ret_Array, @extended & ' Match')

and could you please point me a good tutorial or place to start to learn about RegEx in AutoIT? It looks very powerful!

Thanks and have a great weekend

[PhoenixXL]

Removed the mentioned bugs

could you please point me a good tutorial or place to start to learn about RegEx

For learning RegEx, effort is necessary.

Try to learn and understand from examples available,

thereafter make your own.

For a start look at my signature

Regards

Phoenix XL

Edited January 6, 2013 by PhoenixXL

[jchd]

AutoIt uses a regexp library named PCRE. The version included is a little bit outdated but differences are for advanced users.

The best PCRE regexp tutorial available online I know of is online here.

You will also greatly benefit reading the official PCRE documentation available from the PCRE website.

Edited January 6, 2013 by jchd

[michaelslamet]

PhoenixXL, sorry for trouble you again

Notice another bug.

This code should return no result but it return 1 result, seems like the include mask case-sensitive filter doesn't work.

The exclude mask case-sensitive also doesn't work

Local $aTest_Array[3] = ['Testing`_Temp.jpg', 'JOHN.jpeg', 'Boshe.jpeg']
Local $Ret_Array
$Ret_Array = _ArrayFindAllEx($aTest_Array, '*_Temp.jp?g|*h?.jp?g', True, 'Test*.jp?g|b*.jpeg', False)

Also when there is no exclude mask, it doesn't return @extended value, so I replace this line:

If Not $sExcludeMask Then Return $aRet

with this:

If Not $sExcludeMask Then Return SetError(Not IsArray($aRet), UBound($aRet), $aRet)

Is that correct?

Thanks a lot!!

[PhoenixXL]

again updated

Hope so now its clean

Edited January 6, 2013 by PhoenixXL

[michaelslamet]

jchd, thanks a lot for pointing me the RegEx tutorial website. I'm going to learn about it.

Last night I download a PCRE toolkit by GeoSoft, but I guess I need to learn RegEx first before know how to use the toolkit.

I hope some day I can understand and write my own RegEx

[michaelslamet]

PhoenixXL, one word: AWESOME!

You have no idea how much the function help me.

I think you should put the function in your signature, I believe others will need it too.

I modify it a little bit to suit my need. Since my array is consist of complete path+filename and I need to search only againts the filename, I filter it out using GetFileName function

expandcollapse popup

; #FUNCTION# ====================================================================================================================
; Name ..........: _ArrayFindAllEx
; Description ...: Similar to _ArrayFindAll with Include, Exclude masks
; Syntax ........: _ArrayFindAllEx(Const Byref $aArray, $sIncludeMask[, $fIncSenstive = True[, $sExcludeMask = ''[,
;                $fExcSenstive = True]]])
; Parameters ....: $aArray           - [in/out and const] The Array to search for the values.
;                $sIncludeMask   - A string value.
;                $fIncSenstive   - [optional] A boolean value. Default is True.
;                $sExcludeMask   - [optional] A string value. Default is ''.
;                $fExcSenstive   - [optional] A boolean value. Default is True.
;                $fCheckFileNameOnly - [optional] A boolean value. Default is False. If True then the $aArray is a complete path+filename and we want to use the filename part only for searching
; Return values .: Sucess - Returns the array of the Index of the found values
; - @extended is set to the number of items found
; Failure - Returns '' and sets @error to 1
; Author ........: Phoenix XL
; Modified ......:
; Remarks .......:
; Related .......: _ArrayFindAll
; Link ..........:
; Example .......: Yes
; ===============================================================================================================================
Func _ArrayFindAllEx(ByRef Const $aArray, $sIncludeMask, $fIncSenstive = True, $sExcludeMask = '', $fExcSenstive = True, $fCheckFileNameOnly = False)
;Set Sensitivity Values
If $fIncSenstive Then
$fIncSenstive = ''
Else
$fIncSenstive = '(?i)'
EndIf
If $fExcSenstive Then
$fExcSenstive = ''
Else
$fExcSenstive = '(?i)'
EndIf

;Make the Include Mask Pattern
$sIncludeMask = StringRegExpReplace($sIncludeMask, '(?s)([^\w|])', '[\1]')
$sIncludeMask = StringRegExpReplace($sIncludeMask, '(?s)(\[\?\])', '.?')
$sIncludeMask = StringRegExpReplace($sIncludeMask, '(?s)(\[\+\])', '.+')
$sIncludeMask = $fIncSenstive & '(?s)\A(' & StringRegExpReplace($sIncludeMask, '(?s)(\[\*\])', '.*') & ')\z'

Local $aRet[1]
;Debug Out Include Mask Patterns
ConsoleWrite($sIncludeMask & @CR)

;Get the to-include Strings
For $i = 0 To UBound($aArray) - 1
If $fCheckFileNameOnly Then
If StringRegExp(GetFileName($aArray[$i]), $sIncludeMask) Then _ArrayAdd($aRet, $i)
Else
If StringRegExp($aArray[$i], $sIncludeMask) Then _ArrayAdd($aRet, $i)
EndIf
Next
_ArrayDelete($aRet, 0)
If Not IsArray($aRet) Then Return 0
If Not $sExcludeMask Then Return SetExtended( UBound( $aRet ), $aRet )

;Make the Exclude Mask Pattern
$sExcludeMask = StringRegExpReplace($sExcludeMask, '(?s)([^\w|])', '[\1]')
$sExcludeMask = StringRegExpReplace($sExcludeMask, '(?s)(\[\?\])', '.?')
$sExcludeMask = StringRegExpReplace($sExcludeMask, '(?s)(\[\+\])', '.+')
$sExcludeMask = $fExcSenstive & '(?s)\A(' & StringRegExpReplace($sExcludeMask, '(?s)(\[\*\])', '.*') & ')\z'

;Debug Out Exclude Mask Patterns
ConsoleWrite($sExcludeMask & @CR)
Local $nDeleted = 0
;Delete the to-exclude strings
For $i = 0 To UBound($aRet) - 1
If $fCheckFileNameOnly Then
If StringRegExp(GetFileName($aArray[$aRet[$i - $nDeleted]]), $sExcludeMask) Then $nDeleted += Number(_ArrayDelete($aRet, $i - $nDeleted) > 0)
Else
If StringRegExp($aArray[$aRet[$i - $nDeleted]], $sExcludeMask) Then $nDeleted += Number(_ArrayDelete($aRet, $i - $nDeleted) > 0)
EndIf
Next
;Done
Return SetError(Not IsArray($aRet), UBound($aRet), $aRet)

EndFunc ;==>_ArrayFindAllEx

; #FUNCTION# ======================================================================================================
; Name...........: GetFileName
; Description ...: Returns the file name of the given file path
; Syntax.........: GetFileName($sFilePath)
; Parameters ....: $sFilePath - File path
; Return values .: Success - The file name
;                Failure - -1, sets @error to:
;                |1 - $sFilePath is not a string
; Author ........: Renan Maronni <renanmaronni@hotmail.com>
; Modified.......:
; Remarks .......:
; Related .......:
; Link ..........:
; Example .......:
; ===================================================================================================================

Func GetFileName($sFilePath)

Local $aFolders = ""
Local $FileName = ""
Local $iArrayFoldersSize = 0

If (Not IsString($sFilePath)) Then
     Return SetError(1, 0, -1)
EndIf

$aFolders = StringSplit($sFilePath, "\")
$iArrayFoldersSize = UBound($aFolders)
$FileName = $aFolders[($iArrayFoldersSize - 1)]

Return $FileName

EndFunc ;==>GetFileName

Thanks a lot!

Edited January 7, 2013 by michaelslamet

[jchd]

Right! Shame on me: I forgot to mention Geosoft good PCRE toolbox tool. Its main advantage is that it uses the same regexp engine and version as you do, being coded with AutoIt. So you can be sure that if a pattern works in the tool, then it will work in your program.

In comparison most other tools are potentially deceptive since they use other flavors of regexp.

Its little drawback is that it behaves as a baby CPU hog.