Melba23

Beta _FileListToArrayRec - added functionalities

16 posts in this topic

#1 ·  Posted (edited)

Hi,

This recent thread brought up the subject of allowing user regexes within _FileListToArrayRec. guinness then suggested using a callback function so that users could do even more specific selection. As a result I have developed a Beta version of the UDF adding such functionalities - and also added the option to return a 2D array when using a custom function, something that has been a frequent request in the past. ;)

The main changes to the UDF code are as follows:

 

- 1. The UDF will now accept a user-defined regex in any of the 3 sections of the $sMask parameter - such a regex must be enclosed in double colons ( ::regex:: ) to distinguish it from a normal path/wildcard string. No error checking is done on the regex itself - the user must ensure that it is correct.

- 2. The main UDF will now accept a function in place of the $sMask parameter and internally has separate sections for running this "user function" and "normal" code to avoid any slowdown when used without a user function - combining the two meant that at least one If had to be added to each item parsed, which increased the execution time by about 5%. Note that when used with a user function, the $sMask parameter is forced to "*" (all items on the path with no exclusions) and the $iReturn parameter only defines whether files and/or folders are to be returned (the user function must deal with all other restrictions). The $iRecur parameter still works as before, but the $iSort & $iReturnPath parameters are ignored (the former because it can only work on 1D arrays (see point 4 below), while the user function must determine the latter). This was done to ensure that the internal UDF code runs as fast as possible because calling the user function for each returned item can obviously add significantly to the overall execution time.

- 3. The user function must accept 3, and only 3, parameters (the full path, name and attributes of the item) and must return the data to be included in the final returned array - if nothing is to be added, then the @error macro must be set. Otherwise there is no restriction on the user function itself - although as it is run on every item on the path, it should be as fast as possible.

- 4. If a user function is passed to the UDF, an added UDF parameter ($iUserRet_Cols) comes into play. This determines the number of columns in the returned array - the UDF defaults to a 1D array but setting the parameter to a higher value will return a 2D array with that number of columns. Obviously the user function must return suitably formatted data to be added to the return array - a simple string if the return array is 1D; an array with sufficent elements to fill the columns of a 2D return array. Note that if the number of elements in the array passed by the user function does not match the defined return array, then nothing is added.

As a result of all thesse changes, there is almost no change to the execution speed of the UDF in "normal" mode, the "direct regex" mode can make the return list more accurate, and there is now a very flexible "user function" option allowing experienced users to fine-tune the return and add further data. :)

Here is the Beta function with a short example included:

FLTAR_SRE_and_Func.au3

Comments on the utilty and implementation of the new functionalities would be most welcome. :)

M23

Edit: Now with regex syntax checking. :)

Edited by Melba23

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites



#2 ·  Posted (edited)

As a base you used this function from 3.3.13.19 ? or not relased yet 3.3.13.20 ? (i do not remember if there was any changes in 3.3.13.20)

I want to compare changes with WinMerge.

edit:

... for learning and for testing.

Edited by mLipok

Signature beginning:   Wondering who uses AutoIT and what it can be used for ?
* GHAPI UDF - modest begining - comunication with GitHub REST API *
ADO.au3 UDF     POP3.au3 UDF     XML.au3 UDF    How to use IE.au3  UDF with  AutoIt v3.3.14.x  for other useful stuff click the following button

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST API *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 - BETA * ADO.au3 UDF SMTP Mailer UDF *

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Best coding practices * 

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * 

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

Signature last update: 2017-06-04

Share this post


Link to post
Share on other sites

mLipok,

The base code was the current repository version of the function - so I suppose you could say 3.3.13.20. But I do not believe that WinMerge will help you too much - when I tried a comparison it shows that almost everything was changed whereas most of the "changed" code was just moved inside If..EndIf structures. Better to work through the code and see for yourself. ;)

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

Since I only recently updated to Windows 7, I still don't have much software installed on this machine: it's quite complicated maintaining legacy software but I have no choice right now. Anyway, I just installed beta and, after some inconvenience (AV deleted the exe), I took a look at this. I think these are quite cool features. :)

Edited by czardas

Share this post


Link to post
Share on other sites

jguinch & czardas,

Thanks for the feedback. :thumbsup:

At time of posting 23 downloads are shown - anyone else from among those members feel like commenting? Or does anyone else feel like downloading and testing? :huh:

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

I was just wondering what might be the consequences of someone entering a badly formatted regexp. I don't suppose there's a way to catch such an error, or is there?

Share this post


Link to post
Share on other sites

czardas,

I did think about that when rewriting the UDF but decided that parsing the regex within the UDF would be too great a burden. However, rethinking the matter I feel that perhaps running the passed regex through a simple StringRegExp call and checking for errors might might be a way to do a basic sanity check - although I still believe that we must leave most of the regex responsibility with the coder. I will see what can be done. :)

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

#9 ·  Posted (edited)

I hadn't thought so deeply about it, but it occurred to me that no checks at all may cause a number of issues. I was thinking if it was possible to circumvent some such problems, it would be preferable. I haven't looked in any detail at the code, I just remember you mentioning something like this earlier on.

Edited by czardas

Share this post


Link to post
Share on other sites

czardas,

Something like this would be easy to add:

; Check if regex passed
If StringLeft($sList, 2) = "::" And StringRight($sList, 2) = "::" Then
    ; Set it directly
    $vMask = StringTrimRight(StringTrimLeft($sList, 2), 2)
    ; Run basic check on regex pattern
    StringRegExp("::", $vMask)
    ; If pattern error
    If @error = 2 Then Return 0
The UDF would fail (with the existing parameter-linked code in @extended) if a pattern syntax error is detected. Obviously this is no guarantee that a regex will return what the user intended, but will at least check that the syntax is legal. :)

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

#11 ·  Posted (edited)

It's probably the best you can really hope for. I thnk it's definately worth considering. Throw some bad patterns at it and see if it breaks anything.

Edited by czardas

Share this post


Link to post
Share on other sites

czardas,

 

Throw some bad patterns at it and see if it breaks anything

What do you think I did before posting? :P

Now if you use a regex pattern with a syntax error that returns @error 2 (as per the StringRegExp page in the Help file) then the UDF fails with @error/@extended pointing to the parameter that gave the error. Best that can be done I feel. :)

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

#14 ·  Posted

Hello,

My first post ;-)

Thank you for this usefull function.

My 2 cents: I tryed the following regexp: "::^.*(A01|EKI)[ -].*\..*$::" as a pattern in your function that gives an error but this one  "::^.*A01[ -].*\..*$::" doesn't.

So I was wondering if the regexp's "OR" symbolysed by | was available or if I did a regexp syntax error ?

Thanks

 

Share this post


Link to post
Share on other sites

#15 ·  Posted (edited)

  H2Fooko,

Welcome to the AutoIt forums.

What error is returned by the function? When I test those patterns I get no errors.

M23

Edit: I see the problem - the "|" is being interpreted as a parameter delimiter and not as a RegEx "OR". I will look into the problem.

Edited by Melba23

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

#16 ·  Posted (edited)

H2Fooko,

I have a solution, but it requires the user to make a small change to the syntax used for the RegEx. If "|" (OR) is used within a RegEx, it must be doubled to "||" so that the UDF can distinguish it from the standard parameter delimiter. So your RegEx would read:

The UDF then replaces any "||" found with a user-defined Unicode character (unlikely to be found in a filename), splits the entire string on any "|" delimiters, and finally resets any Unicode characters back into a single "|" so that the RegEx functions correctly within the PCRE engine.

This amended code works for me, can you please check that it works for you too?

And do you consider that the proposed solution of doubling the "OR" character is acceptable from a user's standpoint?

I have a better solution. The UDF now parses the entire parameter replacing any "|" (OR) characters within a RegEx (defined as being bounded by "::") with a user-defined Unicode character as explained above so that the parameter can be correctly split on the delimiters. Once split, the user-defined Unicode character can then be changed back to get a working RegEx. This means that the RegEx syntax can remain as expected.

Please check that this version works for you as well as it does for me: FLTAR_SRE_and_Func.au3

M23

Edit: Added a bit more error-checking when looking for RegExes within the parameter.

Edited by Melba23

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now