Jump to content
Tosyk

[solved] copy file if found in array

Recommended Posts

I'm making a script to search items from one array (small one) inside another (big one).

item in small array - partial path to file (attached civ_female_act3.model_materials.txt)
item in big array - full path to file (attached TextureList.lst)

if item from small array is found in big one then file need to be copied from the full path into another folder. I'm searching for $find atm. Script:

#include <Array.au3>
#include <File.au3>

Global $aArrayInput, $aArrayOutput, $matArray
Global $sDir = @ScriptDir
Global $sFileName = '*'
Global $sFileExtension = 'texture.dds' ; or '*' for all files or 'pdf' for just PDFs
Global $sListPath = @ScriptDir & '\TextureList.lst'
Global $find = 'textures\cperrella\fabric\cloth_knits\wrinkles_01_n.texture' & '.dds'
Global $matInput = @ScriptDir & '\civ_female_act3.model_materials.txt'
Global $sExclude

$aArrayOutput = _FileListToArrayRec( $sDir, $sFileName & '.' & $sFileExtension, 1, 1, 0, 2 )
If IsArray($aArrayOutput) Then
   If FileExists($sListPath) Then
      _FileSearchInTXT()
   Else
      _FileWriteFromArray($sListPath, $aArrayOutput)
      _FileSearchInTXT()
;~       _ArrayDisplay( $aArrayOutput )
   EndIf
EndIf

_FileReadToArray($matInput, $matArray, $FRTA_COUNT)
_ArrayDelete($matArray, $matArray[1])

$sExclude = "===*|---*|shader*|material*| *"

$sExclude = StringReplace($sExclude, ".", "\.")
$sExclude = StringReplace($sExclude, "?", ".")
$sExclude = StringReplace($sExclude, "*", ".*?")

$iIndex = 0
For $i = 0 To UBound($matArray) - 1
    If NOT StringRegExp($matArray[$i], $sExclude) Then
        $matArray[$iIndex] = $matArray[$i]
        $iIndex += 1
    EndIf
Next
Redim $matArray[$iIndex]


For $i = UBound($matArray) - 1 To 0 Step -1
    If $matArray[$i] = "" Then
        _ArrayDelete($matArray, $i)
    EndIf
Next

_ArrayTrim($matArray, 17, 0)
_ArrayDisplay($matArray, "2D array - no count (trimmed)", Default, 8)


;~ ----------------------------------------------------------------------------------------------
Func _FileSearchInTXT()
If $sListPath <> '' And $find <> '' Then
    _FileReadToArray($sListPath, $aArrayInput)
    For $i = 1 To UBound($aArrayInput) - 1
        If StringInStr($aArrayInput[$i], $find) Then
           ConsoleWrite("Found file: " & $aArrayInput[$i] & @CRLF)
        EndIf
     Next
Else
    MsgBox(48, 'Error', 'A file was not picked or what to find was cancelled/empty!')
EndIf
EndFunc    ;==>_FileSearchInTXT

please help me to do this.

 

TextureList.zip

Edited by Tosyk

Share this post


Link to post
Share on other sites

And your question is?


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2020-10-10 - Version 1.5.2.1) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2020-12-15 - Version 1.6.3.1) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX_GUI (2020-06-27 - Version 1.3.2.0) - Download
Outlook Tools (2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
PowerPoint (2017-06-06 - Version 0.0.5.0) - Download - General Help & Support
Excel - Example Scripts - Wiki
Word - Wiki
Task Scheduler (2019-12-03 - Version 1.5.1.0) - Download - General Help & Support - Wiki

Tutorials:
ADO - Wiki, WebDriver - Wiki

 

Share this post


Link to post
Share on other sites
41 minutes ago, Tosyk said:

if item from small array is found in big one then file need to be copied from the full path into another folder

this is the idea, sorry for not been clear :) right now it's just display the small array and saves big array as file.

here's where I get small array:

_ArrayDisplay($matArray, "2D array - no count (trimmed)", Default, 8)

after this I need to check all the items from small array ($matArray) inside big array ($aArrayOutput) and if string is found copy the file into another folder. I don't understand how to do it

Share this post


Link to post
Share on other sites

We need more information:

  • About how many records in the small and large array do we talk?
  • Is one/both of the arrays sorted? If yes, we could do a binary search to enhance processing speed

My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2020-10-10 - Version 1.5.2.1) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2020-12-15 - Version 1.6.3.1) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX_GUI (2020-06-27 - Version 1.3.2.0) - Download
Outlook Tools (2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
PowerPoint (2017-06-06 - Version 0.0.5.0) - Download - General Help & Support
Excel - Example Scripts - Wiki
Word - Wiki
Task Scheduler (2019-12-03 - Version 1.5.1.0) - Download - General Help & Support - Wiki

Tutorials:
ADO - Wiki, WebDriver - Wiki

 

Share this post


Link to post
Share on other sites

Basic comparison logic would be as follows (untested), but as @water said, there are better and more efficient ways of doing it.

For $i = 0 To UBound($smallArray - 1)
    For $j = 0 To UBound($bigArray - 1)
        If $smallArray[$i] = $bigArray[$j] Then
            FileCopy("Path\To\Source\" & $smallArray[$i], "Path\To\Destination\Filename.file", $FC_NOOVERWRITE)
        Else
            ConsoleWrite("No match found for " & $smallArray[$i] & @CRLF)
        EndIf
    Next
Next

 

Share this post


Link to post
Share on other sites

small array is up to 1000 items, big one can be around 50000 items

also, I'm sure there's a cool and speed solution. is just I don't really a programmer person so I'm trying to do a clear code :)

Edited by Tosyk

Share this post


Link to post
Share on other sites

If speed doesn't matter, then I would start with Sidleys solution.
You need to decide what to do when the copied file already exists in the target directory. Currently an existing file doesn't get overwritten.


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2020-10-10 - Version 1.5.2.1) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2020-12-15 - Version 1.6.3.1) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX_GUI (2020-06-27 - Version 1.3.2.0) - Download
Outlook Tools (2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
PowerPoint (2017-06-06 - Version 0.0.5.0) - Download - General Help & Support
Excel - Example Scripts - Wiki
Word - Wiki
Task Scheduler (2019-12-03 - Version 1.5.1.0) - Download - General Help & Support - Wiki

Tutorials:
ADO - Wiki, WebDriver - Wiki

 

Share this post


Link to post
Share on other sites

I change the solution like this:

For $i = 0 To UBound($matArray - 1)
    For $j = 0 To UBound($aArrayOutput - 1)
        If $matArray[$i] = $aArrayOutput[$j] Then
            FileCopy($aArrayOutput[$j], @ScriptDir, $FC_NOOVERWRITE)
        Else
            ConsoleWrite("No match found for " & $matArray[$i] & @CRLF)
        EndIf
    Next
Next

but it returned:

...
No match found for 
No match found for 
No match found for 
"d:\MyTools\tex_collector.au3" (54) : ==> Array variable has incorrect number of subscripts or subscript dimension range exceeded.:
If $matArray[$i] = $aArrayOutput[$j] Then
If $matArray[$i] = ^ ERROR
>Exit code: 1    Time: 5.851

 

Edited by Tosyk

Share this post


Link to post
Share on other sites

Can you please post an example of the small and large array (if you like as files) so we can play with your script?


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2020-10-10 - Version 1.5.2.1) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2020-12-15 - Version 1.6.3.1) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX_GUI (2020-06-27 - Version 1.3.2.0) - Download
Outlook Tools (2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
PowerPoint (2017-06-06 - Version 0.0.5.0) - Download - General Help & Support
Excel - Example Scripts - Wiki
Word - Wiki
Task Scheduler (2019-12-03 - Version 1.5.1.0) - Download - General Help & Support - Wiki

Tutorials:
ADO - Wiki, WebDriver - Wiki

 

Share this post


Link to post
Share on other sites

already did in the first post:

item in small array - attached civ_female_act3.model_materials.txt
item in big array - attached TextureList.lst

Share this post


Link to post
Share on other sites

What is this part of your scipt meant for?
Do you want to exclude all lines containing the specified pattern $sExclude?

$sExclude; Lines with certain strings should be excluded
$sExclude = "===*|---*|shader*|material*| *"
$sExclude = StringReplace($sExclude, ".", "\.")
$sExclude = StringReplace($sExclude, "?", ".")
$sExclude = StringReplace($sExclude, "*", ".*?")

; Remove all lines 
$iIndex = 1
For $i = 0 To UBound($matArray) - 1
    If Not StringRegExp($matArray[$i], $sExclude) Then
        $matArray[$iIndex] = $matArray[$i]
        $iIndex += 1
    EndIf
Next
ReDim $matArray[$iIndex]

 


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2020-10-10 - Version 1.5.2.1) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2020-12-15 - Version 1.6.3.1) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX_GUI (2020-06-27 - Version 1.3.2.0) - Download
Outlook Tools (2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
PowerPoint (2017-06-06 - Version 0.0.5.0) - Download - General Help & Support
Excel - Example Scripts - Wiki
Word - Wiki
Task Scheduler (2019-12-03 - Version 1.5.1.0) - Download - General Help & Support - Wiki

Tutorials:
ADO - Wiki, WebDriver - Wiki

 

Share this post


Link to post
Share on other sites

@water

as input (and small array) I have $matArray, here's list of actions by which I clear this array of redundant lines:

;~ ----------------------------------------------------------------------------------------------
;~ CLEARING matArray START
;~ ----------------------------------------------------------------------------------------------
$sExclude = "===*|---*|shader*|material*| *"

$sExclude = StringReplace($sExclude, ".", "\.")
$sExclude = StringReplace($sExclude, "?", ".")
$sExclude = StringReplace($sExclude, "*", ".*?")

$iIndex = 0
For $i = 0 To UBound($matArray) - 1
    If NOT StringRegExp($matArray[$i], $sExclude) Then
        $matArray[$iIndex] = $matArray[$i]
        $iIndex += 1
    EndIf
Next
Redim $matArray[$iIndex]


For $i = UBound($matArray) - 1 To 0 Step -1
    If $matArray[$i] = "" Then
        _ArrayDelete($matArray, $i)
    EndIf
Next

_ArrayTrim($matArray, 17, 0)
;~ ----------------------------------------------------------------------------------------------
;~ CLEARING matArray END
;~ ----------------------------------------------------------------------------------------------

 

Share this post


Link to post
Share on other sites
37 minutes ago, water said:

Do you want to exclude all lines containing the specified pattern $sExclude?

yes! also exclude empty lines

Share this post


Link to post
Share on other sites

Those UBounds should read UBound($Variable) - 1, and not UBound($Variable-1), sorry about that. Don't know if it'll make a difference.

Share this post


Link to post
Share on other sites

You want to exclude just lines that START with the pattern or even lines that CONTAIN the patterns?
Right now all lines that CONTAIN "shader" (like "A87F9241FDCE1112textures\default_pink_shader\default_pink_shader_n.texture") get ignored.


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2020-10-10 - Version 1.5.2.1) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2020-12-15 - Version 1.6.3.1) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX_GUI (2020-06-27 - Version 1.3.2.0) - Download
Outlook Tools (2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
PowerPoint (2017-06-06 - Version 0.0.5.0) - Download - General Help & Support
Excel - Example Scripts - Wiki
Word - Wiki
Task Scheduler (2019-12-03 - Version 1.5.1.0) - Download - General Help & Support - Wiki

Tutorials:
ADO - Wiki, WebDriver - Wiki

 

Share this post


Link to post
Share on other sites
14 minutes ago, water said:

You want to exclude just lines that START with the pattern or even lines that CONTAIN the patterns?

this 

14 minutes ago, water said:

Right now all lines that CONTAIN "shader" (like "A87F9241FDCE1112textures\default_pink_shader\default_pink_shader_n.texture") get ignored.

oh, I didn't realize this :/ thanks for point that out

Share this post


Link to post
Share on other sites

As far as I can test this code does what you want it to do.

  • I have modified the script so that all arrays hold the number of rows in the first row.
  • I have modified the PCRE so the search pattern has to be found at column 1 of each record
  • I have added some debugging code. Can easily be removed
#include <Array.au3>
#include <File.au3>

; Documentation:
;   All arrays store the row/column count in row 0

Global $aArrayInput, $aArrayOutput, $aMat
Global $sDir = @ScriptDir
Global $sFileName = '*'
Global $sFileExtension = 'texture.dds' ; or '*' for all files or 'pdf' for just PDFs
Global $sListPath = @ScriptDir & '\TextureList.lst'
Global $sFind = 'textures\cperrella\fabric\cloth_knits\wrinkles_01_n.texture' & '.dds'
Global $sMatInput = @ScriptDir & '\civ_female_act3.model_materials.txt'

$aArrayOutput = _FileListToArrayRec($sDir, $sFileName & '.' & $sFileExtension, 1, 1, 0, 2)
; if @error then Exit ConsoleWrite("_FileListToArrayRec: @error=" & @error & ", IsArray($aArrayOutput): " & IsArray($aArrayOutput) & @CRLF)
If IsArray($aArrayOutput) Then
    If Not FileExists($sListPath) Then _FileWriteFromArray($sListPath, $aArrayOutput, 1) ; Do not write the record count to the file
    _FileSearchInTXT()
Else
    Global $aArrayOutput[1] = [0] ; Modified: $aArrayOutput needs to be defined as an array as _FileListToArray might not return/create the array
EndIf

_FileReadToArray($sMatInput, $aMat, $FRTA_COUNT)
_ArrayDisplay($aMat, "before Moving rows") ; ==> Just for debugging can be removed

; Lines with certain strings should be excluded
Global $sExclude = "(?i)^===*|(?i)^---*|(?i)^shader*|(?i)^material*|(?i)^template*|^\s|^ *"
$sExclude = StringReplace($sExclude, ".", "\.")
; $sExclude = StringReplace($sExclude, "?", ".") ; REMOVED as it would kill (?i) in the pattern
$sExclude = StringReplace($sExclude, "*", ".*?")

; Remove all lines holding on of the $sExclude patterns or empty lines
Global $iOutIndex = 1
For $iInIndex = 1 To $aMat[0]
    If Not StringRegExp($aMat[$iInIndex], $sExclude) And $aMat[$iInIndex] <> "" Then
        ConsoleWrite("Moved  : " & $iInIndex & " | " & $iOutIndex & " | " & $aMat[$iInIndex] & @CRLF)  ; ==> Just for debugging can be removed
        $aMat[$iOutIndex] = $aMat[$iInIndex]
        $iOutIndex += 1
    Else
        ConsoleWrite("Ignored: " & $iInIndex & " | " & $iOutIndex & " | " & $aMat[$iInIndex] & @CRLF) ; ==> Just for debugging can be removed
    EndIf
Next
_ArrayDisplay($aMat, "After moving rows") ; ==> Just for debugging can be removed
ReDim $aMat[$iOutIndex]
$aMat[0] = $iOutIndex - 1
_ArrayDisplay($aMat, "After ReDim") ; ==> Just for debugging can be removed

For $i = 1 To $aMat[0]
    For $j = 1 To $aArrayOutput[0]
        If $aMat[$i] = $aArrayOutput[$j] Then
            ConsoleWrite("Copying file " & $aArrayOutput[$j] & " to " & @ScriptDir & @CRLF) ; ==> Just for debugging can be removed
            FileCopy($aArrayOutput[$j], @ScriptDir, $FC_NOOVERWRITE)
        Else
            ConsoleWrite("No match found for " & $aMat[$i] & @CRLF)
        EndIf
    Next
Next

_ArrayTrim($aMat, 17, 0, 1)
_ArrayDisplay($aMat, "Array (trimmed)", Default, 8)
Exit

;~ ----------------------------------------------------------------------------------------------
Func _FileSearchInTXT()
    If $sListPath <> '' And $sFind <> '' Then
        _FileReadToArray($sListPath, $aArrayInput, $FRTA_COUNT)
        For $i = 1 To $aArrayInput[0]
            If StringInStr($aArrayInput[$i], $sFind) Then
                ConsoleWrite("Found file: " & $aArrayInput[$i] & @CRLF)
            EndIf
        Next
    Else
        MsgBox(48, 'Error', 'A file was not picked or what to find was cancelled/empty!')
    EndIf
EndFunc   ;==>_FileSearchInTXT

 


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2020-10-10 - Version 1.5.2.1) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2020-12-15 - Version 1.6.3.1) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX_GUI (2020-06-27 - Version 1.3.2.0) - Download
Outlook Tools (2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
PowerPoint (2017-06-06 - Version 0.0.5.0) - Download - General Help & Support
Excel - Example Scripts - Wiki
Word - Wiki
Task Scheduler (2019-12-03 - Version 1.5.1.0) - Download - General Help & Support - Wiki

Tutorials:
ADO - Wiki, WebDriver - Wiki

 

Share this post


Link to post
Share on other sites

thanks! I'll try it right now. can you explain what's going on here(?):

ReDim $aMat[$iOutIndex]
$aMat[0] = $iOutIndex - 1
_ArrayDisplay($aMat, "After ReDim") ; ==> Just for debugging can be removed

after this operation I get array almost 2 times shorter

edit:

also, in a small array I have strings like:

characters/civilians/clothing/har_female_knittedcap/textures/har_female_knittedcap_n.texture

but in big array this should be:

D:\_r\to_con\_gm\characters\civilians\clothing\har_female_knittedcap\textures\har_female_knittedcap_n.texture.dds

and script couldn't find the string because of reversed "\". how it can be reversed back or ignored or something else so the string could be found in big array?

Edited by Tosyk

Share this post


Link to post
Share on other sites

The code before ReDim removed all empty lines and lines matching the exclude pattern.
Let's say the array consists of 11 rows (row 0 holds the row count + 10 rows holding your data). 3 rows were removed (empty or matching the exclude pattern). Then the code is reduced to 8 rows (row 0 holds the row count + 7 rows holding your data). The I set the row count in row 0 to 7.


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2020-10-10 - Version 1.5.2.1) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2020-12-15 - Version 1.6.3.1) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX_GUI (2020-06-27 - Version 1.3.2.0) - Download
Outlook Tools (2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
PowerPoint (2017-06-06 - Version 0.0.5.0) - Download - General Help & Support
Excel - Example Scripts - Wiki
Word - Wiki
Task Scheduler (2019-12-03 - Version 1.5.1.0) - Download - General Help & Support - Wiki

Tutorials:
ADO - Wiki, WebDriver - Wiki

 

Share this post


Link to post
Share on other sites
1 hour ago, water said:

Then the code is reduced to 8 rows

why the code doing this after the array is cleared pretty well?

also, script couldn't find the string because of reversed "\". how it can be reversed back or ignored or something else so the string could be found in big array?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...