change FileCopy function a bit

Tosyk · August 24, 2019

Hello, I have a script that do 2 things when you drag TXT file on it:

scan given folder sctructure and creates list of found files
search for files in the created list by given TXT with file names

Here's the an example TXT:

============================================================
ant-plastic-storage-bin-c_tmp0
============================================================
cit-storage-bins-01-color
cit-storage-bins-01-normal_dx10
cit-storage-bins-01-roughness_dx10
missingtexture_dx10

Here's the script:

#cs
;rename .ini as script name or compiled .exe
;put below strings into that .ini
[Setting]
;put 1 if you need to replace files
Replace=0
;put path where to search
Folder=''
;put 1 if you need search in subfolders
SubDir=0
;put 1 if you need to keep file structure
Build=0
;put 1 if you need to update filelist after the process will finish
Update=0
#ce

Global $sIniFile = StringRegExpReplace(@ScriptFullPath, '(^.*)\.(.*)$', '$1')& '.ini'
Global $sFolder = IniRead ($sIniFile, 'Setting', 'Folder', @ScriptDir)
Global $iReplace = Int(IniRead ($sIniFile, 'Setting', 'Replace', 0))
Global $iSubDir = Int(IniRead ($sIniFile, 'Setting', 'SubDir', 0))
Global $iBuild = Int(IniRead ($sIniFile, 'Setting', 'Build', 0))
Global $iUpdate = Int(IniRead ($sIniFile, 'Setting', 'Update', 0))

Global $sListFileSearch = @ScriptDir& '\ListSearch.lst'
Global $aListFileSearch = LoadListSearch_()
Global $sListFileSave = @ScriptDir& '\ListSave.lst'
Global $aListFileSave = LoadListSave_()

If $CmdLine[0] Then
    While $CmdLine[0]
        FileRead_($CmdLine[$CmdLine[0]])
        $CmdLine[0] -= 1
    WEnd
EndIf

SaveListSave_()
If $iUpdate Then
    SaveListSearch_()
EndIf

Func SaveListSearch_()
    Local $_iA, $_hFile = FileOpen($sListFileSearch, 2 +8)
    For $_iA = 1 To $aListFileSearch[0]
        FileWrite($_hFile, $aListFileSearch[$_iA]& @CRLF)
    Next
    FileClose($_hFile)
EndFunc

Func SaveListSave_()
    Local $_iA, $_hFile = FileOpen($sListFileSave, 2 +8)
    For $_iA = 1 To $aListFileSave[0]
        FileWrite($_hFile, $aListFileSave[$_iA]& @CRLF)
    Next
    FileClose($_hFile)
EndFunc

Func FileCopy_($s_FileIn)
    Local $sFileOut
    If $iBuild Then
        $sFileOut = StringReplace($s_FileIn, $sFolder, @ScriptDir& '\Backup')
    Else
        $sFileOut = @ScriptDir& '\Backup\'& StringRegExpReplace($s_FileIn, '(^.*)\\(.*)$', '$2')
    EndIf
    FileCopy($s_FileIn, $sFileOut, $iReplace ? 9 : 8)
EndFunc

Func FileSearch_($_sName)
    Local $_iA, $_sFile
    For $_iA = 1 To $aListFileSearch[0]
        $_sFileName = StringRegExpReplace($aListFileSearch[$_iA], '(^.*)\\(.*)$', '$2')
        $_sFileName = StringRegExpReplace($_sFileName, '(^.*)\.(.*)$', '$1')
        Switch $_sFileName
            Case $_sName, $_sName& '_dx10'
                $aListFileSave[0] += 1
                ReDim $aListFileSave[$aListFileSave[0] +1]
                $aListFileSave[$aListFileSave[0]] = $_sFileName
                FileCopy_($aListFileSearch[$_iA])
                If $iUpdate Then
                    $aListFileSearch[$_iA] = $aListFileSearch[0]
                    ReDim $aListFileSearch[$aListFileSearch[0]]
                    $aListFileSearch[0] -= 1
                EndIf
                Return 1
        EndSwitch
    Next
EndFunc

Func CheckIsSave_($_sName)
    Local $_iA
    For $_iA = 1 To $aListFileSave[0]
        If ($aListFileSave[$_iA] = $_sName) Or ($aListFileSave[$_iA] = $_sName& '_dx10') Then
            Return 1
        EndIf
    Next
    Return 0
EndFunc

Func FileRead_($s_File)
    Local $h_File = FileOpen($s_File)
    Local $_sName, $i_Pos, $_sOldName, $_iStopper
    While 1
        $_sName = FileReadLine($h_File)
        If @error Then
            FileClose($h_File)
            ExitLoop
        EndIf
        If StringLeft($_sName, 9) = 'Material ' Then
            $_iStopper = True
            ContinueLoop
        ElseIf StringLeft($_sName, 1) = '=' Then
            $_iStopper = ($_iStopper = False)
            ContinueLoop
        ElseIf $_iStopper Then
            ContinueLoop
        EndIf
        If StringInStr($_sName, 'TEXT ') Then
            $_sName = StringRegExpReplace($_sName, '(^.*)\TEXT (.*)$', '$2')
            $_sName = StringRegExpReplace($_sName, '(^.*)\ (.*)$', '$1')
        EndIf
        If Not CheckIsSave_($_sName) Then
            FileSearch_($_sName)
        EndIf
    WEnd
EndFunc

Func LoadListSave_()
    Local $_sList
    If FileExists($sListFileSave) Then
        $_sList = FileRead($sListFileSave)
    EndIf
    Return StringSplit($_sList, @CRLF, 1)
EndFunc

Func CreateListSearch_($s_Folder)
    Local $h_Find = FileFindFirstFile($s_Folder& '\*.*')
    Local $s_Find, $_sList
    While 1
        $s_Find = FileFindNextFile($h_Find)
        If @error Then
            ExitLoop
        EndIf
        If StringInStr(FileGetAttrib($s_Folder& '\'& $s_Find) , 'D') Then
            If $iSubDir Then
                $_sList &= CreateListSearch_($s_Folder& '\'& $s_Find)
            EndIf
        Else
            $_sList &= $s_Folder& '\'& $s_Find& @CRLF
        EndIf
    WEnd
    FileClose($h_Find)
    Return $_sList
EndFunc

Func LoadListSearch_()
    Local $_sList
    If Not FileExists($sListFileSearch) Then
        $_sList = CreateListSearch_($sFolder)
        Local $_hFile = FileOpen($sListFileSearch, 2 +8)
        FileWrite($_hFile, $_sList)
        FileClose($_hFile)
    Else
        $_sList = FileRead($sListFileSearch)
    EndIf
    Return StringSplit($_sList, @CRLF, 1)
EndFunc

note: there's a feature in the code which gives possibility to search filenames by adding _dx10 suffix to normal names. That feature searches files with normal names, say Normal_Name.pps, and special names, say Normal_Name_dx10.pps. the problems is that it chooses "suffixed" names to copy but not the normal named files.

I need to have possibility to copy both files. Also would be nice to have a feature to define these two files be size and copy them into different folders.

p.s.; this script was originally posted here: http://autoit-script.ru/threads/avtomatizacija-poisk-fajlov-rekursivno-po-dannomu-spisku.26970/page-2 and there's no answers for long time and I broke my head while trying to find a way to solve that puzzle.

Tosyk · August 25, 2019

can anyone help me? I'm sure there's some little thing but I can't see it :/

FrancescoDiMuro · August 25, 2019

@Tosyk

The script could be easily rewritten, since this seems to be just a little bit outdated.

If you try to rewrite it, you'd probably have less problems than adapt it to your needs

Tosyk · August 25, 2019

3 hours ago, FrancescoDiMuro said:

@Tosyk

The script could be easily rewritten, since this seems to be just a little bit outdated.

If you try to rewrite it, you'd probably have less problems than adapt it to your needs

the thing is I can't wright code, that's why I came here. I can only make some simple operations like copy/move files around

can you help me with it?

FrancescoDiMuro · August 25, 2019

@Tosyk

Tell us what's the end-goal of the script you want to make, and we'll help you out

Tosyk · August 25, 2019

I have a TXT files with list of filenames in each. Also I have a really huge resource database (just a random folder hierarhy) with various of files - mostly textures: DDS/PNG/TGA. my goal is to drop TXT on a script and collect files from database by filename list.

the script I have working perfectly - it creates a list of files from given (by INI file) database path and then searches that database for files TXT wants, then copy found files to the directory I set.

Note: sometimes database files have files with "_dx10" or ".1" suffixes but TXT filenames doesn't have them. In that case script should search normal (without suffix) and advanced (with suffix) files, then define them by size and copy both into different directories.

extension should be ignored - any extension should work

here's example of 2 TXT with file lists:

Material B7E671143D244B
====================================
TEXT 2F3139D816C34D 1
TEXT B6A968EF2505A2 1
TEXT 35206697A04F91 1
TEXT EB485AF490D83D 1
TEXT 0DAB42294BD9B3 1
TEXT 3D6525BEE360E1 0

Material D6906B886B06E3
====================================
TEXT 0CCECCCCFB62AE 1
TEXT 1E14CB29AB43F0 1
TEXT FB7F0DCE9B5950 1

Material 431831490FD5C9
====================================
TEXT 9D77B6474696D8 1
TEXT D04DEE5DF130A4 1
TEXT B6A968EF2505A2 1
TEXT 8C37245A4F0F81 1
TEXT A39FAFFC5ABC06 1
TEXT D24199644F2EE6 1

============================================================
ant-plastic-storage-bin-c_tmp0
============================================================
cit-storage-bins-01-color
cit-storage-bins-01-normal_dx10
cit-storage-bins-01-roughness_dx10
missingtexture_dx10

in that examples all lines with "===" and "Material " should be ignored, also word "TEXT " and number " 1" should be ignored — only hash between is matter

one more thing: path to database need to be configured within INI file AND I need to have an opportunity to drop a number of TXT files on the script.

Edited August 25, 2019 by Tosyk

tarretarretarre · August 26, 2019

Here is a regex that will match your txt file lists according to your needs:

https://regex101.com/r/ktKaBr/1

and here is another to match the other files:

https://regex101.com/r/TKAQTD/2

Look at https://www.autoitscript.com/autoit3/docs/functions/StringRegExp.htm

the regex flags could be put in (?<here>) in the end of your string.

Edited August 26, 2019 by tarretarretarre

FrancescoDiMuro · August 26, 2019

@Tosyk
_FileListToArrayRec() would be a nice solution that fits your needs, since it allows you to set inclusion and exclusion filters, so you can set what you want and what you don't want.
The function can be recursive as well in subfolders, so it will look for specified files in all the subfolders, given a "root" directory.
Then, a simple FileCopy would all the rest.
A simple script would be:

IniRead() to read the settings of the script;
_FileListToArray() in the Database folder, so you get a list of "Database" files;
For Each Database file, open it, read all the files to search, and use _FileListToArray(Rec) to find all the files;
Use FileCopy() if the file is found, otherwise continue the loop;
Do all these steps until there are no Database files in the array

mikell · August 26, 2019

Not sure I understood the requirements correctly but here is a suggestion
You might first concatenate the content of several TXT files (this is easy), then apply some regex

#Include <Array.au3>

$txt = "Material B7E671143D244B" & @crlf & _ 
    "====================================" & @crlf & _ 
    "TEXT 2F3139D816C34D 1" & @crlf & _ 
    "TEXT B6A968EF2505A2 1" & @crlf & _ 
    "TEXT 35206697A04F91 1" & @crlf & _ 
    "TEXT EB485AF490D83D 1" & @crlf & _ 
    "TEXT 0DAB42294BD9B3 1" & @crlf & _ 
    "TEXT 3D6525BEE360E1 0" & @crlf & _ 
    @crlf & _ 
    "Material D6906B886B06E3" & @crlf & _ 
    "====================================" & @crlf & _ 
    "TEXT 0CCECCCCFB62AE 1" & @crlf & _ 
    "TEXT 1E14CB29AB43F0 1" & @crlf & _ 
    "TEXT FB7F0DCE9B5950 1" & @crlf & _ 
    @crlf & _ 
    "Material 431831490FD5C9" & @crlf & _ 
    "====================================" & @crlf & _ 
    "TEXT 9D77B6474696D8 1" & @crlf & _ 
    "TEXT D04DEE5DF130A4 1" & @crlf & _ 
    "TEXT B6A968EF2505A2 1" & @crlf & _ 
    "TEXT 8C37245A4F0F81 1" & @crlf & _ 
    "TEXT A39FAFFC5ABC06 1" & @crlf & _ 
    "TEXT D24199644F2EE6 1" & @crlf & _ 
    @crlf & _ 
    "============================================================" & @crlf & _ 
    "ant-plastic-storage-bin-c_tmp0" & @crlf & _ 
    "============================================================" & @crlf & _ 
    "cit-storage-bins-01-color" & @crlf & _ 
    "cit-storage-bins-01-normal_dx10" & @crlf & _ 
    "cit-storage-bins-01-roughness_dx10" & @crlf & _ 
    "missingtexture_dx10"
Msgbox(0,"original", $txt)

; first remove the titles
$txt = StringRegExpReplace($txt, '(=+\R)?\N+\R=+\R', "")
Msgbox(0,"cleaned", $txt)

; then get the names
$res1 = StringRegExp($txt, 'TEXT\h(\H+)', 3)
_ArrayDisplay($res1, "case 1")

$res2 = StringRegExp($txt, '(?m)^\S+$', 3)
_ArrayDisplay($res2, "case 2")

; get all
$res3 = StringRegExp($txt, 'TEXT\h(\H+)|(?m)^\S+$', 3)
_ArrayDisplay($res3, "case 3")

Once you've got the names then use them to build filters to use - as Francesco said - with _FileListToArrayRec
Here is a sample snippet

#include <File.au3>
#include <Array.au3>

; define the folder to search
$Dir = StringRegExpReplace(@Autoitexe, '(.+)\\[^\\]+', "$1") & "\Include\"

; define the filters
$name1 = "*ListView*"
$name2 = "GuiTool*"

; get the file list
$alist = _FileListToArrayRec($Dir, $name1 & ";" & $name2)
_ArrayDisplay($alist)

Then loop through the resulting array using FileGetSize to sort the files and copy them where you need to

Edited August 26, 2019 by mikell

Tosyk · August 26, 2019

thanks for all the responces guys really appreciated!

I'm afraid I can not use Arrays because database (and yes, I call simple folder hierarhy a 'database') can be really huge - like hundreds of thousands of files. The easiest way is to build filelist from database and the search it for filenames, then copy files when found. with arrays searches can be really slow, like really really slow - I tried that already with the guy you made me that initial script.

also, what don't you like this function(?):

Func FileRead_($s_File)
    Local $h_File = FileOpen($s_File)
    Local $_sName, $i_Pos, $_sOldName, $_iStopper
    While 1
        $_sName = FileReadLine($h_File)
        If @error Then
            FileClose($h_File)
            ExitLoop
        EndIf
        If StringLeft($_sName, 9) = 'Material ' Then
            $_iStopper = True
            ContinueLoop
        ElseIf StringLeft($_sName, 1) = '=' Then
            $_iStopper = ($_iStopper = False)
            ContinueLoop
        ElseIf $_iStopper Then
            ContinueLoop
        EndIf
        If StringInStr($_sName, 'TEXT ') Then
            $_sName = StringRegExpReplace($_sName, '(^.*)\TEXT (.*)$', '$2')
            $_sName = StringRegExpReplace($_sName, '(^.*)\ (.*)$', '$1')
        EndIf
        If Not CheckIsSave_($_sName) Then
            FileSearch_($_sName)
        EndIf
    WEnd
EndFunc

the combinations of filenames can be any and this function cover all needs I have at the moment

mikell · August 27, 2019

I didn't mean that you have to read the whole content of the database to an array, I meant get only the filenames which fit the requirements defined in the TXT file(s) , these filenames only are returned by _FileListToArrayRec
So depending of the expected number of these filtered filenames, you could put them into an array indeed

BTW if you have 'hundreds of thousands of files' then the best way by far would be to store all the filenames into a SQLite database with table(s) to store the properties of each file (name, path, size, date etc). The SQlite engine is extremely powerful, fast and handy

Sign In

change FileCopy function a bit

Recommended Posts

Tosyk

Tosyk

FrancescoDiMuro

Tosyk

FrancescoDiMuro

Tosyk

tarretarretarre

FrancescoDiMuro

mikell

Tosyk

mikell

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members

Browse

AutoIt Resources

Release

Beta