asgarcymed Posted November 25, 2007 Share Posted November 25, 2007 I am trying to make a very long script, which analyzes the hexadecimal dump of each file's header (get binary/hex bytes in the beginning of file), and compares it with an internal set (database) of pre-defined conditional rules. [PSEUDO-CODE] { Dim header header = get(dump) file header If header = "000ff00aa" Then file = "PDF" If header = "aff0ee000h" Then file = "CHM" (...) Sometimes, files downloaded in eMule (ed2k) are fakes - porno movies with wrong extension (iso, pdf, zip). When I try to open them and I got an error message, I hex-edit them and there it is - the AVI header!! Renaming file to avi, allows me to see the (porno) movie (damn, so many GB of download traffic wasted for nothing!!)... That's the reason why decided to create (or, at least, try to create) this script... It is very big, very CPU and RAM consumer, and always puts my PC totally unresponsive/"frozen" (I must press on-off button) when the target path has a relatively high number of files/folders... Here it is what I want to be helped to - put this script able to tolerate a high number of files/folders. It is very incomplete yet (few extensions/file types detected); I must add much more headers' rules; but I feel very disappointed by now, because although very incomplete, it stills not to be useful since it only can process a very small number of files... While this will not corrected, I will not continue to add new headers' rules, because there is an high risk of aborting/giving up from this project, as I cannot put it minimally useful... It is really very important to me getting this to work, so I post this help request... I post the script in 2 ways: as a code-box and as attachment (because code-box may not be well-readable). PS - RecurseDir is a function not made by me; I copied from other post in AutoIt Forums; since AutoIt has no built-in recursivity... Thanks in advance. Regards. expandcollapse popup#include <Array.au3> #include <File.au3> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; $path = "C:\test" ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Dim $dir, $mask, $dont_recurse, $dump, $return_dirs ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; RecurseDir($dir, $mask, $dont_recurse = False, $dump = "", $return_dirs = False) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Func RecurseDir($dir, $mask, $dont_recurse = False, $dump = "", $return_dirs = False) ;debug(@ScriptLineNumber & ": " & "func: RecurseDir " & $dir) Dim $n_dirnames[333333] ; maximum number of directories which can be scanned Local $n_dircount = 0 ; ^ could be set much higher, if required Local $n_file Local $n_search Local $n_tfile Local $file_array Local $filenames Local $filecount Local $dircount = 1 Local $msg = "error" ; if there was an "\" on the end of the given directory, remove that.. If StringRight($dir, 1) = "\" Then $dir = StringTrimRight($dir, 1) $n_dirnames[$dircount] = $dir If Not FileExists($dir) Then Return 0 While $dircount > $n_dircount ; keep on looping until all directories are scanned.. $n_dircount += 1 $n_search = FileFindFirstFile($n_dirnames[$n_dircount] & "\*.*") While 1 ; find all subdirs in this directory and store them in a array.. $n_file = FileFindNextFile($n_search) If @error Then ExitLoop ; skip directory references.. If $n_file = "." Or $n_file = ".." Then ContinueLoop $n_tfile = $n_dirnames[$n_dircount] & "\" & $n_file ; if it's a directory, add it to the list of directories to be processed.. If StringInStr(FileGetAttrib($n_tfile), "D") And Not $dont_recurse Then $dircount += 1 $n_dirnames[$dircount] = $n_tfile EndIf WEnd FileClose($n_search) ; multiple masks.. If StringInStr($mask, ",", 2) Then $mask_array = StringSplit($mask, ",") Else ; or else create a dummy array.. Dim $mask_array[2] = [1, $mask] EndIf ; loop through the array of masks.. For $mask_c = 1 To $mask_array[0] ; find all files that match this mask.. $n_search = FileFindFirstFile($n_dirnames[$n_dircount] & "\" & $mask_array[$mask_c]) If $n_search = -1 Then ContinueLoop While 1 $n_file = FileFindNextFile($n_search) If @error Then ExitLoop ; end of dir If $n_file = "." Or $n_file = ".." Then ContinueLoop $n_tfile = $n_dirnames[$n_dircount] & "\" & $n_file If Not StringInStr(FileGetAttrib($n_tfile), "D") Then $filecount += 1 $filenames &= $n_tfile & @LF EndIf WEnd FileClose($n_search) Next WEnd ; flip to a string and back to remove extraneous entries ; this is quicker than redimming on every loop If $return_dirs Then $tmp_str = "" $i = 1 While $n_dirnames[$i] <> "" $tmp_str &= $n_dirnames[$i] & "|" $i += 1 WEnd $tmp_str = StringTrimRight($tmp_str, 1) $n_dirnames = StringSplit($tmp_str, "|") Return $n_dirnames EndIf $filenames = StringTrimRight($filenames, 1) If $filenames = "" Then Return 0 $file_array = StringSplit($filenames, @LF) ; dump results to a file.. If $dump Then $dump_file = FileOpen($dump, 2) FileWrite($dump_file, $filenames) FileClose($dump_file) EndIf Return ($file_array) EndFunc ;==>RecurseDir ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Dim $dir_rec $dir_rec = RecurseDir($path, "*.*") ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; If FileExists(@DesktopDir & "\OK.csv") Then FileDelete(@DesktopDir & "\OK.csv") ElseIf FileExists(@DesktopDir & "\Fakes or Unknown files.csv") Then FileDelete(@DesktopDir & "\Fakes or Unknown files.csv") EndIf FileWrite(@DesktopDir & "\OK.csv", "Path" & "," & "Extension" & "," & "File Type" & Chr(13)) FileWrite(@DesktopDir & "\Fakes or Unknown files.csv", "Path" & "," & "Extension" & Chr(13)) $total_number_analyzed_files = 0 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; For $n = 1 To $dir_rec[0] ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Dim $path_split, $szDrive, $szDir, $szFName, $szExt $path_split = _PathSplit($dir_rec[$n], $szDrive, $szDir, $szFName, $szExt) $open_file = FileOpen($dir_rec[$n], 16) $read_file = FileRead($open_file) Switch $szExt ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Case ".exe" ;;; EXE ;;; If BinaryMid($read_file, 1, 2) = "MZ" Then FileWrite(@DesktopDir & "\OK.csv", $dir_rec[$n] & "," & $szExt & "," & "MS-DOS/Windows Executable" & Chr(13)) Else FileWrite(@DesktopDir & "\Fakes or Unknown files.csv", $dir_rec[$n] & "," & $szExt) EndIf ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Case ".zip" ;;; ZIP ;;; If BinaryMid($read_file, 1, 2) = "PK" Then FileWrite(@DesktopDir & "\OK.csv", $dir_rec[$n] & "," & $szExt & "," & "ZIP File (compressed)" & Chr(13)) Else FileWrite(@DesktopDir & "\Fakes or Unknown files.csv", $dir_rec[$n] & "," & $szExt) EndIf ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Case ".chm" ;;; CHM ;;; If BinaryMid($read_file, 1, 4) = "ITSF" Then FileWrite(@DesktopDir & "\OK.csv", $dir_rec[$n] & "," & $szExt & "," & "CHM File (Help - Compiled HTML)" & Chr(13)) Else FileWrite(@DesktopDir & "\Fakes or Unknown files.csv", $dir_rec[$n] & "," & $szExt) EndIf ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Case ".rar" ;;; RAR ;;; If BinaryMid($read_file, 1, 4) = "Rar!" Then FileWrite(@DesktopDir & "\OK.csv", $dir_rec[$n] & "," & $szExt & "," & "RAR File (compressed)" & Chr(13)) Else FileWrite(@DesktopDir & "\Fakes or Unknown files.csv", $dir_rec[$n] & "," & $szExt) EndIf ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Case ".wri" ;;; WRI ;;; If Hex(BinaryMid($read_file, 1, 6)) = "31BE000000AB" Then FileWrite(@DesktopDir & "\OK.csv", $dir_rec[$n] & "," & $szExt & "," & "Write Document" & Chr(13)) Else FileWrite(@DesktopDir & "\Fakes or Unknown files.csv", $dir_rec[$n] & "," & $szExt) EndIf ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Case ".wmv" ;;; WMV ;;; If Hex(BinaryMid($read_file, 1, 6)) = "3026B2758E66" Then FileWrite(@DesktopDir & "\OK.csv", $dir_rec[$n] & "," & $szExt & "," & "WMV (Windows Media Video)" & Chr(13)) Else FileWrite(@DesktopDir & "\Fakes or Unknown files.csv", $dir_rec[$n] & "," & $szExt) EndIf ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Case ".pdf" ;;; PDF ;;; If BinaryMid($read_file, 1, 7) = "%PDF-1." Then FileWrite(@DesktopDir & "\OK.csv", $dir_rec[$n] & "," & $szExt & "," & "PDF (Portable Document File)" & Chr(13)) Else FileWrite(@DesktopDir & "\Fakes or Unknown files.csv", $dir_rec[$n] & "," & $szExt) EndIf ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Case ".rtf" ;;; RTF ;;; If BinaryMid($read_file, 1, 7) = "{\rtf1\" Then FileWrite(@DesktopDir & "\OK.csv", $dir_rec[$n] & "," & $szExt & "," & "RTF (Ritch Text Format)" & Chr(13)) Else FileWrite(@DesktopDir & "\Fakes or Unknown files.csv", $dir_rec[$n] & "," & $szExt) EndIf ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Case ".pdb" ;;; PDB ;;; Select Case StringInStr(BinaryToString(BinaryMid($read_file, 1, 256)), "SDocSilX") ;;; iSilo's PDB File ;;; FileWrite(@DesktopDir & "\OK.csv", $dir_rec[$n] & "," & $szExt & "," & "iSilo's PDB File" & Chr(13)) Case StringInStr(BinaryToString(BinaryMid($read_file, 1, 256)), "TEXtREAd") ;;; iSilo's PDB File ;;; FileWrite(@DesktopDir & "\OK.csv", $dir_rec[$n] & "," & $szExt & "," & "iSilo's PDB File" & Chr(13)) Case StringInStr(BinaryToString(BinaryMid($read_file, 1, 256)), "ToGoToGo") ;;; iSilo's PDB File ;;; FileWrite(@DesktopDir & "\OK.csv", $dir_rec[$n] & "," & $szExt & "," & "iSilo's PDB File" & Chr(13)) Case Else FileWrite(@DesktopDir & "\Fakes or Unknown files.csv", $dir_rec[$n] & "," & $szExt) EndSelect ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Case ".djvu" Or ".djv" ;;; DJVU = DJV ;;; If StringInStr(BinaryToString(BinaryMid($read_file, 1, 256)), "DJVMDIRM") Then FileWrite(@DesktopDir & "\OK.csv", $dir_rec[$n] & "," & $szExt & "," & "DJVU=DJV File" & Chr(13)) Else FileWrite(@DesktopDir & "\Fakes or Unknown files.csv", $dir_rec[$n] & "," & $szExt) EndIf ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Case ".mov" ;;; MOV ;;; Select Case StringInStr(BinaryToString(BinaryMid($read_file, 1, 112)), "moov") FileWrite(@DesktopDir & "\OK.csv", $dir_rec[$n] & "," & $szExt & "," & "Apple Quick Time Movie" & Chr(13)) Case StringInStr(BinaryToString(BinaryMid($read_file, 1, 112)), "mdat") FileWrite(@DesktopDir & "\OK.csv", $dir_rec[$n] & "," & $szExt & "," & "Apple Quick Time Movie" & Chr(13)) Case Else FileWrite(@DesktopDir & "\Fakes or Unknown files.csv", $dir_rec[$n] & "," & $szExt) EndSelect ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Case ".mht" ;;; MHT ;;; Select Case StringInStr(BinaryToString(BinaryMid($read_file, 1, 256)), "Microsoft Internet Explorer") FileWrite(@DesktopDir & "\OK.csv", $dir_rec[$n] & "," & $szExt & "," & "MHT (MIME-HTML)" & Chr(13)) Case Else FileWrite(@DesktopDir & "\Fakes or Unknown files.csv", $dir_rec[$n] & "," & $szExt) EndSelect ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; EndSwitch ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; $total_number_analyzed_files = $total_number_analyzed_files + 1 Next MsgBox(64, "The END!", $total_number_analyzed_files & " files were analyzed." & Chr(13) & Chr(13) & "See ""OK.csv"" and ""Fakes or Unknown files.csv"" files, located at your Desktop.") MLMK - my blogging craziness... Link to comment Share on other sites More sharing options...
NerdFencer Posted July 9, 2008 Share Posted July 9, 2008 your lag problem is easily solved with one line of code in your primary loop Sleep(2) _________[u]UDFs[/u]_________-Mouse UDF-Math UDF-Misc Constants-Uninstaller Shell Link to comment Share on other sites More sharing options...
AdmiralAlkex Posted July 9, 2008 Share Posted July 9, 2008 your lag problem is easily solved with one line of code in your primary loop Sleep(2)Let's try to be consistent please? It should be Sleep(10) .Some of my scripts: ShiftER, Codec-Control, Resolution switcher for HTC ShiftSome of my UDFs: SDL UDF, SetDefaultDllDirectories, Converting GDI+ Bitmap/Image to SDL Surface Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now