highfeeling Posted November 22, 2007 Share Posted November 22, 2007 Hello to all of you ! I've just discover AutoIt (great product !) and I think it could respond to my need, but I'm not sure ... Here is my goal : . I have a log directory . Inside this directory, there is many subdirectories . Inside all these subdirectories, there is thousands of Synch*.log files . Inside of some of these log files, there is some errors. Then : . I would like to scan all of these log files . Search for those which contain error messages . catch error code and error message . produce at the end a report file, which will contain the following : ----------------------------------------------------------------------------------------------- Error n° XXX : <description of the error> : <number ot times this error occurs" <list of files which contain this error> <list of files which contain this error> <list of files which contain this error> <list of files which contain this error> ----------------------------------------------------------------------------------------------- Error n° YYY : <description of the error> : <number ot times this error occurs" <list of files which contain this error> <list of files which contain this error> <list of files which contain this error> <list of files which contain this error> ----------------------------------------------------------------------------------------------- As I can see, there is many ways to do it But can you tell me please what is the simplest way to implement this need ? Many thanks in advance ! Link to comment Share on other sites More sharing options...
BrettF Posted November 22, 2007 Share Posted November 22, 2007 Welcome to the forums You should start by being able to find and display the files. Do a forum search for a recursive file search. Then _ArrayDisplay the results. See what you get? Do you think you can think of a way to use that data? I'll give a hint. A For...Next loop. Then what you want to do is look at the FileOpen, FileRead, FileWrite, and _FileReadToArray and _ArraySearch. That should give you a hefty list to get you started Good luck with this Brett Vist my blog!UDFs: Opens The Default Mail Client | _LoginBox | Convert Reg to AU3 | BASS.au3 (BASS.dll) (Includes various BASS Libraries) | MultiLang.au3 (Multi-Language GUIs!)Example Scripts: Computer Info Telnet Server | "Secure" HTTP Server (Based on Manadar's Server)Software: AAMP- Advanced AutoIt Media Player | WorldCam | AYTU - Youtube Uploader Tutorials: Learning to Script with AutoIt V3Projects (Hardware + AutoIt): ArduinoUseful Links: AutoIt 1-2-3 | The AutoIt Downloads Section: | SciTE4AutoIt3 Full Version! Link to comment Share on other sites More sharing options...
highfeeling Posted November 22, 2007 Author Share Posted November 22, 2007 Welcome to the forums You should start by being able to find and display the files. Do a forum search for a recursive file search. Then _ArrayDisplay the results. See what you get? Do you think you can think of a way to use that data? I'll give a hint. A For...Next loop. Then what you want to do is look at the FileOpen, FileRead, FileWrite, and _FileReadToArray and _ArraySearch. That should give you a hefty list to get you started Good luck with this Brettok thanks for infos !!!When I will finish, I will publish it, ok ? Link to comment Share on other sites More sharing options...
BrettF Posted November 22, 2007 Share Posted November 22, 2007 ok thanks for infos !!!When I will finish, I will publish it, ok ?No problem! Sure! Any example is a good example! Mostly... Vist my blog!UDFs: Opens The Default Mail Client | _LoginBox | Convert Reg to AU3 | BASS.au3 (BASS.dll) (Includes various BASS Libraries) | MultiLang.au3 (Multi-Language GUIs!)Example Scripts: Computer Info Telnet Server | "Secure" HTTP Server (Based on Manadar's Server)Software: AAMP- Advanced AutoIt Media Player | WorldCam | AYTU - Youtube Uploader Tutorials: Learning to Script with AutoIt V3Projects (Hardware + AutoIt): ArduinoUseful Links: AutoIt 1-2-3 | The AutoIt Downloads Section: | SciTE4AutoIt3 Full Version! Link to comment Share on other sites More sharing options...
ptrex Posted November 22, 2007 Share Posted November 22, 2007 @AllMaybe this can be of any use.MS LogParserSpecialy designed for this type of job.regardsptrex Contributions :Firewall Log Analyzer for XP - Creating COM objects without a need of DLL's - UPnP support in AU3Crystal Reports Viewer - PDFCreator in AutoIT - Duplicate File FinderSQLite3 Database functionality - USB Monitoring - Reading Excel using SQLRun Au3 as a Windows Service - File Monitor - Embedded Flash PlayerDynamic Functions - Control Panel Applets - Digital Signing Code - Excel Grid In AutoIT - Constants for Special Folders in WindowsRead data from Any Windows Edit Control - SOAP and Web Services in AutoIT - Barcode Printing Using PS - AU3 on LightTD WebserverMS LogParser SQL Engine in AutoIT - ImageMagick Image Processing - Converter @ Dec - Hex - Bin -Email Address Encoder - MSI Editor - SNMP - MIB ProtocolFinancial Functions UDF - Set ACL Permissions - Syntax HighLighter for AU3ADOR.RecordSet approach - Real OCR - HTTP Disk - PDF Reader Personal Worldclock - MS Indexing Engine - Printing ControlsGuiListView - Navigation (break the 4000 Limit barrier) - Registration Free COM DLL Distribution - Update - WinRM SMART Analysis - COM Object Browser - Excel PivotTable Object - VLC Media Player - Windows LogOnOff Gui -Extract Data from Outlook to Word & Excel - Analyze Event ID 4226 - DotNet Compiler Wrapper - Powershell_COM - New Link to comment Share on other sites More sharing options...
Zedna Posted November 22, 2007 Share Posted November 22, 2007 For inspiration you may also look at my LogView project - it's in my signature Resources UDF ResourcesEx UDF AutoIt Forum Search Link to comment Share on other sites More sharing options...
highfeeling Posted November 23, 2007 Author Share Posted November 23, 2007 Thanks for help, all of you ! So ... Here is my script : CODE; ; Script de suivi des synchros Tapas #include <Date.au3> #include <Array.au3> #include <INet.au3> Dim $FileCount = 0 Dim $Currentfile = 0 Dim $NbErrors = 0 Dim $NbSynchKO = 0 Dim $UserCode = "" Dim $ErrorsCodeArray Dim $ErrorsOccursArray Dim $ErrorsLabelArray Dim $ErrorCode = "" Dim $SynchroUserCode Dim $SynchroLogFile Dim $SynchroStart Dim $SynchroEnd Dim $SynchroErrorCode Dim $SynchroVolume Dim $Start Dim $End Dim $Volume Dim $Bu If FileExists(@ScriptDir & "\SynchFollowUp.txt") Then FileDelete(@ScriptDir & "\SynchFollowUp.txt") Endif ;-------------------------------------------------------------------------------- ; BU : CIGARET FRANCE ;-------------------------------------------------------------------------------- If FileExists("c:\windows\2PRD_Synch_followup.bat") Then FileDelete("c:\windows\2PRD_Synch_followup.bat") Endif FileWriteLine ( "c:\windows\2PRD_Synch_followup.bat", "@echo off") FileWriteLine ( "c:\windows\2PRD_Synch_followup.bat", "net use Q: /DELETE") FileWriteLine ( "c:\windows\2PRD_Synch_followup.bat", "net use Q: \\sv841\S$ adminsv841 /USER:sv841\administrator") $Retour = RunWait ( @Comspec & ' /C c:\windows\2PRD_Synch_followup.bat',"", @SW_MINIMIZE) If FileExists("c:\windows\2PRD_Synch_followup.bat") Then FileDelete("c:\windows\2PRD_Synch_followup.bat") Endif ;ScanFolder("Q:\apps\siebel78\tapp\siebsrvr\log", "CIGARET FRANCE") ;ScanFolder("D:\Projects\Tapas\Synchro followup\log", "CIGARET FRANCE") ScanFolder("Q:\apps\siebel78\tapp\siebsrvr\log", "CIGARET FRANCE") ;-------------------------------------------------------------------------------- ;-------------------------------------------------------------------------------- ; BU : CIGARET SPAIN ;-------------------------------------------------------------------------------- If FileExists("c:\windows\2PRD_Synch_followup.bat") Then FileDelete("c:\windows\2PRD_Synch_followup.bat") Endif FileWriteLine ( "c:\windows\2PRD_Synch_followup.bat", "@echo off") FileWriteLine ( "c:\windows\2PRD_Synch_followup.bat", "net use Q: /DELETE") FileWriteLine ( "c:\windows\2PRD_Synch_followup.bat", "net use Q: \\sv840\R$ adminsv840 /USER:sv840\administrator") $Retour = RunWait ( @Comspec & ' /C c:\windows\2PRD_Synch_followup.bat',"", @SW_MINIMIZE) If FileExists("c:\windows\2PRD_Synch_followup.bat") Then FileDelete("c:\windows\2PRD_Synch_followup.bat") Endif ScanFolder("Q:\apps\siebel78\tapp\siebsrvr\log", "CIGARET SPAIN") ;-------------------------------------------------------------------------------- ;-------------------------------------------------------------------------------- ; BU : CIGAR SPAIN ;-------------------------------------------------------------------------------- If FileExists("c:\windows\2PRD_Synch_followup.bat") Then FileDelete("c:\windows\2PRD_Synch_followup.bat") Endif FileWriteLine ( "c:\windows\2PRD_Synch_followup.bat", "@echo off") FileWriteLine ( "c:\windows\2PRD_Synch_followup.bat", "net use Q: /DELETE") FileWriteLine ( "c:\windows\2PRD_Synch_followup.bat", "net use Q: \\sv841\T$ adminsv841 /USER:sv841\administrator") $Retour = RunWait ( @Comspec & ' /C c:\windows\2PRD_Synch_followup.bat',"", @SW_MINIMIZE) If FileExists("c:\windows\2PRD_Synch_followup.bat") Then FileDelete("c:\windows\2PRD_Synch_followup.bat") Endif ScanFolder("Q:\apps\siebel78\tapp\siebsrvr\log", "CIGAR SPAIN") ;-------------------------------------------------------------------------------- ;------------------------------------------------------------------------------- ; Functions ;------------------------------------------------------------------------------- Func ScanFolder($SourceFolder, $Bu) Local $Search Local $File Local $FileAttributes Local $FullFilePath $ErrorsCodeArray = _ArrayCreate("Error Code") $ErrorsOccursArray = _ArrayCreate("Occurs") $ErrorsLabelArray = _ArrayCreate("Error Label") $SynchroUserCode = _ArrayCreate("User Code") $SynchroLogFile = _ArrayCreate("Log File ") $SynchroStart = _ArrayCreate("Start ") $SynchroEnd = _ArrayCreate("End ") $SynchroErrorCode = _ArrayCreate("Error Code") $SynchroVolume = _ArrayCreate("Volume") $NbErrors = 0 $NbSynchKO = 0 $Currentfile = 0 ProgressOn("Progress Meter", "BU : " & $Bu & " in progress ...", "Count how many log files to handle") ;count how many files to handle CountFiles($SourceFolder) $Search = FileFindFirstFile($SourceFolder & "\Sync*.log") While 1 If $Search = -1 Then ExitLoop EndIf $File = FileFindNextFile($Search) If @error Then ExitLoop $FullFilePath = $SourceFolder & "\" & $File $FileAttributes = FileGetAttrib($FullFilePath) If StringInStr($FileAttributes,"D") Then ScanFolder($FullFilePath) Else $UserCode = "" AnalyzeFile($FullFilePath, $Bu) If $NbErrors > 0 then $NbSynchKO += 1 Endif EndIf $Currentfile +=1 WEnd ;------------------------------------------------------------------------------- ; Generate Report ;------------------------------------------------------------------------------- FileWriteLine(@ScriptDir & "\SynchFollowUp.txt"," ") FileWriteLine(@ScriptDir & "\SynchFollowUp.txt","************************************") FileWriteLine(@ScriptDir & "\SynchFollowUp.txt","* Synchronization follow-up") FileWriteLine(@ScriptDir & "\SynchFollowUp.txt","* Date and time : " & _Now() ) FileWriteLine(@ScriptDir & "\SynchFollowUp.txt","************************************") FileWriteLine(@ScriptDir & "\SynchFollowUp.txt"," ") FileWriteLine(@ScriptDir & "\SynchFollowUp.txt",";-------------------------------------------------------------------------------") FileWriteLine(@ScriptDir & "\SynchFollowUp.txt","; Global Statistics for BU : " & $Bu) FileWriteLine(@ScriptDir & "\SynchFollowUp.txt",";-------------------------------------------------------------------------------") FileWriteLine(@ScriptDir & "\SynchFollowUp.txt","Number of synchronizations : " & $FileCount) FileWriteLine(@ScriptDir & "\SynchFollowUp.txt","Number of synchronizations with errors : " & $NbSynchKO) FileWriteLine(@ScriptDir & "\SynchFollowUp.txt"," ") FileWriteLine(@ScriptDir & "\SynchFollowUp.txt",";-------------------------------------------------------------------------------") FileWriteLine(@ScriptDir & "\SynchFollowUp.txt","; Global Errors follow-up for BU : " & $Bu) FileWriteLine(@ScriptDir & "\SynchFollowUp.txt",";-------------------------------------------------------------------------------") FileWriteLine(@ScriptDir & "\SynchFollowUp.txt"," ") For $element = 0 to (UBound($ErrorsCodeArray) - 1) FileWriteLine(@ScriptDir & "\SynchFollowUp.txt", StringLeft($ErrorsCodeArray[$element] & " ",10) & " - " & StringLeft($ErrorsOccursArray[$element] & " ",8) & " - " & $ErrorsLabelArray[$element]) Next FileWriteLine(@ScriptDir & "\SynchFollowUp.txt"," ") FileWriteLine(@ScriptDir & "\SynchFollowUp.txt",";-------------------------------------------------------------------------------") FileWriteLine(@ScriptDir & "\SynchFollowUp.txt","; Global Users errors follow-up for BU : " & $Bu) FileWriteLine(@ScriptDir & "\SynchFollowUp.txt",";-------------------------------------------------------------------------------") FileWriteLine(@ScriptDir & "\SynchFollowUp.txt"," ") For $element = 0 to (UBound($SynchroUserCode) - 1) FileWriteLine(@ScriptDir & "\SynchFollowUp.txt", StringLeft($SynchroUserCode[$element] & " ",10) & " | " & StringRight($SynchroLogFile[$element],18) & " | " & $SynchroStart[$element] & " | " & $SynchroEnd[$element] & " | " & StringLeft($SynchroErrorCode[$element] & " ",10) & " | " & $SynchroVolume[$element]) Next ProgressOff() EndFunc Func AnalyzeFile($FileName, $Bu) $File = FileOpen($FileName, 0) $NbErrors = 0 $SynchInError = 0 $Volume = "" ProgressSet( (($Currentfile *100) / $FileCount), StringRight($FileName, 18) & " analyze in progress ...") ; Check if file opened for reading OK If $file = -1 Then MsgBox(0, "Error", "Unable to open file.") Exit EndIf ; Read in lines of text until the EOF is reached $Start = "" $End = "" While 1 $line = FileReadLine($file) If @error = -1 Then ExitLoop If $Start = "" Then $Start = StringMid ($line, 6, 19) $End = StringMid ($line, 26, 19) Endif If $SynchInError = 1 Then If StringInStr($line, "Node ", 0) <> 0 Then $UserCode = StringMid ($line,StringInStr($line, "Node ", 0)+ 5, 8) $Volume = StringMid ($line,StringInStr($line, "Node ", 0)+ 14, 70) Endif Endif If StringInStr($line, "err=", 0) <> 0 Then $NbErrors += 1 $SynchInError = 1 $ErrorCode = StringMid ($line,StringInStr($line, "err=", 0)+ 4, 7) $ErrorLabel = StringMid ($line,StringInStr($line, "err=", 0)+ 12, 100) $Pos = _ArraySearch ($ErrorsCodeArray, $ErrorCode, 0, 0, 0, True) If $Pos = -1 Then _ArrayAdd($ErrorsCodeArray,$ErrorCode) _ArrayAdd($ErrorsOccursArray,1) _ArrayAdd($ErrorsLabelArray,$ErrorLabel) Else $ErrorsOccursArray[$Pos] += 1 EndIf _ArrayAdd($SynchroUserCode,"") _ArrayAdd($SynchroLogFile,$FileName) _ArrayAdd($SynchroStart,$Start) _ArrayAdd($SynchroEnd,$End) _ArrayAdd($SynchroErrorCode,$ErrorCode) _ArrayAdd($SynchroVolume,$Volume) EndIf Wend For $element = 0 to (UBound($SynchroUserCode) - 1) If $SynchroUserCode[$element] = "" Then $SynchroUserCode[$element] = $UserCode $SynchroVolume[$element] = $Volume Endif Next SplashOff() FileClose($file) EndFunc Func CountFiles($SourceFolder) $FileCount = 0 $Search = FileFindFirstFile($SourceFolder & "\Sync*.log") While 1 If $Search = -1 Then ExitLoop EndIf $File = FileFindNextFile($Search) If @error Then ExitLoop $FullFilePath = $SourceFolder & "\" & $File $FileAttributes = FileGetAttrib($FullFilePath) If StringInStr($FileAttributes,"D") Then CountFiles($FullFilePath) Else $FileCount += 1 Endif WEnd EndFunc Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now