Find and replace and generate logs

[pranaynanda]

I want to create a find and replace application that traverses deepest in a folder and finds and replaces text and also creates a log of the line number and file name. Any thoughts on where to get started? I need it to replicate deployments with minor changes.

[JohnOne]

You will need to elaborate.

For example, "finds and replaces text" of what?

[l3ill]

Hi pranaynanda,

 

Have a look in my sig at Snippet Browser. This may get your brain juices running in the right direction...

This is an application specifically for searching  for specific pieces of code in au3 files but could be easily modified for txt, ini etc.

You would need to add the change function and...

...adding a logging function would also not be a huge deal.

 

Maybe it helps maybe not...

Good luck !

Bill

[pranaynanda]

@JohnOne Seen Notepad++? What I want is the feature that replaces text in all files in a folder. Along with that I want to log the what was changed and where. I might even create a list view that can be edited and works something like a dictionary.

@l3ill Logging is the hardest part for me. I had a script that could find and replace in multiple files using python. I couldn't get across to use it log the changes.

[l3ill]

Well, lets see what you got.

You will have no trouble getting help here with such a cool project if you show your attempts and ask specific questions...

Bill

[orbs]

@pranaynanda,

are you aware there are many such utilities already exist? here's one i've been using.

if you need to do it on your au3 scripts, then SciTE offers "Replace in Buffers" (in Search > Replace... menu) which can replace text in all currently open au3 files, as well as SciTE Jump (pinned in the "Example Scripts" forum).

if you want to script it yourself, as a leaning practice - then i advise you do it from scratch. you'll find it a lot easier than you might think. looping files (or listing them in advance), then FileRead, StringInStr, StringReplace and FileWrite is pretty much all you need.

[pranaynanda]

@orbs I need it to change configurations of various VMs or patch deployments with little variation. I've tried fnr.exe previously. I t would have been a perfect solution had it been able to log those changes.

I'm weak at programming so I think this thread will be dormant for many weeks to come. Here's a list of things I need to do to get started and develop the logic:

1. Select a folder using windows dialog triggered by a "Browse" button.
2. Create an editable listview with two columns. One to find a string, the other to replace it.
3. I'll have challenges with browsing deepest through a folder
4. Searching for editable files. Some files may have *.config or *.deployment as their extension that also need a bulk of changes so I want to find and replace in them too.
5. Traversing through each file in the folder and it's each subfolder to find those multiple strings.
6. Replace should be easy using StringReplace(). I'll have trouble using it for multiple files as each file can have multiple entries as per dictionary. For example, it can be possible that I need to replace "abc" with "ijk" and "pqr" with "xyz". There can be multiple files that can have both "abc" and "ijk". I will have trouble dealing with that.
7. Finally, I'll have trouble logging all those changes with the files that were changed. Not a hard job but yes, I'll have trouble because I cannot figure out how to do it. Maybe everytime the code replaces a string? How will I deal with line numbers?

I think those are all challenges I can think of right now. Time to script. I'll be back soon.

[orbs]

1 hour ago, pranaynanda said:

I've tried fnr.exe previously. I t would have been a perfect solution had it been able to log those changes.

look here for the complete list of command line arguments of fnr.exe. did you notice  --LogFile? have you tried it?

if you intend to script the entire thing yourself, then i stand by my earlier statement - it is actually quite easy. start doing it and you'll see. _FileListToArrayRec() can get you started; _FileReadToArray() for each file will get you the files line-by-line (if line numbers are a requirement). one StringReplace() call for each dictionary entry, on each line. if really is that simple.

[pranaynanda]

I have this:

$ListView1 = GUICtrlCreateListView("Find|Replace With", 40, 120, 521, 201, BitOR($GUI_SS_DEFAULT_LISTVIEW,$LVS_EDITLABELS), BitOR($LVS_EX_GRIDLINES,$LVS_EX_CHECKBOXES,$LVS_EX_FULLROWSELECT))

Why doesn't this work?

The items in find and replace columns can't be edited in real time.

[orbs]

29 minutes ago, pranaynanda said:

The items in find and replace columns can't be edited in real time.

that's right. the behaviour of the ListView control is such that it cannot be edited in real time. unless, of course, you search the forum for how to do it, and you find this UDF.

[pranaynanda]

Thanks @orbs.BTW, fnr didn't work as expected AT ALL. I ran a simple find and replace in a sample text file which had only "1234" written in it as a test case and I asked it to replace it with "5678" and it couldn't even find a match.

Edited July 19, 2016 by pranaynanda

[hajo4]

On Freitag, 15. Juli 2016 at 0:48 PM, pranaynanda said:

Logging is the hardest part for me. I had a script that could find and replace in multiple files using python. I couldn't get across to use it log the changes.

Maybe you don't need to do the logging yourself - let another program do it.

For example, use a version-control-system, such as svn, git or hg.

Check-in all your configuration-files.

Do your changes, then check-in again (and input a sensible description to label that set of changes).

Then use the version-control-system to show you a diff.

 

[hajo4]

On Freitag, 15. Juli 2016 at 6:27 PM, orbs said:

are you aware there are many such utilities already exist? here's one i've been using.

Also, some small tool for changing text (and many other things): SwissFileKnife,

and a really big toolbox: Perl.

 

[pranaynanda]

On 7/22/2016 at 3:12 PM, hajo4 said:

Then use the version-control-system to show you a diff.

I don't want to compare versions. I only want to log the changes that have been made by find and replace.

[l3ill]

So I had some time to play and updated my Snippet Browser to do what I said above.

If you download the Snippet Browser package  and add the below code to the root file you can test and play with it.

Please be careful, the new code has NO error checking and will change strings in your files.

Best to leave it in the test root folder until you understand it and change it to your needs and add some error checking etc.

Right now it works only with .au3 files but that can be changed with this parameter: line 134

Local $FileList_A = _FileListToArrayRec(@ScriptDir, "*au3", 1, 1, 0, 2)

Main code changes are on lines 90 - 105

Spoiler

expandcollapse popup

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_Icon=SnippetBrowserIcon.ico
#AutoIt3Wrapper_UseUpx=n
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****
#cs ----------------------------------------------------------------------------
    SnippetBrowser1.0.2
    AutoIt Version: 3.3.10.0

    Author: Bill

    Script Function:
    Snippet Browser with Search and Syntax Hi-Liting
#ce ----------------------------------------------------------------------------
#Region ;==================#includes
#include <Array.au3>
#include <File.au3>
#include <ButtonConstants.au3>
#include <GuiButton.au3>
#include <EditConstants.au3>
#include <GuiEdit.au3>
#include <GuiRichEdit.au3>
#include <GUIConstantsEx.au3>
#include <Constants.au3>
#include <StaticConstants.au3>
#include <WindowsConstants.au3>
#include <ListviewConstants.au3>
#include <GuiListView.au3>
#include <GUIToolTip.au3>
#include "_FindInFile.au3"
#include "RESH.au3"
#EndRegion ;==================#includes
Global $sBlistView, $sString, $aColorTable
Global $sLastSearch = ""
Global $iSearchStart = 0
Global $sFolder = @ScriptDir ; works as browser in any folder you put the script in
#Region ;==================GUI
$Form1 = GUICreate("SnippetBrowser", 810, 680, 246, 140)
$hToolTip = _GUIToolTip_Create(0)
$Input1 = GUICtrlCreateInput("Search_Files", 10, 16, 160, 21)
$sBlistView = GUICtrlCreateListView("Path", 10, 45, 790, 253, BitOR($LVS_SINGLESEL, $LVS_SHOWSELALWAYS))
_GUICtrlListView_SetColumnWidth($sBlistView, 0, 770)
$search = GUICtrlCreateButton("File Search", 190, 14, 81, 25)
$searchTip = GUICtrlGetHandle($search)
_GUIToolTip_AddTool($hToolTip, 0, "Search Through Working Dir for Text in File", $searchTip)
$ClearAll = GUICtrlCreateButton("Clear", 290, 14, 81, 25)
$ClearAllTip = GUICtrlGetHandle($ClearAll)
_GUIToolTip_AddTool($hToolTip, 0, "Clear List View of Last Search", $ClearAllTip)
$getFiles = GUICtrlCreateButton("Get Files", 390, 14, 81, 25)
$getFilesTip = GUICtrlGetHandle($getFiles)
_GUIToolTip_AddTool($hToolTip, 0, "Populate List View with AutoIT Files from Working Dir", $getFilesTip)
GUICtrlSetState($getFiles, $GUI_DEFBUTTON)
$Input2 = GUICtrlCreateInput("Search_Text", 10, 306, 160, 21)
$showSelected = GUICtrlCreateButton("Preview", 390, 305, 81, 25)
$showSelectedTip = GUICtrlGetHandle($showSelected)
_GUIToolTip_AddTool($hToolTip, 0, "Shows Preview of Selected Path", $showSelectedTip)
$textSearch = GUICtrlCreateButton("Text Search", 190, 305, 81, 25)
$textSearchTip = GUICtrlGetHandle($textSearch)
_GUIToolTip_AddTool($hToolTip, 0, "Search Through Preview for Text", $textSearchTip)
$label1 = GUICtrlCreateLabel($sFolder, 485, 20, 315, 25)
GUICtrlSetTip(-1, "Working Directory: " & $sFolder, "", 0, 2)
GUICtrlSetData($label1, $sFolder)
$openSelected = GUICtrlCreateButton("Open in SciTE", 290, 305, 81, 25)
$openSelectedTip = GUICtrlGetHandle($openSelected)
_GUIToolTip_AddTool($hToolTip, 0, "Open Selected Path in SciTE", $openSelectedTip)
$throwString = GUICtrlCreateButton("Replace With", 490, 305, 81, 25)
$throwStringTip = GUICtrlGetHandle($throwString)
_GUIToolTip_AddTool($hToolTip, 0, "Replace Highlighted Text with whats typed in the box", $throwStringTip)
$Input3 = GUICtrlCreateInput("New_Text", 590, 306, 160, 21)
;~ $iCheckbox = GUICtrlCreateCheckbox("Tool Tips Off", 600, 305, 185, 25)
$hEdit = _GUICtrlRichEdit_Create($Form1, "", 10, 336, 790, 310, BitOR($ES_MULTILINE, $WS_VSCROLL, $WS_HSCROLL, $ES_AUTOVSCROLL))
_GUICtrlRichEdit_SetBkColor($hEdit, 0xEFEFEF)
GUISetState(@SW_SHOW, $Form1)
#EndRegion ;==================GUI

#Region ; ==================while loop
While 1
    $nMsg = GUIGetMsg()
    Switch $nMsg
        Case $GUI_EVENT_CLOSE
            Exit
;~      Case $iCheckbox
;~          If _IsChecked($iCheckbox) Then
;~              _GUIToolTip_Deactivate($hToolTip)
;~          Else
;~              _GUIToolTip_Activate($hToolTip)
;~          EndIf
        Case $getFiles ;~ Populates ListView with Files from Working Directory~
            _getFiles()

;;;;;++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
        Case $throwString ;~ Now "Replace With"

            Local $sFind = GUICtrlRead($Input2)
            Local $sReplace = GUICtrlRead($Input3)
            Local $iSelect = _GUICtrlListView_GetSelectedIndices($sBlistView, True)
            If $iSelect[0] > 0 Then
                Local $sSelect = StringTrimRight(_GUICtrlListView_GetItemText($sBlistView, $iSelect[1]), 1)
                ConsoleWrite("$sSelect: " & $sSelect & @CR)
                _ReplaceStringInFile($sSelect, $sFind, $sReplace)
                _FileWriteLog(@ScriptDir & "\FindReplaceLog.log",$sSelect & "  "  & $sFind & " Replaced with: " & $sReplace & " Succesfully!")
                MsgBox(0, "Log File Updated", $sFind & " Replaced with: " & $sReplace & " Succesfully!")
            Else
                MsgBox(0, "", "Nothing Selected")
            EndIf
;;;;;;+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

        Case $openSelected ;~ Opens Selected Path in SciTE Editor (New Tab)~
            _openSelected()
        Case $showSelected ;~ Previews Selected Path in Preview Window with Syntax Hiliting (RESH.au3)~Thanks Beege! ~
            _showSelected()
        Case $ClearAll ;~ Empties ListView of foundPaths to Make Room for New Search~
            _GUICtrlListView_DeleteAllItems($sBlistView)
        Case $search ;~ Searches Working Directory for "Text in File" Thanks guinness !~
            _search()
        Case $textSearch ;~ Searches Preview for Text ~ Keep Clicking to Find Next ~ Thanks Melba23 ~
            $sSearchTerm = GUICtrlRead($Input2)
            If $sSearchTerm Then
                ; If new search then reset start
                If $sSearchTerm <> $sLastSearch Then
                    $iSearchStart = 0
                    $sLastSearch = $sSearchTerm
                EndIf
                _GUICtrlRichEdit_SetSel($hEdit, $iSearchStart, $iSearchStart)
                $iIndex = _GUICtrlRichEdit_FindText($hEdit, $sSearchTerm)
                _GUICtrlRichEdit_SetSel($hEdit, $iIndex, $iIndex + StringLen($sSearchTerm))
                $iSearchStart = $iIndex + StringLen($sSearchTerm)
            EndIf
    EndSwitch
WEnd
#EndRegion ; ==================while loop

#Region ; ==================functions
Func _getFiles() ;~ Populates ListView with Files from Working Directory~
    Local $FileList_A = _FileListToArrayRec(@ScriptDir, "*au3", 1, 1, 0, 2)
    For $i = 1 To $FileList_A[0]
        _GUICtrlListView_AddItem($sBlistView, $FileList_A[$i] & @LF)
        ;GUICtrlSetFont($sBlistView, 12, 600)
        ConsoleWrite("$FileList_A[$i]: " & $FileList_A[$i] & @CR)
    Next
EndFunc   ;==>_getFiles

Func _showSelected() ;~ Previews Selected Path in Preview Window with Syntax Hiliting (RESH.au3)~Thanks Beege! ~
    Local Enum $iMacros, $iStrings, $iSpecial, $iComments, $iVariables, $iOperators, $iNumbers, $iKeywords, _
            $iUDFs, $iSendKeys, $iFunctions, $iPreProc, $iComObjects
    Local $aColorTable[13]
    ;notice values can be either 0x or #
    $aColorTable[$iMacros] = '#808000'
    $aColorTable[$iStrings] = 0xFF0000
    $aColorTable[$iSpecial] = '#DC143C'
    $aColorTable[$iComments] = '#008000'
    $aColorTable[$iVariables] = '#5A5A5A'
    $aColorTable[$iOperators] = '#FF8000'
    $aColorTable[$iNumbers] = 0x0000FF
    $aColorTable[$iKeywords] = '#0000FF'
    $aColorTable[$iUDFs] = '#0080FF'
    $aColorTable[$iSendKeys] = '#808080'
    $aColorTable[$iFunctions] = '#000090'
    $aColorTable[$iPreProc] = '#808000'
    $aColorTable[$iComObjects] = 0x993399
    Local $iSelect = _GUICtrlListView_GetSelectedIndices($sBlistView, True)
    If $iSelect[0] > 0 Then
        Local $sSelect = StringTrimRight(_GUICtrlListView_GetItemText($sBlistView, $iSelect[1]), 1)
        _GUICtrlRichEdit_SetText($hEdit, FileRead($sSelect))
        _RESH_SetColorTable($aColorTable)
        _RESH_SyntaxHighlight($hEdit)

    Else
        MsgBox(0, "", "Nothing Selected")
    EndIf
EndFunc   ;==>_showSelected

Func _openSelected() ;~ Opens Selected Path in SciTE Editor (New Tab)~
    Local $iSelect = _GUICtrlListView_GetSelectedIndices($sBlistView, True)
    If $iSelect[0] > 0 Then
        Local $sSelect = StringTrimRight(_GUICtrlListView_GetItemText($sBlistView, $iSelect[1]), 1)
        Return _SciTE_Send_Command(0, WinGetHandle("DirectorExtension"), 'open:' & StringReplace($sSelect, '\', '\\'))
    Else
        MsgBox(0, "", "Nothing Selected")
    EndIf
EndFunc   ;==>_openSelected

Func _search() ;~ Searches Working Directory for "Text in File" Thanks guinness !~
    Local $aArray = _FindInFile(GUICtrlRead($Input1), $sFolder, '*.au3')
    $rows = UBound($aArray)
    For $i = 0 To $rows - 1
        GUICtrlCreateListViewItem($aArray[$i] & @LF, $sBlistView)
    Next
EndFunc   ;==>_search

Func _SciTE_InsertText($sString)
    $sString = StringReplace($sString, '\', '\\')
    _SciTE_ReplaceMarcos($sString)
    Return _SciTE_Send_Command(0, WinGetHandle('DirectorExtension'), 'insert:' & $sString)
EndFunc   ;==>_SciTE_InsertText

Func _SciTE_ReplaceMarcos(ByRef $sString)
    $sString = StringReplace($sString, @TAB, '\t')
    $sString = StringReplace($sString, @CR, '\r')
    $sString = StringReplace($sString, @LF, '\n')
EndFunc   ;==>_SciTE_ReplaceMarcos

Func _SciTE_Send_Command($hHandle, $hSciTE, $sString)
    Local $ilParam, $tData
    If StringStripWS($sString, 8) = "" Then
        Return SetError(2, 0, 0) ; String is blank.
    EndIf
    $sString = ":" & Dec(StringTrimLeft($hHandle, 2)) & ":" & $sString
    ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $sString = ' & $sString & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console
    $tData = DllStructCreate("char[" & StringLen($sString) + 1 & "]") ; wchar
    DllStructSetData($tData, 1, $sString)
    $ilParam = DllStructCreate("ptr;dword;ptr") ; ulong_ptr;dword;ptr
    DllStructSetData($ilParam, 1, 1) ; $ilParam, 1, 1
    DllStructSetData($ilParam, 2, DllStructGetSize($tData))
    DllStructSetData($ilParam, 3, DllStructGetPtr($tData))
    _SendMessage($hSciTE, $WM_COPYDATA, $hHandle, DllStructGetPtr($ilParam))
    Return Number(Not @error)
EndFunc   ;==>_SciTE_Send_Command

Func _IsChecked($iControlID)
    Return BitAND(GUICtrlRead($iControlID), $GUI_CHECKED) = $GUI_CHECKED
EndFunc   ;==>_IsChecked
#EndRegion ; ==================functions

 

 

Type your search text top left Input Box and hit File Search - this will find all instances of this text in every File in the Folder and sub folders and list them in the top ListView

Now pick one and type your Find text and Replace text in the left and right Input boxes in the bottom part

Now when you click Replace With all instances of the Find text will be replaced with the Replace text and a log file will be created/appended with the necessary info.

 

Hope this helps you with your project, let me know if you have any issues.

Bill

 

edit: the GUI edit field will not update on the fly (immediately after pushing the button) only the files themselves...

Edited July 26, 2016 by l3ill
info add

[hajo4]

10 hours ago, pranaynanda said:

I don't want to compare versions. I only want to log the changes that have been made by find and replace.

Um... Such a diff will show all the changes.

Or do you want the list of changes in a custom format ?

[pranaynanda]

On 7/27/2016 at 0:06 AM, hajo4 said:

do you want the list of changes in a custom format ?

YES!

[pranaynanda]

Hi Guys!

I need assistance with this. I can't understand how this works:

#include <File.au3>
#include <Array.au3>
Local $b=0
Local $a=_FileListToArrayRec("E:\Documents\AutoIt\fnr","*",$FLTAR_FILES,$FLTAR_RECUR)
;_ArrayDisplay($a)
For $i=1 To UBound($a)-1 Step 1
;Local $b=0
_FileReadToArray($a[$i],$b)
ConsoleWrite($b)
ConsoleWrite($a)
Next

Please help. Also, I couldn't find RESH.au3 therefore, I couldn't fiddle with @l3ill's snippet browser.

[l3ill]

On 7/26/2016 at 8:17 PM, l3ill said:

If you download the Snippet Browser package  and add the below code to the root file you can test and play with it.

Snippet Browser

[l3ill]

Snippet Browser Package includes everything you need to test this code..

 

zip file