A String Is a String Is a String -- or not?

[SpotCheckBilly]

Hey all,

I'm working on a project that contains a listbox into which I place one of three different log files (for those of you familiar with them HijackThis, RSIT and DDS). When finished, the program will output a fully formatted, "ready to post" solution to assist people trying to remove Malware from their computers.

For quite some time I've been working on a function that extracts certain information from each line and puts it into a Google search box. Since I'm brand-new to programming this has been quite a challenge. Without posting the whole boatload of code, I can say that the listbox is populated properly by either opening an existing file or by pasting one in, taken from the Internet. I've run into a very strange problem. Incidentally, I've been getting a lot of help from Authenticity for which I am eternally grateful.

This is the code I use to extract individual lines from within the listbox:

Local $aItems, $sItems
    $hListbox = $List1
    $aItems = _GUICtrlListBox_GetSelItemsText($hListbox)

    For $iI = 1 To $aItems[0]
        $sStr = $aItems[$iI]
        $sItems &= @LF & $sStr
        
    Next

I follow this with stripping extra quotation marks from the string:

For $iI = 1 To $aItems[0]
If StringInStr($sStr, '"', 0) Then
$sStr=StringReplace($sStr, '"', '')
Clipput($sStr)
EndIf
Next

Following is the code to parse the information I need from the string:

Dim $sStr
Local $dllStr

$aArray= _StringBetween($sStr, '[', ']', -1)
$Name= _ArrayToString($aArray)

$Filepos=StringInStr($sStr, '\', 0 ,-1)
$FileRet=StringTrimLeft($sStr, $Filepos)
$File=StringTrimRight($FileRet, 1)  ; File wNoSwitch

This is abbreviated. These are all Start up entries, in other words set to launch with Windows and there are several different commandline switches that have to be dealt with. The problem I'm running into are straightforward, no switches file paths. (The information on extracting is highlighted in color).

This line (from a HijackThis log) parses correctly:

$sStr='O4 - HKLM\..\Run: [iAAnotif] "C:\Program Files\Intel\Intel Matrix Storage Manager\Iaanotif.exe"'

However, this line (from a DDS log) does not:

$sStr='mRun: [stxTrayMenu] "c:\program files\seagate\systemtray\StxMenuMgr.exe"'

As you can see, the lines are formatted virtually identically. The first one parses out thus:. "IAAnotif" "Iaanotif.exe"

(The surrounding quotation marks are added just before sending to Google).

The line from the DDS log just returns the original string minus the quotation marks. I have absolutely no idea why. I would truly appreciate any input or insight because right now I'm completely baffled. Thanks so much in advance. -- SCB

[dani]

Instead of StringTrim, StringInStr etc, use Regular Expressions. This code parses both your input strings correctly:

#include <Array.au3>

$1 = 'O4 - HKLM\..\Run: [IAAnotif] "C:\Program Files\Intel\Intel Matrix Storage Manager\Iaanotif.exe"'
$2 = 'mRun: [StxTrayMenu] "c:\program files\seagate\systemtray\StxMenuMgr.exe"'

; allow anything -> capture whatever is inbetween [ and ] -> allow anything -> match backslash (\) -> capture whatever is left -> exclude last "
$regEx = '.*\[(.*)\].*\\(.*)"\z'

$r = StringRegExp($1, $regEx, 1)
_ArrayDisplay($r) ; $r[0] = IAAnotif | $r[1] = Iaanotif.exe

$r = StringRegExp($2, $regEx, 1)
_ArrayDisplay($r) ; $r[0] = StxTrayMenu | $r[1] = StxMenuMgr.exe

Edited March 5, 2010 by dani

[JohnOne]

This looks a little off to me

For $iI = 1 To $aItems[0]
If StringInStr($sStr, '"', 0) Then
$sStr=StringReplace($sStr, '"', '')
Clipput($sStr)
EndIf
Next

Should it be

For $iI = 1 To $aItems[0]
If StringInStr($aItems[$iI], '"', 0) Then
$sStr=StringReplace($aItems[$iI], '"', '')
Clipput($sStr)
EndIf
Next

[SpotCheckBilly]

Thanks a lot for the responses. They are greatly appreciated.

@dani -- I am nowhere close to having a handle on Regular Expressions. I downloaded a small program that helps put them together and Authenticity worked with me a little bit on them but I have a long way to go before I would feel comfortable trying to write one into some code. I'll try and incorporate it into my current code and let you know how it works out. Thanks a lot.

@JohnOne -- Your suggestion took care of a little "hiccup" I was getting. Thank you so much. Oddly enough, the code that I had DID strip the quotation marks properly which is why I was having such a problem understanding why only one line parsed properly. Programming is supposed to be based upon pure logic -- it would appear that there are exceptions even to that rule. Looks like I have a long, long road ahead of me. ***Big sigh***

Thanks again for the responses. Have a great weekend. -- SCB

[FaT3oYCG]

Programming is based on "pure logic" sometimes us humans just cant fathom how that works though, glad you solved you problem anyway.

[SpotCheckBilly]

@-- FaT3oYCG (interesting moniker ) I haven't quite figured out how to work the RegExp into my code yet. It would certainly reduce the amount of code for that function for sure. I can convert the array produced to a string and separate the two returns with an @TAB replacing the pipe but since Regular Expressions and I are still pretty much strangers I have yet to figure out how to wrap the two returns with quotation marks (IAAnotif | Iaanotif.exe becomes "IAAnotif" @TAB "Iaanotif.exe"). Doing so will reduce the amount of code required for that function by one half -- at least.

By the way, the logic that differentiates my two example strings would seem to be beyond my grasp, at this point anyway. LOL I must be missing something but I sure can't tell what the heck it is (see "headbanging" smiley above Hee Hee). -- SCB

[dani]

Not sure how you directly cast the array to a string, but you don't need to do that Just do

#include <Array.au3>

$1 = 'O4 - HKLM\..\Run: [IAAnotif] "C:\Program Files\Intel\Intel Matrix Storage Manager\Iaanotif.exe"'
$2 = 'mRun: [StxTrayMenu] "c:\program files\seagate\systemtray\StxMenuMgr.exe"'

; allow anything -> capture whatever is inbetween [ and ] -> allow anything -> match backslash (\) -> capture whatever is left -> exclude last "
$regEx = '.*\[(.*)\].*\\(.*)"\z'

$r1 = StringRegExp($1, $regEx, 1)
$1_1 = $r1[0] ; IAAnotif
$1_2 = $r1[1] ; Iaanotif.exe

$r2 = StringRegExp($2, $regEx, 1)
$2_1 = $r2[0] ; StxTrayMenu
$2_2 = $r2[1] ; StxMenuMgr.exe

MsgBox(0, "", $1_1 & " | " & $1_2 & " | " & $2_1 & " | " & $2_2) ; Don't you just love non-descriptive variable names :x

[SpotCheckBilly]

Hi dani,

The thought behind them wording directly to a string was that it would save me a lot of steps. Problem being, as I started playing around with it, it didn't return as I expected it to. I thought that if there were anything after the final set of quotation Mark's, it would show up after the file name. In other words it would be: Text>> File Text. I discovered rather quickly that if anything followed the file quote, the script would error out.

The part of the log files we are talking about here refers to the startup section of Windows Registry, meaning programs that launch with Windows. Following is a list of what we might see in this section of the log files (all three have the exact same formatting):

'Text: [xxxx] Filepath\File'

'Text: [xxxx] "Filepath\File"'

'Text: [xxxx] Filepath\File -text'

'Text: [xxxx] Filepath\File /text'

'Text: [xxxx] "Filepath\File" -text'

'Text: [xxxx] "Filepath\File" /text'

'Text: [xxxx] text Filepath\File,text'

'Text: [xxxx] text File(no path),text'

The quotation marks surrounding Filepath\File seem to just show up wherever they want to. There's no constant pattern and even experienced programmers at the security sites I belong to have no explanation.

If it were just the two example strings I included with my first post, your solution would work wonderfully. However, depending on how many bells and whistles the client (I don't like to call them victims, even if they are) likes to have launch with Windows, this section of the registry can get quite lengthy. I've worked with people who've had 30+ programs launching would Windows does (including the necessary ones).

So I'm still faced with the same dilemma. The code that I'm using to strip the quotes works quite nicely. Unfortunately, in lengths from the DSS log, they do not parse properly.

"Don't you just love non-descriptive variable names "

Yepper, good thing I have a fairly creative imagination. Coming up with short descriptive names for that many variables can challenge the sanity of even the most well adjusted person. LOL hope you're having a good weekend. -- SCB

[dani]

Regular Expressions are still the key to success here. I have modified my original Reg Ex a little bit to simply catch the filename and end at ".exe", then ignore everything following it. As you can see when you execute this, it ignores all the bullshit I have added to the strings, as long as a certain pattern still exists

#include <Array.au3>

$1 = 'O4 - HKLM\..\Run: [IAAnotif] "C:\Program Files\Intel\Intel Matrix Storage Manager\Iaanotif.exe werwes"'
$2 = 'mRun: [StxTrayMenu] "c:\program files\seagate\systemtray\StxMenuMgr.exe" !:!O@ LSW '
$3 = 'RandomText @$! _)!(@!: [Bing!] "c:\program files\common files\microsoft\search\bing\weHateGoogle.exe!!!! TEST"'
$4 = 'Waaaaaaaaaaaaa@%$:123 :O! [SomethingEvil] "c:\hell\satan.exe jesus go away.." !! fast'
$5 = 'AutoitAUtoitAU3etc... zzz moretext [ohCrap] "c:\programs\seahorse\whatever\OK.exe randomly typing stuff sucks" :x'

; allow anything -> capture whatever is inbetween [ and ] -> allow anything -> match backslash (\) -> capture whatever is left until .exe -> ignore the rest
$regEx = '\A.*\[(.*)\].*\\(.*\.exe).*\z'

For $i = 1 To 5
  $r = StringRegExp(Eval($i), $regEx, 1)
  _ArrayDisplay($r)
Next

Edited March 8, 2010 by dani

[SpotCheckBilly]

OK, I must have messed up here someplace. Because some of the startup entries run under the rumdll32.exe command line, I did a quick If... Then loop substituting .dll for .exe in the RegEx. Then, as you can see I replaced your original number variables with $sStr, and replaced your numeric array variable with $aArray. I used copy/paste to make sure there were no typos, then plugged the whole thing into a little test script that I use. However now I get the following error (code follows) :

TESTScript.au3 (14) : ==> Subscript used with non-Array variable.:

$sStr_1 = $aArray[0]

$sStr_1 = $aArray^ ERROR

#include <Array.au3>

$sStr='dRunOnce: [tscuninstall] %systemroot%\system32\tscupgrd.exe'

If StringInStr($sStr, '.exe', 0) Then
$regEx = '\A.*\[(.*)\].*\\(.*\.exe).*\z'
If StringInStr($sStr, '.dll', 0) Then
$regEx = '\A.*\[(.*)\].*\\(.*\.dll).*\z'
EndIf
EndIf

$aArray= StringRegExp(Eval($sStr), $regEx, 1)

$sStr_1 = $aArray[0] 
$sStr_2 = $aArray[1] 
$Name=$sStr_1
$Filename=$sStr_2

Clipput(chr(34) & $Name & chr(34) & @TAB & chr(34) & $Filename & chr(34) & @LF)

Run('Notepad.exe')
WinWaitActive('Untitled - Notepad')
Send('^V')

You can see that I also changed it back to parsing only one line at a time. Anyway, it looks to me as if the script should work. Where did I go wrong? If nothing else, working on this function has sure taught me a lot about string management LOL. Thank you so much for your help. -- SCB

[dani]

Why do you use Eval ? Don't do that, just use:

$aArray= StringRegExp($sStr, $regEx, 1)

Come on man you could have figured this one out yourself

[SpotCheckBilly]

Well, color me embarrassed! I did a direct copy/paste and skipped right over that when I took it out of the multi-line loop. I should've looked up "Eval". Talk about a rookie mistake! I'm normally pretty obsessive about looking up stuff that I'm not familiar with.

In my defense, though, I'm about two weeks into a nasty flu bug and ran out of daytime N6yQuil. I spent most of yesterday in a nighttime N6yQuil haze. Shoulda stuck with playing "Crazy Eights" and answering e-mail, 'eh? -- SCB

[SpotCheckBilly]

Hey dani,

For my purposes, since I have several different cases that require the file name extraction, I split the Regular Expression to the following:

If StringInStr($sStr, '.exe', 0) Then

$regEx = '\A.*\\(.*\.exe).*\z'

And:

If StringInStr($sStr, '.dll', 0) Then

$regEx = '\A.*\\(.*\.dll).*\z'

I combined them thus:

If StringInStr($sStr, '.exe', 0) Then
$regEx = '\A.*\\(.*\.exe).*\z'
ElseIf StringInStr($sStr, '.dll', 0) Then
$regEx = '\A.*\\(.*\.dll).*\z'
EndIf

To make the results available to all of the cases in which it was required, I did this:

For $iI = 1 To $aItems[0]
If StringInStr($sStr, '.exe', 0) Then
$regEx = '\A.*\\(.*\.exe).*\z'
ElseIf StringInStr($sStr, '.dll', 0) Then
$regEx = '\A.*\\(.*\.dll).*\z'
EndIf

$aArray = StringRegExp($sStr, $regEx, 1)
$Filename= _ArrayToString($aArray)
EndIf
Next

As it turned out, this allows me to eliminate a huge amount of code. I did run into one small problem, however. The regEx appears to be case sensitive. There are instances where xxx.exe is XXX.EXE, the same being true with the .dll's.

Reading through the AutoIt help file I see that it's possible to force case insensitivity by using the ? along with several other characters e.g. ?| etc. What wasn't made clear is where to place them in the regEx. To someone who is at home with Regular Expressions, this is probably a silly question, but to a complete newbie like myself, it's like translating from another language without knowing the syntax, or all of the characters for that matter.

This Google search function is part of a bigger project I started to help me analyze these lengthy log files without having to read them line for line. I'm quite vision impaired due to complications of diabetes and actually reading through these long logs is very difficult. So not only is this my first attempt at writing a program, this Google search function is my first attempt at a UDF. I'm also attempting to teach myself the whole process so you can imagine that I run into quite frequent snags.

Incidentally, my "improvised" code works quite nicely except for the case sensitivity. I really appreciate your insight. Thanks again. -- SCB

[ShawnW]

I'm a bit late to the discussion but I love regex talk.

To make it case insensitive just place (?i) at whatever point you want to turn case insensitivity on. It can be at the beginning or where ever you want. You can change it back to sensitive at any time as well with (?-i)

[Spiff59]

And I would think using the "OR" operator "|" would allow you to trash the StringInStr() tests and just run everything through a single StringRegExp() statement? Make the search group something like: (.*\.exe|.*\.dll)

[ShawnW]

And I would think using the "OR" operator "|" would allow you to trash the StringInStr() tests and just run everything through a single StringRegExp() statement? Make the search group something like: (.*\.exe|.*\.dll)

When you want to do that it is best to group just the literals you want to OR together. You need not capture them though if you use (?: ) instead of ().

Also it is also best to invert greediness when using .* when you want to specify something after it such as exe or dll. Reason being if there were 2 on the same line, it would count everything up until the last match as a single match.

(.*?\.(?:exe|dll))

This will be quicker since it does not have to search a whole line twice when it gets to the end and there is no .exe

[SpotCheckBilly]

Thank you folks so much for your insight. Authenticity, who has been helping me with this function since its inception, pointed out the fact that there are volumes written about Regular Expressions. I guess I would be in the "Precursor to RegEx 101" class LOL. The case sensitivity issue reared it's ugly head because for some reason, the lines produced by these log files sometimes contain all capital letters, sometimes all lowercase and occasionally both. For instance, in the "Startup" category, for the most part the file extensions are all .exe. However, occasionally we have one that has capital letters e.g. .EXE as follows:

dRun: [ctfmon.exe] C:\Windows\System32\CTFMON.EXE

The lines which contain a .dll or.DLL extension all run under rundll32.exe as follows (note the last example which has no file path):

O4 - HKLM..Run: [NvCplDaemon] RUNDLL32.EXE C:\Windows\System32\NvCpl.dll,NvStartup

O4 - HKLM..Run: [NVHotkey] rundll32.exe C:\Windows\System32\nvHotkey.dll,Start

O4 - HKUSS-1-5-19..Run: [WindowsWelcomeCenter] rundll32.exe oobefldr.dll,ShowWelcomeCenter (User 'LOCAL SERVICE')

For these lies I removed the rundll32.exe from the string to rid myself of THAT headache.

I will try your fine examples of modifications to my RegEx and report back on my success. Once again thanks a lot. I'm pretty certain that as I move along in my project, Regular Expressions will prove invaluable in the more I learn about them, the better. Have a great weekend. -- SCB

[SpotCheckBilly]

Hey folks,

OK, I did some work over the weekend. I haven't reworked the entire Google Search function yet, but I'm still running into something odd.

Using ShawnW's example (.*?\.(?:exe|dll)), I came up with the following:

Dim $sStr, $aArray, $RegEx, $Filename

For $iI = 1 To $aItems[0]

$regEx = '\A.*\\(.*?\.(?:exe|dll)).*\z'

$aArray = StringRegExp($sStr, $regEx, 1)
$Filename= _ArrayToString($aArray)

Next

This worked out quite nicely for not only the Startup entries, but the other entries for which I need a filename returned. The code I used for the Startup entries is as follows this is within a Select... EndSelect loop):

Case StringInStr($sStr, '.exe', 0)  ;Startup Entries  

Dim $sStr, $Filename

$aArray= _StringBetween($sStr, '[', ']', -1)
$Name= _ArrayToString($aArray)

Clipput(chr(34) & $Name & chr(34) & @TAB & chr(34) & $Filename & chr(34) & @LF)

This code works great as expected for either .exe or .dll Startup entries. EXCEPT for:

dRun: [ctfmon.exe] c:\windows\system32\CTFMON.EXE

The filename in this case returns an empty string. If I got the RegEx correct, this line should have parsed properly. Suggestions? Or did I completely misunderstand ShawnW's example?

Incidentally I'm currently reworking the rest of the code and so far, this seems to be the only anomaly. Of course, I'm less than half of the way through so things may change quickly. Thanks again everyone. -- SCB

[SpotCheckBilly]

Hey everybody,

I have been reworking my entire block of code for the Google Search UDF using the RegEx that ShawnW provided. So far I've been able to work out every kink except for dealing with the uppercase filenames. For instance, this:

O20 - AppInit_DLLs: C:\PROGRA~1\Google\GOOGLE~3\GOEC62~1.DLL

returns an empty string while this:

O20 - Winlogon Notify: GoToAssist - C:\Program Files\Citrix\GoToAssist\514\G2AWinLogon.dll

Returns the file name G2AWinLogon.dll as it should. So even though the RegEx should be ignoring the case, it doesn't. This only occurs when the entire file name is uppercase characters. Any ideas?

In the meantime I will continue reworking the code for the other line types of each of the different log files. Using that RegEx to parse out the filenames has allowed me to eliminate (probably) around two thirds of the code that it took before. I don't how much difference that will make (if any) in regards to the entire project, but it certainly looks a lot neater and compact. Hope everybody has a great weekend. -- SCB

[jchd]

(?i) is the regexp switch to make it ignore casing.

Post the source string and your regexp.