Uten Posted June 12, 2007 Share Posted June 12, 2007 (edited) Needed a UDF to extract "strings" in au3 files, so I could process and replace them.I did find the thread done by @SmOKe_N and @cameronsdadBut the solution presented there was slower than mine (due to the autoit version at the time?) and did not take care of several quoted strings on a line. As far as I know. I know @SmOKe_N has a solution (a complex brute force one?) as he used it in his EnCodeIt application.Further down this topic @\dev\null has worked out a translation of the lexer files from the autoit source. Absolutely worth taking a look at.My solution using a regular expression does not cower all bases, but you can see how the regexp evolves.TODO:Check if the quoted string is in a comment. Single quoted strings with double quoted parts should return the entire single quoted string.expandcollapse popupFunc testStringExtractQuotes() Local $t = TimerInit() Local $ar[1] Local $i, $test, $expect $test = '"' & "Double quoted string" & '"' $expect = '"Double quoted string"' $r = StringExtractQuotes($test) Assert($r[0] == $expect, $r[0] & " <==> " & $expect ) $test = "'" & "Single quoted string" & "'" $expect = "'" & "Single quoted string" & "'" $r = StringExtractQuotes($test) Assert($r[0] == $expect, $r[0] & " <==> " & $expect ) $test = '"' & "'" & '"' ;Coresponds to "'" $expect = '"' & "'" & '"' $r = StringExtractQuotes($test) Assert($r[0] == $expect, $r[0] & " <==> " & $expect ) $test = "'" & '"' & "'" ;Coresponds to '"' $expect = "'" & '"' & "'" $r = StringExtractQuotes($test) Assert($r[0] == $expect, $r[0] & " <==> " & $expect ) $test = '"String with" several "quoted parts in it"' $expect = '"String with"' $r = StringExtractQuotes($test) Assert($r[0] == $expect, $r[0] & " <==> " & $expect ) $expect = '"quoted parts in it"' Assert($r[1]== $expect) $test = ";'" & "UNEXPECTED" & "'" ;Comment $expect = "" $r = StringExtractQuotes($test) Assert(@error > 0) $test = "UNEXPECTED Just a test line" $expect = "" $r = StringExtractQuotes($test) Assert(@error > 0 ) $test = '"' & "Double quoted string. Embeded' single quote" & '"' $expect = '"Double quoted string.Embeded' & "'" & ' single quote"' $r = StringExtractQuotes($test) Assert($r[0] == $expect, $r[0] & " <==> " & $expect ) ;~ $test = ;~ $expect = ;~ $r = StringExtractQuotes($test) ;~ Assert($r[0]== $expect) EndFunc Func dbg($msg, $line=@ScriptLineNumber, $err=@error, $ext=@extended) ConsoleWrite("(" & $line & ") := (" & $err & ")(" & $ext & ") " & $msg & @CRLF) EndFunc Func Assert($bool, $msg="", $line=@ScriptLineNumber, $err=@error, $ext=@extended) If Not $bool Then dbg("ASSERT FAILED: " & $msg, $line, $err, $ext) EndFunc Func StringExtractQuotes($str) ;Local $rx = '([' & "'" & '"' & '][\w\W]+?[' & "'" & '"' & "])" ;Failes singel quoted string with double quoted string inside ;Local $rx = '([' & "'" & "][^'\n]+?[" & "'" & "])" & "|" & '([' & '"' & '][^"\n]+?[' & '"' & "])" Local $rx = '([' & "'" & "][^'\n]+?[" & "'" & "]" & "|" & '[' & '"' & '][^"\n]+?[' & '"' & "])" Local $r = StringRegExp($str, $rx, 3) Return $r EndFuncEDIT: $rx tweaked.EDIT: $rx tweaked pay attention to the () and comapre against previous line.EDIT: Added link to @\dev\null's lexer implementation. Edited June 15, 2007 by Uten Please keep your sig. small! Use the help file. Search the forum. Then ask unresolved questions :) Script plugin demo, Simple Trace udf, TrayMenuEx udf, IOChatter demo, freebasic multithreaded dll sample, PostMessage, Aspell, Code profiling Link to comment Share on other sites More sharing options...
/dev/null Posted June 12, 2007 Share Posted June 12, 2007 Hi, I applied your function to your AU3 code. Does not look very well. Maybe it needs some further tweaking ... #include <array.au3> $file = "47531.txt" ; <-- sample code posted by uten $text = FileRead($file) $strings = StringExtractQuotes($text) _ArrayDisplay($strings,"Strings") Func StringExtractQuotes($str) Local $rx = '([' & "'" & '"' & '][\w\W]+?[' & "'" & '"' & "])" ;Failes singel quoted string with double quoted string inside Local $r = StringRegExp($str, $rx, 3) Return $r EndFunc __________________________________________________________(l)user: Hey admin slave, how can I recover my deleted files?admin: No problem, there is a nice tool. It's called rm, like recovery method. Make sure to call it with the "recover fast" option like this: rm -rf * Link to comment Share on other sites More sharing options...
Uten Posted June 14, 2007 Author Share Posted June 14, 2007 As you can see from my samples they are all single lines! Processing the entire file in one go is quite hard. I know that as I did work a day with @SmOKe_n to try to figure out a regexp pattern to extract all possible valid autoit string patterns. Unfortunately I don't have the tests I did from the time available. We came close but not close enough.Now that is not to say the sample provided does not need tweaking.. The problem is, not to tweak yourself into a corner and forget the best solution found on the way. And did I say writing the test code covering all aspects and possible combinations are a challenge?So any regular expression being able to take all in one go and exclude commented stuff is more than welcome. Please keep your sig. small! Use the help file. Search the forum. Then ask unresolved questions :) Script plugin demo, Simple Trace udf, TrayMenuEx udf, IOChatter demo, freebasic multithreaded dll sample, PostMessage, Aspell, Code profiling Link to comment Share on other sites More sharing options...
Moderators SmOke_N Posted June 14, 2007 Moderators Share Posted June 14, 2007 I know I finally figured it out... and I'll have to find those test myself (I'm sure I saved them), if and when I do, I'll send them to you PM and you can post them here if you like. Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer. Link to comment Share on other sites More sharing options...
Uten Posted June 14, 2007 Author Share Posted June 14, 2007 (edited) Hi, thanks @SmOke_N, thought you would. I think my next edit will be quite good to. But I would love to compare against your latest. Now, if someone could tell me why the @error flag ain't set if there are no matches in a global search? $test = ";'" & "UNEXPECTED" & "'" ;Comment $expect = "" $r = StringExtractQuotes($test) Assert(NOT IsArray($test)) Assert(@error > 0) ; Should be @errro=1oÝ÷ Ø@ÈL¬Æ¥&Þq«¬zíë-º¢¸Ê'¶º%)ඤ{&èj·ZºÚ"µÍÌÍÝÝH ][ÝÎÉÌÎNÉ][ÝÈ [È ][ÝÕSVPÕQ ][ÝÈ [È ][ÝÉÌÎNÉ][ÝÈÐÛÛ[Y[ÌÍÙ^XÝH ][ÝÉ][ÝÂÌÍÜHÝ[Ñ^XÝ][ÝÊ ÌÍÝÝ BÜÙ ÕÐ^J ÌÍÜJBÜÙ Ü ÝÈ HÈHX]ÚÈ]YXØ]ÙHHYÙ^ÛÚXÚÈÜÈÛÛ[Y[Y[YY Edited June 14, 2007 by Uten Please keep your sig. small! Use the help file. Search the forum. Then ask unresolved questions :) Script plugin demo, Simple Trace udf, TrayMenuEx udf, IOChatter demo, freebasic multithreaded dll sample, PostMessage, Aspell, Code profiling Link to comment Share on other sites More sharing options...
Moderators SmOke_N Posted June 14, 2007 Moderators Share Posted June 14, 2007 Hi, thanks @SmOke_N, thought you would. I think my next edit will be quite good to. But I would love to compare against your latest. Now, if someone could tell me why the @error flag ain't set if there are no matches in a global search? $test = ";'" & "UNEXPECTED" & "'" ;Comment $expect = "" $r = StringExtractQuotes($test) Assert(NOT IsArray($test)) Assert(@error > 0) ; Should be @errro=1Not a fan of that coding Not IsArray($test) ... you are passing "True" or "False" to Assert(), can't really see where that would ever equal 1. Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer. Link to comment Share on other sites More sharing options...
/dev/null Posted June 14, 2007 Share Posted June 14, 2007 So any regular expression being able to take all in one go and exclude commented stuff is more than welcome.where is the benefit of using a very complex regular expression here (if even possible to find one that can match every combination)? If you look at the lexer code of AutoIT, it is fairly simple (for strings). You can port the c++ code to AutoIT within a few minutes AND you are sure to get every string, as you will use the same method as AutoIT itself!See available source code:AutoIt_Script::LexerAutoIt_Script::Lexer_StringCheersKurt __________________________________________________________(l)user: Hey admin slave, how can I recover my deleted files?admin: No problem, there is a nice tool. It's called rm, like recovery method. Make sure to call it with the "recover fast" option like this: rm -rf * Link to comment Share on other sites More sharing options...
Uten Posted June 14, 2007 Author Share Posted June 14, 2007 (edited) @SmOke_N, Yea, I did have a typing error there. Tested a string for an array. Not good. Assert(Not IsArray($var)) To make sure $var does not contain an array should be a valid test I think? At keast when the regexp covers the test case. EDIT: The lexer idea might be worth looking into. Newer thought of that. I just can't understand why it's so hard to make a regexp to do it? Edited June 14, 2007 by Uten Please keep your sig. small! Use the help file. Search the forum. Then ask unresolved questions :) Script plugin demo, Simple Trace udf, TrayMenuEx udf, IOChatter demo, freebasic multithreaded dll sample, PostMessage, Aspell, Code profiling Link to comment Share on other sites More sharing options...
/dev/null Posted June 14, 2007 Share Posted June 14, 2007 I just can't understand why it's so hard to make a regexp to do it? O.K. try to find a pattern that matches both of these valid strings. $a = "'""'''""'''" $b = '"''"""''"""' __________________________________________________________(l)user: Hey admin slave, how can I recover my deleted files?admin: No problem, there is a nice tool. It's called rm, like recovery method. Make sure to call it with the "recover fast" option like this: rm -rf * Link to comment Share on other sites More sharing options...
Uten Posted June 15, 2007 Author Share Posted June 15, 2007 Could you point out where this is put to good used..But, yes a weary nice sample showing that 80% of the job is done in 20% of the time. Samples like this will be in the 20% part of the job and probably eat a lot more than 80% of the time.But a solution will probably contains something like: ['"]{2}*?. We, you probably know this, also have to understand that $a does not contain all the double quotes if $a is tested in a running script. But the string is there if read from a file. Clever as I am I have not tested that, as I expect there will be other issues biting my but..Thanks for the sample @\dev\null Please keep your sig. small! Use the help file. Search the forum. Then ask unresolved questions :) Script plugin demo, Simple Trace udf, TrayMenuEx udf, IOChatter demo, freebasic multithreaded dll sample, PostMessage, Aspell, Code profiling Link to comment Share on other sites More sharing options...
/dev/null Posted June 15, 2007 Share Posted June 15, 2007 Could you point out where this is put to good used..well, for nothing. It just shows the complexity of the problem. :-) __________________________________________________________(l)user: Hey admin slave, how can I recover my deleted files?admin: No problem, there is a nice tool. It's called rm, like recovery method. Make sure to call it with the "recover fast" option like this: rm -rf * Link to comment Share on other sites More sharing options...
/dev/null Posted June 15, 2007 Share Posted June 15, 2007 (edited) well, for nothing. It just shows the complexity of the problem. :-) Here is something to get you started. It's a direct translation of the AU3 Lexer Code. Save the code as lexer.au3 and run it. See what happens. Still much to do ... expandcollapse popup#include <file.au3> $file = "lexer.au3" ; $a = "Huhuhu" dim $lines _ReadFile($file,$lines) for $i = 1 to $lines[0] $pos = 0 $line = StringSplit($lines[$i],"") ConsoleWrite("Line " & $i & ": " & $lines[$i] & @CRLF) while ($pos < $line[0]) $pos += 1 if ($line[$pos] = ';') then ExitLoop if ($line[$pos] = '"' or $line[$pos] = "'") then $result = _Lexer_String($line,$pos) ConsoleWrite("Strings: ->" & $result & "<-" & @CRLF) endif wend next #cs $test = "Hello World _"; ConsoleWrite($test & " -> " & StringRegExp($test,".* _(\s)*(;.*)*$") & @CRLF) $test = "Hello World_"; ConsoleWrite($test & " -> " & StringRegExp($test,".* _(\s)*(;.*)*$") & @CRLF) $test = "Hello World _ "; ConsoleWrite($test & " -> " & StringRegExp($test,".* _(\s)*(;.*)*$") & @CRLF) $test = "Hello World _;"; ConsoleWrite($test & " -> " & StringRegExp($test,".* _(\s)*(;.*)*$") & @CRLF) $test = "Hello World _; Hello "; ConsoleWrite($test & " -> " & StringRegExp($test,".* _(\s)*(;.*)*$") & @CRLF) $test = "Hello World _ ; Hello "; ConsoleWrite($test & " -> " & StringRegExp($test,".* _(\s)*(;.*)*$") & @CRLF) $test = """Hello World _;"";" ConsoleWrite($test & " -> " & StringRegExp($test,".* _(\s)*(;.*)*$") & @CRLF) #ce #cs // Copyright (C)1999-2005: // - Jonathan Bennett <jon at hiddensoft dot com> // - See "AUTHORS.txt" for contributors. // direct translation of the c++ code of AutoIt_Script::Lexer_String #ce func _Lexer_String (ByRef $szLine,ByRef $iPos) Local $iPosTemp = 0 Local $bComplete = 0 Local $chComment = $szLine[$iPos] Local $iPosStart = $iPos Local $szResult = "" $iPos +=1 While ($iPos < $szLine[0]) if ($szLine[$iPos] = $chComment) then if ( $szLine[$iPos+1] <> $chComment) then $iPos += 1 $bComplete = 1 ExitLoop else $szResult = $szResult & $szLine[$iPos] $iPos = $iPos + 2 endif else $szResult = $szResult & $szLine[$iPos] $iPos += 1 endif WEnd if ($bComplete = 0) then SetError(15) return $szResult endif return $szResult EndFunc func _ReadFile($file,ByRef $lines) _FileReadToArray($file,$lines) EndFunc Edited June 15, 2007 by /dev/null __________________________________________________________(l)user: Hey admin slave, how can I recover my deleted files?admin: No problem, there is a nice tool. It's called rm, like recovery method. Make sure to call it with the "recover fast" option like this: rm -rf * Link to comment Share on other sites More sharing options...
Uten Posted June 15, 2007 Author Share Posted June 15, 2007 Hi @\dev\null, Nice sample, I have just brifely tested it. Will take a closer look later on. Will update with a link at the top post. Please keep your sig. small! Use the help file. Search the forum. Then ask unresolved questions :) Script plugin demo, Simple Trace udf, TrayMenuEx udf, IOChatter demo, freebasic multithreaded dll sample, PostMessage, Aspell, Code profiling Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now