Code Scanner

[guinness]

I should have written Firefox. I'm so used to writing your username than the web browser.

[FireFox]

I should have written Firefox. I'm so used to writing your username than the web browser.

Exactly. If this goes on I will be better referenced than the browser itself 

@RTFC

Apologies, end of off-topic.

Edited August 8, 2013 by FireFox

[RTFC]

Hey, I like off-topic!

And thanks for the welcome, everyone.

Re, using Chrome (do we have a forum member named Chrome too? Better check...), just tested FF on another machine and RTE works there, so my personal environment must be to blame (but haven't nailed it down yet).

@guinness: re. globals declared upfront, yes, I got it that you referred to my code; I just thought, there are probably other lazy coders like me that might find such a feature useful.

Can't type anymore on this ridiculously tiny keyboard, feel like a whale playing a piano; laters...

Edited August 8, 2013 by RTFC

[RTFC]

New Version 0.8.5 coming soon in first post.

New features:

* reads Autoit keywords directly from api (thanks guinness!), so should reflect whatever version of AutoIt you are currently using

* INI file for procedural booleans and AutoItIncludes path (for non-default installs)

* Treeviews can now be string-searched like in Scite (Ctrl-F, F3, Shift-F3, with forward (but not backward) wraparound)

* more extensive support for globals:

• multiple globals per line (comma-separated)
• 2-D arraydisplay of globals' references
• listing of globals defined only within UDFs
• *.au3 output of non-native global definitions per #include, with source reference comment

Internally, the code is slightly less messy than initially. Of course, new bugs may have been introduced too.

 

BTW, notice them there colourful colours?

After a complete uninstall and bare-bones reinstall, my Firefox now finally allows me to access the RTE. Shiny!

The fact that the original demented version maintained against all counter-evidence that it was running on a Linux machine may have had something to do with it, although I remain in the dark as to the underlying cause. Oh well, I've put it out of my misery.

Edited August 9, 2013 by RTFC

[guinness]

I tested on the codescanner Au3 script file and the results were incorrect. Did you try it on that file?

[RTFC]

Thanks for letting me know so quickly.

I'm looking into it. Could you perhaps elaborate on what results are incorrect? Thanks.

So far I have found a few non-native globals listed as defined only within UDFs, that are actually predeclared.

Just fixed this, and re-uploaded the zip (same version number).

Oh I see, the report lists some includes as redundant that aren't. My bad. Been completely focussed on those globals. Apologies for uploading a new version too early. I guess I'll reinstate the original version for now...

Edited August 9, 2013 by RTFC

[Iczer]

i get Msgbox right after start:

Fatal I/O error: maximum linelength exceeded in E:\***.au3, line: 0

but there a no zero line in my script...

[RTFC]

@Iczer: Thanks for testing and for spotting that. The error is checked for before the linecount is incremented, this will be corrected in the imminent new update.

The message means that codescanner (function _GetNextLine) is unable to find the first linebreak (@CRLF) within the first 4K of the currently analysed file. If the file concerned is a valid (.au3) text file, please check with a hex editor whether it contains hexcodes 0D0A at the end of each line. If so, then I would have to have a look at that sourcefile (or at least the first few lines of it) to be able to reproduce the error.

@guinness: Right, in my enthusiasm to implement PArseAPI I'd added not just your array Functions, but also FunctionsUDF, thereby excluding at a stroke all UDF extensions, so none of these were ever registering incoming calls, and thus were flagged as redundant if they also contained no globals. Thanks a bunch for spotting it so quickly!

I've re-uploaded the corrected updated source now in the first post. If you were referring to other errors in your last post, then please let me know. I'm still testing for other mistakes / newly introduced bugs myself, of course.

Edited August 9, 2013 by RTFC

[Iczer]

@Iczer: Thanks for testing and for spotting that. The error is checked for before the linecount is incremented, this will be corrected in the imminent new update.

The message means that codescanner (function _GetNextLine) is unable to find the first linebreak (@CRLF) within the first 4K of the currently analysed file. If the file concerned is a valid (.au3) text file, please check with a hex editor whether it contains hexcodes 0D0A at the end of each line. If so, then I would have to have a look at that sourcefile (or at least the first few lines of it) to be able to reproduce the error.

 

first lines is nothing special - it's just:

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
...

but - script encoded as UTF-16 Little Endian. Maybe it's a reason...

tried new version - now it's:

Fatal I/O error: maximum linelength exceeded in E:\***.au3, line: 1

Edited August 9, 2013 by Iczer

[RTFC]

Aha, yes that would explain it. Again, good point, I should have made clear in the header that codescanner expects standard Windows ASCII files, like UTF-8 with only ASCII characters. As far as I know, Windows would normally store programme code text files as ASCII (at least in the UK/US versions I know), even though the OS internally uses UTF-16. But I'm no expert on file encoding standards. Perhaps one of the more experienced members can explain about the situation within AutoIt. I wouldn't be surprised if somebody has already produced a Unicode converter UDF. Would that mean that the maximum line length is 8K bytes for unicode, or still 4K bytes = 2/1K chars (UTF-16/32)?

Anyway, as you can see within the source code, the line nr reported is simply adjusted for the message ahead of the internal counter, which is incremented only after the next line has become defined. It unfortunately doesn't help you at all, I'm afraid.   I'll have a look how involved Unicode  support  would be to implement, but because there are so many flavours, I fear it might be a gargantuan task. Anyone have experience with this?

Okay, just found this:

http://www.autoitscript.com/autoit3/docs/intro/unicode.htm

So basically it's possible if relying on Filereadline. That's funny because the very first test version of the code actually still used that, but I quickly replaced that with struct I/O because it was very, very, very slow. But if you don't mind waiting for an analysis, it wouldn't take much to add a boolean in the INI file and reinstate the original proc as backup option. I'm adding it to my to-do list, okay, it will be in the next release. Beware though, file I/O is the main bottleneck for speed in my code, so expect a significant performance hit on an already glacial pace. Stay tuned...

Edited August 9, 2013 by RTFC

[RTFC]

New release v.0.9 uploaded.

Added Features:

• Unicode support (UTF8/16, and whatever else AutoIt natively supports, see link in previous post)
• Alternative optional Treeview Menu interface (navigate results by subject (general, includes, UDFs, globals) or by output format (text, treeview, arraydisplay 1/2-D))

With many thanks to PsaltyDS for the treeview-doubleclick-handling routine, found here:

'?do=embed' frameborder='0' data-embedContent>>

Unicode support wasn't such a performance slowdown after all, I'm wondering whether in earlier timings I may have compared cached and non-cached...

As this code is my first implementation of treeview controls, the new, alternative Menu probably still  has some glitches, but seems to me more intuitive to navigate than that listview I started out with. A few questions that arose while making it:

1. Is it possible to disable treeview itemlabel editing (so the content of the labels is read-only)? Couldn't find the appropriate flag in the Style settings

2. Is it possible to disable the expanding/collapsing of subtrees? (I realise this goes directly against the designed intention of a treeview, just wondering)

Of course I'd appreciate any reported bugs, but I won't be able to respond in the next few days.

[RTFC]

New Release 1.1 uploaded.

New Features:

• Settings Panel (booleans, interface, AutoIt Includes path)
• Timed Processing (displayed in Treeviewmenu only)

You can now switch show / hide duplicate calls/includes in treeviews from the (new) Settings panel, and new treeviews will reflect this directly (previously one had to either edit the code or the INI file, and restart). Dupes filtering is now done in _TreeView() instead of Build*Tree() funcs.

Minor fixes include:

extrapaths inclusion

treeview labels read-only

various cosmetic changes

[RTFC]

New Release 1.3 (major revision) uploaded

Removed:

• Listview Menu (dynamic updates of display options are already hard enough with one GUI)
• nasty multi-line bug

New features:

• Now Single-Pass! (significantly faster , but also far more processing than previously, so it;s still fairly slow )
• Tracked Features (2-D array) is now sequential, easier to compare to your source
• All $variables are tracked
• All literal strings are tracked (optional, toggle in Settings Panel or INI file; does take longer when enabled), and can be matched with listed parameters in the "Tracked Features" List (a.k.a. the 2-D array $references)
• All native AutoIt functions are tracked, with all parameters
• native AutoIt calls can be included in the Functions TreeView (but set Showduplication=True, or most remain hidden)
• Tree building can be interrupted by pressing <Esc>
• line processing supports doubled quotes within (extracted) strings
• New option Data Dump can store all results as separate text files in a special subdirectory
• additional UDF ReadCSdatadump.au3 (included in latest zip) reads all Data Dump contents back into arrays and variables, so you can use all CodeScanner results directly in your own scripts
• internal reorganisation and many small changes

Obviously, the above constitutes a fairly drastic overhaul, likely introducing new bugs. String processing is probably most prone to glitches, because I only added it today.

Bug reports (preferably with a tiny code sample that reproduces the error), comments, and suggestions for improvement are welcome as always (although I'm gradually moving on to other things now...)

Edited August 30, 2013 by RTFC

[RTFC]

New release 1.4 uploaded.

Added Features:

• many tracking options are now optional, to speed up processing if you don't need the maximum amount of output (adjust in Settings panel or INI file). Large performance penalty: UDF, AU3 call tracking (obviously); Moderate: $variables tracking; rest is minor.
• Main Code Sections (MCS, executable code outside of global/UDF definitions) tracking; it's less obvious than it sounds; MCS may occur in any #include file, as well as in your root file, and #includes may be inserted anywhere, so which sections of code in which file get executed in what order?
• Entry point tracking; when you run the root, which line in which file (root or some #include) is the starting point for actual execution (rather than definition); this is because I use dozens of (nested) includes per project, and anyone of those may have a little testing section for the UDFs it contains, which I tend to forget to comment out when using them in larger builds, so instead of my rootfile's main code it starts running those tests, dumping diagnostics, messing up globals, or worse... Finding these miscreants used to take me ages. Not anymore.
• Exit point tracking; where does your code actually terminate? at an explicit Exit statement, or an implicit exit (end of main code section), or perhaps a conditional Exit inside of a UDF, somewhere deep in your includes tree? All occurrences tracked, warnings added in report if exit point is implicit and outside of root
• Loop tracking; for/next, While/Wend, Do/Until, with nesting level, pair mismatch, and parameter checks
• Function tracking now supports AdlibRegister and Call (providing the function is defined as a single literal string)
• strings and variables listings are now available both sequential and sorted
• internal optimisations, minor bug fixes, cleanup, more annotation, enlarged Remarks section at top of script
• In total now 33 optional outputs

 

[RTFC]

Most utilities you write, debug, use and eventually forget about. But there are projects that just refuse to let go. They invade other projects. They intrude on your thoughts, uninvited. They even mess with your dreams. Anyone familiar with this? It's like a catchy tune you can't get out of your head. So here's another attempt to put this sucker to bed ((I need to sleep again, please...)

Yap, it's another major upgrade, version 2.0

What's new? Let's start with the little stuff:

Added support for:

• guisetonevent
• guictrlsetonevent
• hotkeyset
• guiRegisterMsg
• adlibunregister
• OnAutoItExitRegister func calls
• macros
• redundant globals
• global Enum (Step *) (this one was exceptionally nasty)

and some cosmetic changes to the treeview menu:

• loaded once, loads much faster thereafter
• simplified labels
• you can change in the settings which part(s) of the tree is/are shown "exploded"

So far, so boring (does anyone read these trailing posts anyway, or just the first one? Hello World? Are you there?). So what warrants a major version number bump? Three letters: M, C, F.

MCF = MetaCode Files. You can read all about it in the Tutorial I prepared here:

'?do=embed' frameborder='0' data-embedContent>>

CodeScanner now requires MCF.au3 (please download from the above thread)

MCF.au3 in turn requires MCFinclude.au3 (part of the same MCF package)

MCFinclude.au3 requires AES.au3, by Ward (link provided in the MCF thread)

Still with me?

 

CodeScanner has a new Settings switch WriteMetaCode. When True, CS generates MetaCode files and arrays, used as inputs for the MCF UDFs and its front-end CodeCrypter  (yeah, been kinda busy); all released today. So please read the MCF tutorial.

If you have feedback re. CodeScanner's end of this convoluted contraption, post here, otherwise please post in the MCf thread. Thanks.

Edited October 19, 2013 by RTFC

[RTFC]

Version 2.1 released, MCF-related bug fixes and minor improvements only.

[RTFC]

Version 2.2 released; minor bugs (introduced in the previous version ) fixed.

[JohnOne]

Don't know how I have not seen this until now.

Marvelous.

[RTFC]

Hey JohnOne,

Much obliged!

BTW, your many helpful comments throughout the AutoIt forums have often clarified issues I would have otherwise had to ask. Thanks!

RT

[JohnOne]

Great, then you won't mind me ripping off parts of your code then

Seriously, I've not had a chance to study it yet, but it sounds like a bit of stuff I'll be needing for a long term slow moving project is included in your scripts.

Of course you can tell me to keep walking