New SciTE4AutoIt3 available with SciTE v3.3.7

[Melba23]

Jos,

It has not broken any of my existing old-style scripts yet.

M23

[guinness]

So far so good Jos.

[Jos]

Pulled the Beta Obfuscator because it breaks the obfuscation output so need to figure out how that regression started or really make the decision to stop with the Obfuscator option and rename the program to SourceStripper or something

Jos

[Melba23]

Jos,

 

stop with the Obfuscator option and rename the program to SourceStripper or something

Gets my vote.

M23

Edit: But please keep the /OM bit - it speeds up the exe quite a bit.

Edited January 1, 2014 by Melba23

[guinness]

rename the program to SourceStripper or something 

Yeah, the obfuscation part is outdated and as we have seen people think it protects their script.

[Jos]

Already decided a while back to stop the support of the Obfuscation process and it is just a matter of putting in the time to make the changes to the AutoIt3Wrapper directives and GUI. Planning to do this for the next version of SciTE4AutoIt3.

I've found the regression in the mean time and also implemented some speed improvements (10-20%) for /StripOnly.

Uploaded Obfuscator v 1.1.0.3 to Beta.

Jos

[guinness]

Interesting that you managed to improve the stripping function.

[Jos]

Edit: But please keep the /OM bit - it speeds up the exe quite a bit.

No issue to keep that option.

Jos

[FaridAgl]

Could we have both PreExpand and this new SourceStripper's features in one tool?

I'm pointing to replacing all const/enum with their values.

Just gonna be nice.

[Jos]

Could we have both PreExpand and this new SourceStripper's features in one tool?

 

The "NEW" source stripper will actually be the old Obfuscator program with the Obfuscation part stripped out and the default changed from Obfuscation to /StripOnly.

The PreExpand feature build by guinness in the PreExpand script is not something I am totally convinced yet this is something that really contributes to a substantial size reduction or speed improvement, but I have stated the same to guinness that this really should be a feature that is part of the current Obfuscator process assuming it is worth it.

Maybe people can share their thoughts on this?

Jos

Edited January 2, 2014 by Jos

[guinness]

Well it's sort of a happy medium of using #define in a script without the fuss of adding functionality to either AutoIt3Wrapper of a secondary process in between the two.. I would love it if we had a pre-processor directive where I could define a name with a value that would be replaced before compilation, but we don't unfortunately. So we then get into a situation where people use "magic numbers" and make code hard to read for others just because they see a constant variable as "taking up memory". Silly really.

So, I created PreExpand where the pre-existing Const keyword is used instead of a new directive and thus a script can run normally in or out of SciTE.

Edited January 2, 2014 by guinness

[FaridAgl]

AutoIt doesn't compiles the source to machine language, it means a copy of your written script will be merged to the exe.

When the compiled script got run, the core part of AutoIt (AutoItSC) will read the script line by line and execute what our script should actually do.

Within the AutoItSC there is some kind of text-processor which deals with our written script (Read & Execute). There is also variable/function table, which holds the track of our variables/functions etc.

In programming languages like C++, the code will be converted to machine code in compile time, it means on run-time all of the codes we have written will be copied to the ram by loader, and then the processor knows the rest. There is nothing between what we write in the source, and what is getting executed by processor.

About AutoIt, it's different (See above). Every single character will affect the speed of our script, every single variable too, also the length of variable/function names as well.

Again, in C++ for example, before compiling the following code, the pre-processor will convert:

#define Constant1 0x11223344
#define Constant2 0xAABBCCDD
#define Constant3 "A string..."

DWORD a = Constant1 + Constant2;
MessageBox(NULL, Constant3, "", MB_OK);

To:

DWORD a = 0x11223344 + 0xAABBCCDD;
MessageBox(NULL, "A string...", "", MB_OK);

In AutoIt, we don't have a compiler, just a tool which merge the source to the AutoItSC, also some encryption, handling the resource, manifest etc, but in fact, the source is what we have written, not more, not less.

Now, that we don't have an actual compiler, we must take all we can from a pre-processor, which removes all comments, spaces, empty lines, unused variables, unused functions, and also replaces all of the constants and enum variables with their actual value (See the example of C++ above).

For you to know, I remember some times ago, I have compiled something in C#, then a little after that I decided to decompile my compiled C# exe for some reasons, guess what?

All of the variables and functions where renamed, take a look:

From this:

int MyGlobalVariable1;
int MyGlobalVariableWithLongName;
int MyGlobalVariableWithARellyLongLongNameWhichIsReallyLongIsntIt;

void HiImAFunction(int IHaveAName, int IHaveANameTooButNotForEver)
{
    int GuessWhoIAm = IHaveAName + IHaveANameTooButNotForEver;
}

void IAmTheLongestFunctionNameEverExistedSoShowSomeDamnRespect(int IAmTheParameterOfTheLongestFunctionYouEverSeen)
{
    int LetMeHoldTheResultForYou = MyGlobalVariable1 + MyGlobalVariableWithLongName
                                    + MyGlobalVariableWithARellyLongLongNameWhichIsReallyLongIsntIt
                                    + IAmTheParameterOfTheLongestFunctionYouEverSeen;
}

To this:

int a;
int b;
int c;

void a(int a, int b)
{
    int c = a + b;
}

void b(int d)
{
    int e = a + b + c + d;
}

If I was going to do this, I would convert the above code to this!:

int a;int b;int c;void a(int a,int b){int c=a+b;}void b(int d){int e=a+b+c+d;} 

It decreases the file size and also increases the code execution time,

So we also need our AutoIt pre-processor to do this for us too.

Sorry for the wall of text, just want to say my opinion.

Also I'm not requesting/forcing anyone to anything, just my opinion and what I knew.

Please correct me if I'm wrong.

Edit:

Sorry for the typo and misspelling, English is not my native language.

Edited January 2, 2014 by D4RKON3

[FaridAgl]

@Jos

As I'm a fan of your obfuscator's /CF option, here is a feature request.

It's been for a while that I'm using AutoItObject in my scripts, I'm not here to talk about that but obfuscator's /CF option doesn't handles functions which are used as methods of claess,

Here is a simple example showing the issue:

expandcollapse popup

Func Accounts()
    Local $hDatabase = _SQLite_Open("Databases\Accounts.db")

    Local $sQuery = ""
    $sQuery &= "CREATE TABLE IF NOT EXISTS Accounts("
    $sQuery &= "Username TEXT PRIMARY KEY NOT NULL COLLATE NOCASE,"
    $sQuery &= "Password TEXT NOT NULL,"
    $sQuery &= "Servers INTEGER NOT NULL,"
    $sQuery &= "RoomId INTEGER NOT NULL,"
    $sQuery &= "RoomName TEXT NOT NULL,"
    $sQuery &= "ExpiryDate TEXT,"
    $sQuery &= "Comments TEXT"
    $sQuery &= ");"

    _SQLite_Exec($hDatabase, $sQuery)

    Local $oClass = AutoItObject_Class()
    With $oClass
        .AddProperty("Base64", $ELSCOPE_PRIVATE, Base64())
        .AddProperty("Misc", $ELSCOPE_PRIVATE, Misc())

        .AddProperty("DatabaseHandle", $ELSCOPE_PRIVATE, $hDatabase)

        .AddProperty("Type", $ELSCOPE_READONLY, Accounts_Type())

        .AddMethod("Account", "Accounts_Account", True)

        .AddMethod("Select", "Accounts_Select")
        .AddMethod("Insert", "Accounts_Insert")
        .AddMethod("Update", "Accounts_Update")
        .AddMethod("Delete", "Accounts_Delete")
        .AddMethod("Import", "Accounts_Import")

        .AddMethod("Usernames", "Accounts_Usernames")
        .AddMethod("IsDuplicated", "Accounts_IsDuplicated")

        .AddDestructor("Accounts_Destructor")
    EndWith
    Return $oClass.Object
EndFunc


Func Accounts_Destructor($this)
    _SQLite_Close($this.DatabaseHandle)
EndFunc

Func Accounts_Type()
    ;...
EndFunc

Func Accounts_Account($this, $sUsername, $sPassword, $iServers, $iRoomId, $sRoomName, $sExpiryDate, $sComments)
    ;...
EndFunc

Func Accounts_Select($this, $sUsername)
    ;...
EndFunc

Func Accounts_Insert($this, $sUsername, $sPassword, $sServers, $sRoomId, $sRoomName, $sExpiryDate = "", $sComments = "")
    ;...
EndFunc

Func Accounts_Update($this, $sUsername, $sPassword = Null, $sServers = Null, $sRoomId = Null, $sRoomName = Null, $sExpiryDate = Null, $sComments = Null)
    ;...
EndFunc

Func Accounts_Delete($this, $sUsername)
    ;...
EndFunc

Func Accounts_Import($this, $sFilePath, $fReplace)
    ;...
EndFunc

Func Accounts_Usernames($this, $iType, $sSearch = "")
    ;...
EndFunc

Func Accounts_IsDuplicated($this, $sUsername)
    ;...
EndFunc

As you can see, function names are referred in .AddMethod and .AddDestructor as string, if I run obfuscator on this script all of the functions will be stripped out and this will break the whole script.

I saw obfuscator takes care of functions referred as string in GUICtrlSetOnEvent, GUIRegisterMsg, etc... so just was thinking is it possible to have the same feature for this two methods as well (.AddMethod and .AddDestructor).

Currently I'm using #Obfuscator_Ignore_Funcs for all of my functions within a class, but I'm loosing the /CF option on them :|

Would be great if you could do anything.

Thanks.

Edited January 2, 2014 by D4RKON3

[UEZ]

I still get the em:
 

+>01:24:04 Starting AutoIt3Wrapper v.2.1.4.0 SciTE v.3.3.7.0 ;  Keyboard:00000407  OS:WIN_7/Service Pack 1  CPU:X64 OS:X64    Environment(Language:0409  Keyboard:00000407  OS:WIN_7/Service Pack 1  CPU:X64 OS:X64)

>Running AU3Check (3.3.10.2)  from:C:Program Files (x86)AutoIt3

+>01:24:06 AU3Check ended.rc:0

! Input file is UTF8 encoded. Obfuscator does not support this and will be skipped.

although it should be fixed

...

12/28/2013
*** Merged the updates of SciTE v 3.3.7 by Neil Hodgson with our own version of SciTE. (Jos)
*** Updated Production config files to 3.3.10.0
*** Updated AutoIt3Wrapper v2.1.4.0 (Jos)
    - Use "c:\users\\Appdata\Local\Autoit v3\Aut2Exe" as Temp directory for compiling and resource updating like aut2exe.
    - Fixed issue with Compile both and Pragma(x64,true)
    - Fixed inputfile in case Obfuscator needs to run but is skipped due to unsupported utf format.
    - Fixed UPX selection option to default to N and adding the Directive when checkbox is checked.
*** Updated Obfuscator.exe v1.0.31.1 (Jos)
    - Changed the code to leave #include statements that include a a3x file.
...

Br,

UEZ

[FaridAgl]

Last night, after I have posted my previews post, I started to imagining that if obfuscator has a way which I could add some function name to a list (file) which obfuscator would handle their parameters as function names, and handles them in /SO and /CF option, that was just great.

Today, after I have checked this thread to see if there is any response, in the previews page I saw the latest obfuscator for download, I have downloaded that and when I was going to copy the file to obfuscator's folder guess what? I saw there is a file named Obfuscator.dat. For some unknown reasons I just opened that in a text editor and I faced the following texts:

Functionname,Parameter to check,0=check for function/1=check for Variable
adlibenable,1,0
adlibregister,1,0
adlibunregister,1,0
call,1,0
dllcallbackregister,1,0
eval,1,1
guictrlregisterlistviewsort,2,0
guictrlsetonevent,2,0
guiregistermsg,2,0
guisetonevent,2,0
hotkeyset,2,0
isdeclared,1,1
objevent,2,0
onautoitexitregister,1,0
onautoitexitunregister,1,0
opt,2,0
traysetonevent,2,0
trayitemsetonevent,2,0

It was a CSV file which holds the name and parameter number of functions which their parameters should be treated as function names.

Did you get it?!

Just added to following line and what I had posted in my previews post got solved:

addmethod,2,0
adddestructor,1,0

Wondering how close the answers could be and how much blind we (Talking about myself) could be! 

Edited January 3, 2014 by D4RKON3

[Jos]

@Jos

As I'm a fan of your obfuscator's /CF option, here is a feature request.

This is an Obfuscation task which won't be in the next version of Obfuscator/Au3Stripper.

As to your feature request: I see you found the table that is used by OBfuscator to scan for literal string Func or Variable names in functions.

This table will remain to facilitate the stripping functionality as that also needs to know whether or not Func's are used.

Jos

[FaridAgl]

OK, thanks.

And, anything about ?

[Jos]

Which part of the "text wall" wasn't covered yet?

When it is about PreExpanding variables in the source by Au3Stripper/Obfuscator, I can tell you I am still the fence at this moment as there are many pitfalls that needs handling.

I am also still wondering if there is a notable improvement in speed or size and haven't seen much input from others that proves otherwise.

Jos

Edited January 3, 2014 by Jos

[Jos]

I still get the em:

 

+>01:24:04 Starting AutoIt3Wrapper v.2.1.4.0 SciTE v.3.3.7.0 ;  Keyboard:00000407  OS:WIN_7/Service Pack 1  CPU:X64 OS:X64    Environment(Language:0409  Keyboard:00000407  OS:WIN_7/Service Pack 1  CPU:X64 OS:X64)

>Running AU3Check (3.3.10.2)  from:C:Program Files (x86)AutoIt3

+>01:24:06 AU3Check ended.rc:0

! Input file is UTF8 encoded. Obfuscator does not support this and will be skipped.

although it should be fixed

...

12/28/2013
*** Merged the updates of SciTE v 3.3.7 by Neil Hodgson with our own version of SciTE. (Jos)
*** Updated Production config files to 3.3.10.0
*** Updated AutoIt3Wrapper v2.1.4.0 (Jos)
    - Use "c:\users\\Appdata\Local\Autoit v3\Aut2Exe" as Temp directory for compiling and resource updating like aut2exe.
    - Fixed issue with Compile both and Pragma(x64,true)
    - Fixed inputfile in case Obfuscator needs to run but is skipped due to unsupported utf format.
    - Fixed UPX selection option to default to N and adding the Directive when checkbox is checked.
*** Updated Obfuscator.exe v1.0.31.1 (Jos)
    - Changed the code to leave #include statements that include a a3x file.
...

Br,

UEZ

Which entry in the History log makes you think that Obfuscator supports UTF8 encoded files?

It was AutoIt3Wrapper that didn't handle the Obfuscator exiting without producing the scriptfile_Obfuscated.au3 files correctly that was fixed.

Jos

[FaridAgl]

>PreExpand already does the job, however it needs "a little" more improvements.

It's mostly about the amount of memory usage, not the script size. Maybe a developer could answer this.

For the script size, /CF and /CV are great.

- Shouldn't obfuscator strip out all #forceref lines? Because it doesn't now.