source code for AutoitSC.bin?

[jfrandol]

Hi,

A few days ago I found that I could not Digitally Sign an AutoIt-generated .exe file and still be able to run it.

Jon wrote in response:

>>Unfortunately, the signing modifies the exe file so that when autoit tries to scan >>it and unpack the script it won't be able to find it (the long delay) and then fail.

It appears that the code that scans for and unpacks the script from the self contained .exe is not in the downloadable source ? (I couldn't find it,I guess it would be the source code for AutoitSC.bin?)

I would make a stab at 'fixing' this if I knew where to look - since I have some experience in C (a little in C++) - and I really like AutoIT's ability to make a small .EXE with no runtime dependancies - BUT in todays world, some people would really like me to sign my small .exe's (it makes them feel better...)

Does anyone know where I should look for this code ?, or have any other thoughts or comments on this topic ?

Thanks,

John F.

[AznSai]

If you could sign the file before compiling your script would that help?

[MHz]

Some source should remain secret. Scanning and decompilation code would perhaps be the most sensitive of the source code. Signing an executable could be a bigger risk, then without it. If anyone can change the source for signing, then it would be Jon for this. Risk would need to be considered. If it is even possible?

Some archive types can be signed. This allows for safe transport of it's contents. This maybe an alternative?

[jfrandol]

thanks for your thoughts/comments.

>>If you could sign the file before compiling your script would that help?

Not really (if i understand that you are saying that I would sign the .au3 file before compiling it) , the goal is to end up with an executable that I can send out that is Signed in the normal Microsoft Code Signing way. (so when someone tries to run it from a web download link for example, they get the nice dialog that shows that it is signed and that this is the publisher and do you want to trust it, (if that publisher is not already trusted)

>>Some source should remain secret, Scanning and decompilation code would perhaps be the most sensitive of the source code.

I don't think so - see the myriad discussions on 'security through obscurity'...

Microsoft Code signing works by adding a digital signature structure to the Microsoft PE (portible Executable) format file, which we know as an .exe - the point I am trying to make is that the AutoitSC.bin extraction code as a 'bug' in that it can't extract the autoit script from a standard PE executable. (the bug may really be in the UPX code embedded in the AutoitSC.bin - or perhaps there is no bug there at all, but I think so (not having seen it so may not understand).

(ref: http://msdn.microsoft.com/library/default....gsignature.asp)

The point is, Any Executable should be able to be Signed in the standard way, and still run but Autoit created executables currently cannot... I think this is a 'bug' - or at least an inconvenience. (perhaps only to me )

thanks again for your thoughts.

John F.

[Valik]

Did it occur to you that perhaps we manipulate the PE format for our own purposes?

[Nutster]

Jon is currently rewriting how Aut2Exe works. Currently, a checksum of some sort is inserted into the executable so that the integrity of the executable can be verified, then the whole thing is compressed using UPX. If the problem is in UPX, we can not do much about it and the behaviour should verified with other executables and then brought to the attention of UPX.

The checksum is used to verify that the AutoIt program has not been manipulated/corrupted after it was made with Aut2Exe, which could render it useless. Unless we can see a way around this, I do not see this happening soon. May Jon can add an option to Aut2Exe to add a digital signature.

Edited September 1, 2005 by Nutster

[Jon]

If the signing adds data to the PE header and not the end of the file then it maybe possible to get it to work. Is there a way to sign something without having to buy a private key? I can't test otherwise.

[jfrandol]

Thanks for your time again Jon, and Valik and Nutster

OOPS- ACKK - this post is superceded by the next one - please read it first... then back to here if you want... (John F)

--create_testcert_and_sign.bat.txt

I tested the UPX compression on a seperate signed executeable (notepad.exe) using upx.exe and it failed with an error message - so the whole thing is likely a limitation of the upx library's current abilitys - so as Nutster Said , I should be bringing this to their attention first. (and they already know about it)

Below is a transcript of my test - and attached is the batch file which allows anyone with the .NET SDK 1.1 to create a test certificate and to sign an executable with it.

Thanks for all of your time.. I hope it is only a limitiation in UPX - if that turns out to be the case, perhaps one could have an option to 'turn off' upx compression for the autoit executable ? - but I do not want to take up more of your time if you don't wish to persue it - though I will be glad to help in any way I can if anyone has further interest.

---

If I read this correctly: http://msdn.microsoft.com/library/default....ngsignature.asp

Then the certificate is in the PE header (or it may be in a location specified by the directory entry in the PE header, but I think it is in the header, since it makes the statement that "the certificate is not loaded into memory by the pe loader..)

----

--transcript

C:\Program Files\AutoIt3\Aut2Exe>notepad sign_with_testcert.bat

C:\Program Files\AutoIt3\Aut2Exe>sign_with_testcert.bat notepad.exe

Warning: This file is signed, but not timestamped.

Succeeded

C:\Program Files\AutoIt3\Aut2Exe>upx_1.9.b.exe notepad.exe

Ultimate Packer for eXecutables

Copyright © 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005

UPX 1.93 beta Markus F.X.J. Oberhumer & Laszlo Molnar Feb 7th 2005

File size Ratio Format Name

-------------------- ------ ----------- -----------

upx_1.9.b: notepad.exe: CantPackException: compressing certificate info is not supported

Packed 1 file: 0 ok, 1 error.

C:\Program Files\AutoIt3\Aut2Exe>upx_1.9.b.exe --version

upx 1.93 beta

NRV data compression library 0.84

UCL data compression library 1.03

--end transcript

Edited September 1, 2005 by jfrandol

[jfrandol]

Thanks for your time again Jon, and Valik and Nutster

I tested the UPX compression on a seperate signed executeable (notepad.exe) using upx.exe and it failed with an error message - so the whole thing is likely a limitation of the upx library's current abilitys -  so as Nutster Said , I should be bringing this to their attention first. (and they already know about it)

<{POST_SNAPBACK}>

OOPS !! - my test above is really invalid - as I tested attempting to Compress a signed executable - whereas for the current AutoIT case, I just want to be able to SIGN a Compressed executable (compressed by the autoit compile process) and have it continue to run..

I can do this with notepad.exe, but not with an autoit-created.exe

Thanks,

John F.

[Valik]

PE sections can be marked as "discardable" there-by allowing the loader to free the memory associated with them (The import table is discardable, for example). The certificate or signature is probably inserted in a new PE section which is discarded by the loader since it's not needed. Depending on how applications look for this section, it may or may not be compatible with AutoIt. At any rate, the binary has to be signed before AutoIt compiles the script so you will have to compile AutoItSC.bin before compiling to have any chance. This is the same theory as replacing/modifying resources so you can find plenty of information on how to do that with AutoIt by searching the forum.

[jfrandol]

Depending on how applications look for this section, it may or may not be compatible with AutoIt. 

<{POST_SNAPBACK}>

Valik,

No applications (that I know of.. or need..) look for the signature at the application's runtime - so it can be discarded as you describe, its just at download time (or later when someone right clicks and checks the property sheet) that the signature is verified, before IE passes it on to be executed if during a download..

I just want the Entire .exe to be signed - extracting portions of it and trying to sign them does not help me..

I wonder about your comment:

>> Did it occur to you that perhaps we manipulate the PE format for our own purposes?

Because I see the PE format as a standard that should be followed, not manipulated outside the standard - but perhaps the standard has to be broken in order to compress parts of the executable in a 'non-standard' way, but I don't think it should have to be... if it does have to break it, then it would be nice to have the option to turn off the compression when compiling some script into an .exe. (perhaps I should read the standard.. - or look at the UPX source..)

Placing a standard signature in the PE format executable does not necessarily have to break the autoitsc.bin stub's ability to extract script source (or bytecode) from the PE exe and run it.. does it ? )

Thanks for your thoughts.

John F.

[Valik]

I wonder about your comment:

>> Did it occur to you that perhaps we manipulate the PE format for our own purposes?

Because I see the PE format as a standard that should be followed, not manipulated outside the standard - but perhaps the standard has to be broken in order to compress parts of the executable in a 'non-standard' way, but I don't think it should have to be...  if it does have to break it, then it would be nice to have the option to turn off the compression when compiling some script into an .exe.  (perhaps I should read the standard..    - or look at the UPX source..)

Who said anything about compression? You're stuck on this notion of compression. UPX compresses and changes the PE structure of the file and creates its own sections but that is totally irrelevant to your "problem". I also believe there is already a "don't compress" option in the beta version of the compiler so you can test this for yourself if you want.

I don't think the PE format places any limitation that things can't be added. At least I've never seen anything. There are pre-defined sections, yes, and these have identifiers for "easy" programmatic access, however, I've not seen anything saying the PE format is limited to only the Microsoft designated sections. In fact, one of the nice things about the PE format is that it's structured so that as long as you play nice, you can add virtually anything you want to a binary without breaking it.

Placing a standard signature in the PE format executable does not necessarily have to break the autoitsc.bin stub's ability to extract script source (or bytecode) from the PE exe and run it..  does it ? )

Yes, it does have to break it. If it changes the PE structure at all after AutoIt has already compiled the script into the binary, then it's very possible that AutoIt will no longer be able to find the embedded script. I suspect the same would hold true in the inverse: If you sign AutoItSC.bin and then compile it, I bet the signature will become invalid due to post-signing modification of the file. You're really in a catch-22 here.

It's like the typical scenario where a version control system is a life saver. Say you have 2 users and they both download a copy of FileA off their local intranet. UserA modifies line 3, saves it and uploads it. UserB also also modifies line 3, saves it and uploads it. Guess what, UserA just lost his changes. In a version control system environment, (either locking or merging), the changes would committed by UserA would never be lost unless consciously overwritten by UserB or somebody else.

In other words, one or more parties involved have to be aware of the others or they won't be able to get along on the playground.

If you need to sign an AutoIt file, then it's going to have to be wrapped in an archive. Modifying how we do things just to accommodate a single thing is a massive undertaking. If Jon can find a way so that things can play nice together, then that's great, but I'm not going to hold my breath on that because what we have are two applications that both want to modify the PE format of a file.

[jfrandol]

>>Who said anything about compression?

I assumed that is why upx is involved in this process.. perhaps there is some other reason ?

>> you can add virtually anything you want to a binary without breaking it.

exactly... like a digital signature...

I'll check out the beta..

[Valik]

>>Who said anything about compression?

I assumed that is why upx is involved in this process.. perhaps there is some other reason ?

Yes, UPX is for compression. But taking UPX out won't help. Right now, signing a compressed executable will break UPX's sections (If it will sign at all). Otherwise, without UPX, you're going to break AutoIt's.

>> you can add virtually anything you want to a binary without breaking it.

exactly...  like a digital signature...

I'll check out the beta..

Yes, but the point you're not absorbing is we have already modified it once. We have already made our changes to the file's structure. The signature tool knows nothing of our changes so it comes in and does it's thing. Now the binary is "tainted" to AutoIt because it was modified post-compilation and now AutoIt can no longer find the embedded script. You can only add one thing to the file unless all programs involved understand and acknowledge the existence of one another and can modifying the resulting file accordingly so as not to break the other.

[Valik]

This is not a bug. I've just moved your "bug report" on the topic to the NO BUG forum. It is not a bug; AutoIt is working exactly how it's supposed to. You do not fully understand how everything works and you can't label it as a bug because you lack necessary information. The behavior is undesirable to you, but it is not a bug. Please stop creating new threads on it and also please refrain from calling it a bug. It is not a bug.

[jfrandol]

This is not a bug.  It is not a bug.

<{POST_SNAPBACK}>

I think you should let Jon read it and decide.. instead of "filing" it. he seemed ammeneble to the subject (I know he is busy)

I guess I think I do understand - and we disagree - I think it is a 'bug' if the autoit compiler creates something that purports to be a windows executable that is not signable.

But as you seem irritated about the subject, and the source code is not available for me to understand or clarify. I will not discuss it further.

Thanks for your time, I do appreciate the responses.

John F.

[Raindancer]

I think it is a 'bug' if the autoit compiler creates something that purports to be a windows executable that is not signable.

<{POST_SNAPBACK}>

The AutoIt Exe isn't a standard EXE. It is the AutoIt Engine with your script inside (bytecode?) executed at runtime... just pretends to be a standalone compiled program... Therefore I understand why it isn't just 'easy to implement'...

But I also think that it would be nice to be able to sign your AutoIt-Pseudo-EXEs.

I always get a message that the issuer of this program isn't known when I start one of my compiled scripts at work from network drive. Due to our paranoid IT we can't tell windows to just accept this software. but if it were signed I think there should be a way for my IT to allow my Scripts to run from Network too, without this annoying message (it is special annoying with tools i run at startup).

[Valik]

I never said that Jon wouldn't or couldn't fix the issue. However, would you call it a bug that the Window's operating system does not support ELF format binaries? Just because something doesn't currently work or because something doesn't behave like you want to in software does not automatically mean it is a bug.

My irritation stems from the fact that you have created 3 separate threads about this in 3 separate forums. It also stems from the fact that you insist on telling us how things should be when you neither fully understand the PE format nor understand anything about AutoIt's architecture. You have the two most knowledgeable (or at least openly knowledgeable, I won't discount any closet experts) people of the PE format on the development team telling you things but you're insisting that we have a bug on our hands and it must be fixed and we're trying to explain to you without giving away the farm why things don't work. There are reasons why the source to AutoItSC.bin isn't available and there are reasons why signing an executable don't work. Can you give us the benefit of the doubt that perhaps Jon knows what he is doing and I know what I'm talking about stop making threads and calling something a bug when it's not?

If Jon is able to make AutoIt play nice with this particular digital signature tool, that's great... for you. But how do w know it will work with anything else that may modify an executable? Will it work with other wrapper programs that have been known to fail in the past with the same problem you encounter? It would be cool if AutoIt binaries could be used with those tools but I hope that Jon feels the same way that I do and that if a compromise can't be made to where AutoIt can work with them, then AutoIt's architecture should take precedence.

[jfrandol]

Sorry, I apologize for ever calling it a bug, you are correct, it it just a weakness, or just a small inconvenience for me in the way I wish to use it to develop .exe programs. If i wish to sign them, I will either recompress them with a different tool, or use a different program to develop my .exe

I never intended to do anything but offer any help that I could to help improve the output of the really great tool AutoIt - I would never have suggested anything If I didn't think it was a great tool that has no equal in the particular arena it serves.

I think it is interesting that AutoIt generated .exe's are the first ones I have ever encountered that are not digitally signable, so I percieved that as a weakness in their wrapper-construction. But their current architecture certainly gives them many strengths and I would choose those first , If one had to choose (as you say).

My experience with .exes has been mostly with ones developed with Microsoft tools.

I will minimize my threads in the future- I didn't think 3 were excessive, 1 in the support area, then moved to the development area as it got more technical, then posted a 'bug' in case any of the real development team wished to understand my experience.

But I am new here - I will lurk a while - I didn't mean to barge in too strongly, understand that this communication method sometimes causes conflicts where there really are none - I know I agree with many of your posts about keeping AutoIT pure and architectually clean (I was glad you made the comment about not having multiple line continuation characters for different purposes, and that it should be fixed 'correctly' or not at all - that is the way it should be for any changes that would possibly come out of this discussion also - and I can see why you would choose to make no changes at all for this item.

Thanks again for your time.

John F.

[jfrandol]

One last note - If anyone still cares - you were right about re-compressing the auto-it .exe with another tool - that is a great work around that solves my "issue".

I used the iexpress.exe tool available from microsoft to re-compress my autoit created executable before signing it, this allowed it to be signed and still be executed with one click of a small .exe.

To use IExpress from windows XP, open the Start->Run box and type, "iexpress.exe".

perhaps I will look into adding that option to the autoit compiler wrapper utility. (the one that adds a resource fork and then runs a post-utility.. I think )