Advanced Development - Unicode and 64bit

[Jon]

I'm aware that myself and many other devs are on record saying that this post can't exist...so pretend like it never happened.

We currently have alpha test builds for both a Unicode and x64 version of AutoIt3.exe

Big thanks to Valik for all his work on getting the 64bit up to its current state.

Note: These builds are cutting edge, omg might blow up your machine versions. They are purely for those people who want to help us test the more obscure bugs that may not have been caught with the normal version of AutoIt.

Also, please don't ask when they will be complete - they are very much concept atm. But we would appreciate those of you who needed them to give them a test and report any bugs. Bugs in this sort of thing tend to be subtle and hard for us to find!

Unicode

Now used in production version

Oddities/Issues

- ControlSend/Send() converts text to ANSI. (Will look at improving after initial bugs fixed).

- Oh, and of course won't run on non unicode OSes like 9x.

64 bit / x64

Current alpha: http://www.autoitscript.com/autoit3/files/beta/autoit/x64

Oddities/Issues

Is currently compiled as Unicode so has all the previously mentioned issues, plus:

- DllCall / DllStruct not implemented.

[Jon]

And for those wondering how long it took.

Valik's been working with the 64 bit stuff on and off since December and I just spent every waking hour for 4 days straight (aren't holidays grand) doing the Unicode edits. Almost every line of the AutoIt sources has been checked or changed (2 million characters) in that time. You know there are going to be bugs after that...

A hobby??!!

[Sunaj]

It has been such a good experience developing an application with AutoIt - every time everything seems to work just perfectly.. you and the developer team goes and makes the core running the whole show even better thereby making my code perfect += 1! The need for using Chinese chars in my app has just emerged and unicode support happens just two days later.. incredible! Only thing is that I can't really begin testing the unicode build before there's a workaround for the Send() function.. Can only hope it proves doable to find a solution for that particular issue

Cheers for doing a really good job on developing the core.

Sunaj

EDIT: splelling fixed

I'm aware that myself and many other devs are on record saying that this post can't exist...so pretend like it never happened.

We currently have alpha test builds for both a Unicode and x64 version of AutoIt3.exe

Big thanks to Valik for all his work on getting the 64bit up to its current state.

Note: These builds are cutting edge, omg might blow up your machine versions. They are purely for those people who want to help us test the more obscure bugs that may not have been caught with the normal version of AutoIt.

Also, please don't ask when they will be complete - they are very much concept atm. But we would appreciate those of you who needed them to give them a test and report any bugs. Bugs in this sort of thing tend to be subtle and hard for us to find!

(...)

Edited April 4, 2007 by Sunaj

[Jon]

Only thing is that I can't really begin testing the unicode build before there's a workaround for the Send() function..

That part is not likely to happen. The send function relies on the API telling it which keystroke to use. All the API functions in that area are ANSI based and I can't see how it would be able to work out the keystrokes required for chinese letters (multiple keystrokes per character isn't it?).

The best solution for this (and for most other situations) is to try and use ControlSetText which can happily use unicode chars and is more reliable than Send() (Send() is a "last resort" function).

[Sunaj]

Haven't looked into the ControlSetText function - will do so and if I can make it work I'll return with feedback (will take -time- as translation/test of Chinese chars will have to be done by a third party). Thanks for your suggestion!

That part is not likely to happen. The send function relies on the API telling it which keystroke to use. All the API functions in that area are ANSI based and I can't see how it would be able to work out the keystrokes required for chinese letters (multiple keystrokes per character isn't it?).

The best solution for this (and for most other situations) is to try and use ControlSetText which can happily use unicode chars and is more reliable than Send() (Send() is a "last resort" function).

[Jon]

Updated the Unicode test executables with a weekend of bug fixes.

[NELyon]

OO Unicode version. Much needed

[LOULOU]

Is anyone can explain me the advantage of the unicode version of autoit instead of present version ? Thanks for the anwser

[Jon]

- Works with all language characters rather than just the 256 ANSI chars (chinese, russian, etc.)

- Windows APIs are natively Unicode - so when you run a non-unicode program it has to convert all strings to and from unicode on the fly (slowdown, although not a massive difference in AutoIt).

[Jon]

From a post I made on the private dev forum:

I've got all the main tools (AutoIt, Aut2Exe, Au3Info) compiled and working in Unicode mode - the size difference (~15KB) is tiny so that's good (and performance is the same/better).

In the past 13 days, page requests to the AutoIt web site by OS:

NT based: 5616143 (97.3%)

9x based: 151679 (2.7%)

The Win 9x audience is microscopic (we knew that) - we only really support them for legacy scripting situations (like when I have to do a script for a crazy customer )

What I propose:

- We use the Unicode versions as the main live versions. AutoIt3.exe, Au3Info.exe, Aut2Exe.exe

- For Win9x we include just an ANSI version of AutoIt3.exe (Call it AutoIt3a.exe like MS tools do sometimes). No other ANSI versions are supplied (no compiler, decompiler, au3info). Purely exists to allow a plain text script to run on 9x. The official line is that we support 9x just for the basic interpreter.

[MHz]

The official line is that we support 9x just for the basic interpreter.

IMO, sounds reasonable. At least 95% of my scripts execute uncompiled. Only if I intend to share scripts is when I will compile to make it standalone. The best kept secret is to not compile if you already have the interpreter as it has so much to offer as in edit easily, no compile actions needed... When it comes to security of an not showing the source, then compile and distribute. 9x users will need to upgrade if wanting to develop and share their creation to public but still have the opportunity to run a script freely on their PC.

I support 9x, but a time of legacy support will slip away at some point.

[Zedna]

I'm using WIN98SE still on some machines.

I agree with Jon with his concept only with request to have also compiler (Aut2Exe) also in ANSI version.

I compile into EXE all my scripts that I'm using.

But if developers decide to not make ANSI Aut2Exe it can be done easy workaround:

- make BAT file for every script which call autoit3.exe myscript.au3.

- or make default "Run" as action for au3 extension

EDIT: disadvantage: with this workaround can't be done:

- FileInstall

- icons in resources

Just my opinion.

EDIT2:

Also another situation for usefullness of ANSI compiler:

When I will make some AutoIt application and want to place it somewhere on the internet for wide download,

then for users with WIN9x there could be EXE version compiled with this ANSI compiler.

I'm not using AutoIt as scripting language only but also like programing language.

Edited April 14, 2007 by Zedna

[Jon]

Unicode version now supplied as the default in the latest beta.

[Jon]

This table might be useful for showing what file formats you can use with AutoIt and in which editor (the notation varies between editors):

AutoIt: ANSI

Notepad: ANSI

SciTe: 8 bit / Code Page Property

AutoIt: UTF16 Little Endian

Notepad: Unicode

SciTe: UCS-2 Little Endian

AutoIt: UTF16 Big Endian

Notepad: Unicode big endian

SciTe: UCS-2 Big Endian

AutoIt: UTF8

Notepad: UTF-8

SciTe: UTF-8 with BOM

[Jon]

Latest version of AutoIt 64bit is available at the beta url. Should work exactly the same as normal AutoIt - just without DLL related functions. The dll stuff requires assembly and the 64bit compiler no longer has inline assembly so it's a boggling problem.

[Jon]

This table might be useful for showing what file formats you can use with AutoIt and in which editor (the notation varies between editors):

AutoIt: ANSI

Notepad: ANSI

SciTe: 8 bit / Code Page Property

AutoIt: UTF16 Little Endian

Notepad: Unicode

SciTe: UCS-2 Little Endian

AutoIt: UTF16 Big Endian

Notepad: Unicode big endian

SciTe: UCS-2 Big Endian

AutoIt: UTF8

Notepad: UTF-8

SciTe: UTF-8 with BOM

Included this table in the documentation and at http://www.autoitscript.com/autoit3/docs/intro/unicode.htm

[WeMartiansAreFriendly]

Included this table in the documentation and at http://www.autoitscript.com/autoit3/docs/intro/unicode.htm

Do we need to compile for ansi, and unicode seperately for ansi win9x user support, and winnt unicode support??

[Jos]

Do we need to compile for ansi, and unicode seperately for ansi win9x user support, and winnt unicode support??

its a bit confusing question, but you either compile with Ansi or Unicode so assume to answer to your question is yes.

An Ansi compiled script will work on ALL platforms but lacks Unicode support

An Unicode compiled script will NOT work on Win9x/Winme.

[Jon]

Added the @Unicode macro to the latest release. Usually this won't be needed as AutoIt takes care of ANSI/Unicode differences. But certain DLLCalls() or SendMessages or GUICtrlSendMsg calls may use messages that need to use Unicode or ANSI strings. This macro should help workaround such differences (or allow us/users to write Include files that define the correct messages depended on the environment).

[randallc]

Hi,

I thought this might have helped convert the code sent by "GUICtrlSendMsg calls"; but I still don't know how to convert that message for Date control GUI combi "style"; no solutions have helped; perhaps it is unrelated to the unicode change?

BVest, randall

Edited May 20, 2007 by randallc