Detect Unicode

[willichan]

Does anyone know a way to detect the encoding of a string, whether ASCII/ANSI/UTF-8/UTF-16/UTF-32?

I would like to do something like FileGetEncoding() but with a string value instead of a file.

[abberration]

I haven't tested, but this thread looks like you can easily modify the code to just return the encoding:

'?do=embed' frameborder='0' data-embedContent>>

[jchd]

Pointing to a 7-year old thread clearly claimed pointless isn't going to help, rather add to confusion.

willichan,

All AutoIt strings are using UCS-2 encoding, which is the most useful subset of UTF16 also known as BMP-0.

[Myicq]

You probably need something to compare it to which is "meaningfull".  Any random array of bytes can be represented as single or double bytes, and same array of bytes can be represented in many ways depending on ansi encoding you choose. Example:

ÿÿ in Western ansi is same as яя in Cyrillic ansi. But same bytes. And яя in UTF8 will be meaningless represented in ansi - but is still legal ansi.

[mLipok]

look for @Lazycat UDFs

http://autoit.darkhost.ru/udfs.html

Edited November 11, 2023 by mLipok

[willichan]

All AutoIt strings are using UCS-2 encoding, which is the most useful subset of UTF16 also known as BMP-0.

I guess that explains why so many of my scripts seem to be failing for my Japanese users.  Is there any way around this?

look for @LazyCat UDFs

That is the FileGetEncoding function I was referring to in the OP.  Given what jchd said, I think I know now why I was not able to re-tool it to work with user entered data.

--- Edit ---

@jchd:  Never mind.  I feel silly.  I needed to be using AscW and ChrW, and don't need to worry about the encoding.  I just need to account for the unicode alpha-numerics (like full width vs. half width alpha-numerics used in Japanese), and that is a simple one.

Edited November 22, 2013 by willichan

[jchd]

Good that you got it working. I don't believe that the restriction from UTF16 to UCS-2 would be an issue for everyday's Japanese.