How do I Unicode

[MaxPlankpar]

Moving forward, what is there to expect from the unicode ansi direction?  If there are issues that have been decided upon it would be helpful to have clarity.  I had been using the older version to clobber UTF-8 for error handling but am wondering what direction might be taken, as there are still issues, I expect (perhaps), to be addressed.  If this has been addressed already, please bite my head off, thank you...

<I>Yes/4/1 No/4/2 Vote contest winner A or B (none of the above)</I><!-wong fomant->

[BrewManNH]

You aren't making any sense, what exactly is it that you're asking?

[MaxPlankpar]

The purpose was generality.  I was wondering if it will be on the front burner again or if an idea was forming in the consensus about how certain "transcriptions" might be handled.  I can see there are still issues but  i'm not a "real" coder & don't know the internals of AutoIt.

Give a programmer the correct code and he might still suck as a programmer.  he can do his work for a lifetime & still be busy with other time consuming problems - Me

Edited October 8, 2016 by MaxPlankpar
forgot the quote

[BrewManNH]

I still don't know what it is you're asking. 

38 minutes ago, MaxPlankpar said:

I was wondering if it will be on the front burner again

Explain what IT is first. 

I get that it's something to do with UTF-8 but beyond that I have no idea what you're asking about it.

Be detailed in your explanation, and perhaps someone with some insight can tell you something about whatever it might be.

[TheDcoder]

Here is a Tip: AutoIt handles the encoding for you! Everything is stored in UTF-16 internally...

[MaxPlankpar]

    I was curious about how different OS versions (language wise) handle native language chars.  Mostly, I wish there where a switch to flatten the "IT" UTF-16 to ANSI.  I've put a slight mud hole in that argument already as I'm sure you know.  I reiterate.

   It would come in handy.  The native text changes a lot/very much across platforms & OS versions.

Edited October 10, 2016 by MaxPlankpar
there was/wish there is

[jchd]

???

Flatten Unicode to ANSI? Are you serious about encoding the whole Unicode charset (128172 distinct possible characters) into a single 256-character ANSI charset?

Things are working the other way: Unicode is the only possible character set able to represent portably all characters and symbols ever and forever used by humans. So any OS/application Unicode-aware can actually represent portably all of cyrillic, latin, greek, thai, chinese, ..., you_name_it codepoints into the same string, something ANSI is obviously unable to do.

[MaxPlankpar]

17 hours ago, jchd said:

???Flatten Unicode to ANSI?  serious (128172 distinct possible characters) into a single 256-character ANSI charset

???? ya!  I could use some of that action.  That is how it was before.  I would love to have a switch for that ????.

Edited October 11, 2016 by MaxPlankpar

[jchd]

I don't get it.

Please explain a real-world situation where the problem you have occurs, if any.

[MaxPlankpar]

How do I Unicode?
This is how I do it now:

openwith $somelanguagechars

Let's say I want to use wo3 or wŏ or &#0335; or 我
If it is flat I can use it in AutoIT Or dos Or Linux Or Python Or Android Or Stupid-Font-Error Or etc.,...
If it is flat it is my Standard-Native everywhere.  

Say I have _wo__A through  _wo__Z with language definitions in _cn _en

If I have _de, _fr with something like ex: $string="Lernt sie wo3&#25105_cn__A de5_&#30340_cn__B &#0233_galer_fr__A"
$string="Lernt sie 我的 égaler"

then "it" does what I need. I can run this various ways with filters without confusion.

Webkit is cool.  What if I want wo3_cn.html for some reason?

If I have a switch to flatten it, now my dictionary works across platform properly & I have notification of incident without undo error.

Without a switch I use  the old autoit version until I figure a better way.

How do I do it better? If you have some answers please share.

https://www.autoitscript.com/forum/topic/184936-unicode-utf-bom-thing/

[JLogan3o13]

Why, exactly, are you starting an entirely new thread on what is basically the same topic as the one you linked to?

[MaxPlankpar]

I didn't ask for a new way to address this problem before.  I was hopping to sway sentiment.  I must now seriously consider that it looks like it is frozen.  I guess I need help on across platform issues.  My whole system is in need of a rewrite if I upgrade.

Please, I ask the masses, to pontificate on your JAVA's & jerryriggers!

I have but only half an arse, please help to make me whole!

Edited October 12, 2016 by MaxPlankpar

[JLogan3o13]

I don't see this as anything but a continuation of the conversation. As such, as as you're asking for cross-platform opinions rather than specific issues for AutoIt code, it also doesn't belong in General Help & Support. Threads merged and moved.

Edited October 12, 2016 by JLogan3o13

[jchd]

@MaxPlankpar,

Your posts don't make any sense. You throw in undefined terms (e.g. "flat"), you use sentences that make no sense to anyone except you (e.g. " Say I have _wo__A through  _wo__Z with language definitions in _cn _en ") and you mix completely distinct things in a row as if they were of the same nature (e.g. AutoIt, Python [languages], DOS, Linux & Androïd [OSes], Stupid-Font-Error [only you can tell what's that]). Finally you say that the "old AutoIt version" does "it" well without telling us which version you're talking about, nor the issue you're hitting with more recent versions.

After reading your prose multiple times (yeah, really) in a suicidal attempt to help you, you seem to expect something ("a switch" [???]) that would magically detect the language of words based on the characters they contain. That or append the language to each word known to your dictionary????

What are "de5", "wo3", " _cn__A", " _cn__B", " _fr__A "?

Can't you explain yourself clearly? Something along the line (guesswork here):

"I have a big dictionary with words from many languages. Given an input string with several words from different languages I need to lookup my dictionary to find them along with the language they belong to and transform the string using a proprietary encoding defined below. < definition of the encoding>.

The reason for using such encoding is <sensible rationale>.

Using version U.V.W of AutoIt I used to run this code: <running example code snippet>
but it doesn't work anymore using version X.Y.Z and all I get is the following error: <error description>

Can you help me solve the issue or guide me towards a more efficient way to achieve what I need?"

[MaxPlankpar]

_cn Chinese, _fr French, _en English, _de German https://en.wikipedia.org/wiki/Language_localisation Tags

我 wo3 I, me - Chinese Char & pinion

的 de5 = 's, possesive - Chinese word (whoops, I shouldn't have used this word, causes confusion)

cn__A, cn__B  - homonyms may have A-Z assignments

&#0233_galer_fr__A  - stands in, in my dict for  "égaler" a _fr french word

autoit version 3.3.0.0 versions have old BOM ansi storage that I called "flat".  The number of bits is minimal. Hence "?" is a searchable string.

autoit version 3.3.14.2 has no switch to force ANSI. It looks like it does but it is not so.  I cannot search stings like "?". They will be corrupt unicode.

Say for instance some Chinese character gets transposed to "ae", or " œ" needs to remain as is.  I can't just search for every "ae" in a reasonable time frame. Aside from all the other problems that don't make any sense go.

The ref to the Android(can't change font without root), OS diff problems, etc., mentioned as one thing:

Because these are issues that all cause corruption. I don't know how the android screws up fonts but it does. These issues arise repeatedly so I addressed them in one sentence. I have a finite amount of time, skill & patience to figure out why I cannot copy a clean piece of data. Is the memory degrading? Why is there no memory leak on the SD card? What is wrong with everything? Lock it down md5, somebody close the damn blast door. I don't know about all that. I just want it to work and ANSI was working before.

Thank you for being crazy on my behalf and the time it took for you to become that way.  I did not mean to drive you there.  It literarily was not what I envisaged.

P.S. I kind of like the way you clearly, concisely & succinctly summed up my problem.

Edited October 14, 2016 by MaxPlankpar

[JLogan3o13]

12 hours ago, MaxPlankpar said:

I have a finite amount of time, skill & patience

So do we, and thus far your posts have made you seem like you're babbling incoherently. If you cannot take the time to be clear and concise, no one is going to take the time to try to figure out what you're talking about.

[MaxPlankpar]

$FO_ANSI (512) = Use ANSI reading and writing mode.

This doesn't work plain & simple

Nope, I got nothin full stop

[jchd]

Can you once post code to show that?