MaxPlankpar

How do I Unicode

18 posts in this topic

#1 ·  Posted

Moving forward, what is there to expect from the unicode ansi direction?  If there are issues that have been decided upon it would be helpful to have clarity.  I had been using the older version to clobber UTF-8 for error handling but am wondering what direction might be taken, as there are still issues, I expect (perhaps), to be addressed.  If this has been addressed already, please bite my head off, thank you...

<I>Yes/4/1 No/4/2 Vote contest winner A or B (none of the above)</I><!-wong fomant->

Share this post


Link to post
Share on other sites



#2 ·  Posted

You aren't making any sense, what exactly is it that you're asking?


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

The purpose was generality.  I was wondering if it will be on the front burner again or if an idea was forming in the consensus about how certain "transcriptions" might be handled.  I can see there are still issues but  i'm not a "real" coder & don't know the internals of AutoIt.

Give a programmer the correct code and he might still suck as a programmer.  he can do his work for a lifetime & still be busy with other time consuming problems - Me

Edited by MaxPlankpar
forgot the quote

Share this post


Link to post
Share on other sites

#4 ·  Posted

I still don't know what it is you're asking. 

38 minutes ago, MaxPlankpar said:

I was wondering if it will be on the front burner again

Explain what IT is first. 

I get that it's something to do with UTF-8 but beyond that I have no idea what you're asking about it.

Be detailed in your explanation, and perhaps someone with some insight can tell you something about whatever it might be.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

#5 ·  Posted

Here is a Tip: AutoIt handles the encoding for you! Everything is stored in UTF-16 internally...


Spoiler

My contributions to the AutoIt Community

Some messages & Apologizes:

If I hurt you, Please accept my apologies, I never (regardless of the situation) mean to hurt anybody!!!

Also, I am very busy with my project so I will appear in the last row of the online list, if you want to contact me: Email@TheDcoder.xyz

Or you can have a nice chat with me in freenode, I use the same nick on freenode too!

3fHNZJ.gif

PLEASE JOIN ##AutoIt AND HELP THE IRC AUTOIT COMMUNITY!

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

    I was curious about how different OS versions (language wise) handle native language chars.  Mostly, I wish there where a switch to flatten the "IT" UTF-16 to ANSI.  I've put a slight mud hole in that argument already as I'm sure you know.  I reiterate.

   It would come in handy.  The native text changes a lot/very much across platforms & OS versions.

Edited by MaxPlankpar
there was/wish there is

Share this post


Link to post
Share on other sites

???

Flatten Unicode to ANSI? Are you serious about encoding the whole Unicode charset (128172 distinct possible characters) into a single 256-character ANSI charset?

Things are working the other way: Unicode is the only possible character set able to represent portably all characters and symbols ever and forever used by humans. So any OS/application Unicode-aware can actually represent portably all of cyrillic, latin, greek, thai, chinese, ..., you_name_it codepoints into the same string, something ANSI is obviously unable to do.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

#8 ·  Posted (edited)

17 hours ago, jchd said:

???Flatten Unicode to ANSI?  serious (128172 distinct possible characters) into a single 256-character ANSI charset

???? ya!  I could use some of that action.  That is how it was before.  I would love to have a switch for that ????.

Edited by MaxPlankpar

Share this post


Link to post
Share on other sites

I don't get it.

Please explain a real-world situation where the problem you have occurs, if any.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

#10 ·  Posted

How do I Unicode?
This is how I do it now:

openwith $somelanguagechars

Let's say I want to use wo3 or wŏ or &#0335; or 我
If it is flat I can use it in AutoIT Or dos Or Linux Or Python Or Android Or Stupid-Font-Error Or etc.,...
If it is flat it is my Standard-Native everywhere.  

Say I have _wo__A through  _wo__Z with language definitions in _cn _en

If I have _de, _fr with something like ex: $string="Lernt sie wo3&#25105_cn__A de5_&#30340_cn__B &#0233_galer_fr__A"
$string="Lernt sie 我的 égaler"

then "it" does what I need. I can run this various ways with filters without confusion.


Webkit is cool.  What if I want wo3_cn.html for some reason?

If I have a switch to flatten it, now my dictionary works across platform properly & I have notification of incident without undo error.

Without a switch I use  the old autoit version until I figure a better way.

How do I do it better? If you have some answers please share.

https://www.autoitscript.com/forum/topic/184936-unicode-utf-bom-thing/

Share this post


Link to post
Share on other sites

#11 ·  Posted

Why, exactly, are you starting an entirely new thread on what is basically the same topic as the one you linked to?


When you're dead, you don't know you're dead - it's only difficult for those that know you. It's the same way when you're stupid...

My Scripts: SCCM UDFInclude Source with Compiled Script, Windows Firewall UDF

Share this post


Link to post
Share on other sites

#12 ·  Posted (edited)

I didn't ask for a new way to address this problem before.  I was hopping to sway sentiment.  I must now seriously consider that it looks like it is frozen.  I guess I need help on across platform issues.  My whole system is in need of a rewrite if I upgrade.

Please, I ask the masses, to pontificate on your JAVA's & jerryriggers!

I have but only half an arse, please help to make me whole!

Edited by MaxPlankpar

Share this post


Link to post
Share on other sites

#13 ·  Posted (edited)

I don't see this as anything but a continuation of the conversation. As such, as as you're asking for cross-platform opinions rather than specific issues for AutoIt code, it also doesn't belong in General Help & Support. Threads merged and moved.

Edited by JLogan3o13

When you're dead, you don't know you're dead - it's only difficult for those that know you. It's the same way when you're stupid...

My Scripts: SCCM UDFInclude Source with Compiled Script, Windows Firewall UDF

Share this post


Link to post
Share on other sites

#14 ·  Posted

@MaxPlankpar,

Your posts don't make any sense. You throw in undefined terms (e.g. "flat"), you use sentences that make no sense to anyone except you (e.g. " Say I have _wo__A through  _wo__Z with language definitions in _cn _en ") and you mix completely distinct things in a row as if they were of the same nature (e.g. AutoIt, Python [languages], DOS, Linux & Androïd [OSes], Stupid-Font-Error [only you can tell what's that]). Finally you say that the "old AutoIt version" does "it" well without telling us which version you're talking about, nor the issue you're hitting with more recent versions.

After reading your prose multiple times (yeah, really) in a suicidal attempt to help you, you seem to expect something ("a switch" [???]) that would magically detect the language of words based on the characters they contain. That or append the language to each word known to your dictionary????

What are "de5", "wo3", " _cn__A", " _cn__B", " _fr__A "?

Can't you explain yourself clearly? Something along the line (guesswork here):

"I have a big dictionary with words from many languages. Given an input string with several words from different languages I need to lookup my dictionary to find them along with the language they belong to and transform the string using a proprietary encoding defined below. < definition of the encoding>.

The reason for using such encoding is <sensible rationale>.

Using version U.V.W of AutoIt I used to run this code: <running example code snippet>
but it doesn't work anymore using version X.Y.Z and all I get is the following error: <error description>

Can you help me solve the issue or guide me towards a more efficient way to achieve what I need?"

1 person likes this

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

#15 ·  Posted (edited)

_cn Chinese, _fr French, _en English, _de German https://en.wikipedia.org/wiki/Language_localisation Tags

我 wo3 I, me - Chinese Char & pinion

 de5 = 's, possesive - Chinese word (whoops, I shouldn't have used this word, causes confusion)

cn__A, cn__B  - homonyms may have A-Z assignments

&#0233_galer_fr__A  - stands in, in my dict for  "égaler" a _fr french word

autoit version 3.3.0.0 versions have old BOM ansi storage that I called "flat".  The number of bits is minimal. Hence "?" is a searchable string.

autoit version 3.3.14.2 has no switch to force ANSI. It looks like it does but it is not so.  I cannot search stings like "?". They will be corrupt unicode.

Say for instance some Chinese character gets transposed to "ae", or " œ" needs to remain as is.  I can't just search for every "ae" in a reasonable time frame. Aside from all the other problems that don't make any sense go.

The ref to the Android(can't change font without root), OS diff problems, etc., mentioned as one thing:

Because these are issues that all cause corruption. I don't know how the android screws up fonts but it does. These issues arise repeatedly so I addressed them in one sentence. I have a finite amount of time, skill & patience to figure out why I cannot copy a clean piece of data. Is the memory degrading? Why is there no memory leak on the SD card? What is wrong with everything? Lock it down md5, somebody close the damn blast door. I don't know about all that. I just want it to work and ANSI was working before.

Thank you for being crazy on my behalf and the time it took for you to become that way.  I did not mean to drive you there.  It literarily was not what I envisaged.

P.S. I kind of like the way you clearly, concisely & succinctly summed up my problem.

Edited by MaxPlankpar

Share this post


Link to post
Share on other sites

#16 ·  Posted

12 hours ago, MaxPlankpar said:

I have a finite amount of time, skill & patience

So do we, and thus far your posts have made you seem like you're babbling incoherently. If you cannot take the time to be clear and concise, no one is going to take the time to try to figure out what you're talking about.


When you're dead, you don't know you're dead - it's only difficult for those that know you. It's the same way when you're stupid...

My Scripts: SCCM UDFInclude Source with Compiled Script, Windows Firewall UDF

Share this post


Link to post
Share on other sites

#17 ·  Posted

$FO_ANSI (512) = Use ANSI reading and writing mode.

This doesn't work plain & simple

Nope, I got nothin full stop

Share this post


Link to post
Share on other sites

#18 ·  Posted

Can you once post code to show that?


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now