Encoding problem

[gimx]

Hello Autoiters

I have a problem with sqlite encoding.

I extract text from an UTF8 webpage and I insert it. But the text is wrong in SQLite. For example the word "telé" give this word in SQLite "telé". I've tried to convert my script into UTF8 but no effect...

Any idea please ?

Thanks

PS : I have last stable version (no beta).

Edited January 20, 2010 by gimx

[BrettF]

Make sure the database is UTF8?

http://www.sqlite.org/pragma.html

Cheers,

Brett

[jchd]

I extract text from an UTF8 webpage and I insert it. But the text is wrong in SQLite. For example the word "telé" give this word in SQLite "telé". I've tried to convert my script into UTF8 but no effect...

What tool are you using to look in SQLite when you say that 'telé' has been stored as "telé".

Also what steps did you go thru to convert your script into UTF-8?

Hint: use a proven SQLite manager to check. A very nice one is SQLite Expert.

There are some problems with SQLite and UTF-8 data: be aware that the SQLite3.exe command-line tool is a DOS console program and as such can only display/enter text using the User ANSI codepage. Non 7-bit Unicode data will display incorrectly and if you enter non 7-bitcharacters with it, they will be stored with the incorrect encoding.

Also be aware that the Scite console will (by default) only display ANSI as well.

Using SQLite and AutoIt extensively for international business application, I can say that this is a working combination.

That said, if you need to do serious work with SQLite and really use non-ASCII characters, then you will need an SQLite extension for scalar and collation functions. You can use ICU but it's huge and slow. I've developped an alternative extension which I intend to post anytime soon in he Example code forum. In the meantime and if you feel it can be of interest to you feel free to ask.

I also have another extension able to determine if a given column contains valid UTF text. This extension also offers some encoding conversions.

[gimx]

Hello,

Thanks for reply.

I've converted my script with Notepad++.

I've downloaded SQLite Expert and datas display correctly... So I go to finish my script, export datas in Excel and see if it's fine. If I have a problem, I come back

Thank you