Jump to content
Sign in to follow this  
gimx

Encoding problem

Recommended Posts

gimx

Hello Autoiters :D

I have a problem with sqlite encoding.

I extract text from an UTF8 webpage and I insert it. But the text is wrong in SQLite. For example the word "telé" give this word in SQLite "telé". I've tried to convert my script into UTF8 but no effect...

Any idea please ?

Thanks :huggles:

PS : I have last stable version (no beta).

Edited by gimx

Share this post


Link to post
Share on other sites
BrettF
jchd

I extract text from an UTF8 webpage and I insert it. But the text is wrong in SQLite. For example the word "telé" give this word in SQLite "telé". I've tried to convert my script into UTF8 but no effect...

What tool are you using to look in SQLite when you say that 'telé' has been stored as "telé".

Also what steps did you go thru to convert your script into UTF-8?

Hint: use a proven SQLite manager to check. A very nice one is SQLite Expert.

There are some problems with SQLite and UTF-8 data: be aware that the SQLite3.exe command-line tool is a DOS console program and as such can only display/enter text using the User ANSI codepage. Non 7-bit Unicode data will display incorrectly and if you enter non 7-bitcharacters with it, they will be stored with the incorrect encoding.

Also be aware that the Scite console will (by default) only display ANSI as well.

Using SQLite and AutoIt extensively for international business application, I can say that this is a working combination.

That said, if you need to do serious work with SQLite and really use non-ASCII characters, then you will need an SQLite extension for scalar and collation functions. You can use ICU but it's huge and slow. I've developped an alternative extension which I intend to post anytime soon in he Example code forum. In the meantime and if you feel it can be of interest to you feel free to ask.

I also have another extension able to determine if a given column contains valid UTF text. This extension also offers some encoding conversions.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites
gimx

Hello,

Thanks for reply.

I've converted my script with Notepad++.

I've downloaded SQLite Expert and datas display correctly... So I go to finish my script, export datas in Excel and see if it's fine. If I have a problem, I come back :D

Thank you :huggles:

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×