Sign in to follow this  
Followers 0
Fossil Rock

UNICODE TO ANSI conversion

19 posts in this topic

#1 ·  Posted (edited)

I have a DOS based program that outputs a txt file using UNICODE encoding. When I try to read a line of text in is comes back as gibberish. If I save the file using ANSI encoding I can read it fine. I can not do anything about the output file while the program is running so I was wondering if anyone new a way to convert the Unicode to Ansi when I read it into my script. Or possibly read the Unicode but it not show as jibberish.

Thanks

Edited by Fossil Rock

Agreement is not necessary - thinking for one's self is!

My-Colors.jpg

cuniform2.gif

Share this post


Link to post
Share on other sites



google for stingconverter. If you know the file name just pump it into this program and give it an output name and then read the output file. Maybe not optimal but it works.

Share this post


Link to post
Share on other sites

google for stingconverter. If you know the file name just pump it into this program and give it an output name and then read the output file. Maybe not optimal but it works.

Already thought of that, thanks. But the optimal part is what I'm after. I can convert the file just by using the 'TYPE' command i.e

TYPE Program.log > Program.txt

but I need to constantly be reading the file so this would probably add alot of cpu time or memory consumption.

Another idea would be to strip the encoding, but I don't know how to do that.


Agreement is not necessary - thinking for one's self is!

My-Colors.jpg

cuniform2.gif

Share this post


Link to post
Share on other sites

I had to do this too, I needed to read in the file and also save it.

What I did was used notepad in a hidden window and used ControlRead to read the text from notepad's hidden window. Once I had the text I closed the hidden window.

I did te reverse when I wanted to save it.

Share this post


Link to post
Share on other sites

I had to do this too, I needed to read in the file and also save it.

What I did was used notepad in a hidden window and used ControlRead to read the text from notepad's hidden window. Once I had the text I closed the hidden window.

I did te reverse when I wanted to save it.

What kind of performance hit did you take? Do you read/convert a lot of lines?


Agreement is not necessary - thinking for one's self is!

My-Colors.jpg

cuniform2.gif

Share this post


Link to post
Share on other sites

What kind of performance hit did you take? Do you read/convert a lot of lines?

This seems to be the only way to do this. Do you have any sample code. I need to know how to open notepad with a specific file.

Thanks


Agreement is not necessary - thinking for one's self is!

My-Colors.jpg

cuniform2.gif

Share this post


Link to post
Share on other sites

This seems to be the only way to do this. Do you have any sample code. I need to know how to open notepad with a specific file.

Thanks

nopade.exe "somefilename.txt"

I think that is how it goes.

JS


AutoIt Links

File-String Hash Plugin Updated! 04-02-2008 Plugins have been discontinued. I just found out.

ComputerGetInfo UDF's Updated! 11-23-2006

External Links

Vortex Revolutions Engineer / Inventor (Web, Desktop, and Mobile Applications, Hardware Gizmos, Consulting, and more)

Share this post


Link to post
Share on other sites

#8 ·  Posted (edited)

nopade.exe "somefilename.txt"

I think that is how it goes.

JS

Thanks, after experimenting, the actual command is Run("Notepad.exe somefilename.txt") - path optional to needs

I appreciate the help.

Edited by Fossil Rock

Agreement is not necessary - thinking for one's self is!

My-Colors.jpg

cuniform2.gif

Share this post


Link to post
Share on other sites

Thanks, after experimenting, the actual command is Run("Notepad.exe somefilename.txt") - path optional to needs

I appreciate the help.

My apologies. I thought you realised I meant for use with the Run() command.

JS


AutoIt Links

File-String Hash Plugin Updated! 04-02-2008 Plugins have been discontinued. I just found out.

ComputerGetInfo UDF's Updated! 11-23-2006

External Links

Vortex Revolutions Engineer / Inventor (Web, Desktop, and Mobile Applications, Hardware Gizmos, Consulting, and more)

Share this post


Link to post
Share on other sites

#10 ·  Posted (edited)

Hello,

this little CMD File can convert all files you want. The sample is for registry files :P

@Echo Off

echo UniCode to ANSI converter for Registry files.

md ~ANSI

for %%a in (*.reg *.?reg?) do type "%%a">"~ANSI\%%a"

for %%a in (*.reg *.?reg?) do attrib -r -h -s "%%a"

cd ~ANSI

for %%a in (*.reg *.?reg?) do move /y "%%a" ..

cd..

rd ~ANSI

:end of script

Copy it to yout temp directory an run it from a dos box

Edited by Micha1405

Share this post


Link to post
Share on other sites

Hello,

this little CMD File can convert all files you want. The sample is for registry files :P

@Echo Off

echo UniCode to ANSI converter for Registry files.

md ~ANSI

for %%a in (*.reg *.?reg?) do type "%%a">"~ANSI\%%a"

for %%a in (*.reg *.?reg?) do attrib -r -h -s "%%a"

cd ~ANSI

for %%a in (*.reg *.?reg?) do move /y "%%a" ..

cd..

rd ~ANSI

:end of script

Copy it to yout temp directory an run it from a dos box

Interesting.. so can you convert them back to Unicode in the same way?? if so how?

Share this post


Link to post
Share on other sites

Hello,

this little CMD File can convert all files you want. The sample is for registry files :P

@Echo Off

echo UniCode to ANSI converter for Registry files.

md ~ANSI

for %%a in (*.reg *.?reg?) do type "%%a">"~ANSI\%%a"

for %%a in (*.reg *.?reg?) do attrib -r -h -s "%%a"

cd ~ANSI

for %%a in (*.reg *.?reg?) do move /y "%%a" ..

cd..

rd ~ANSI

:end of script

Copy it to yout temp directory an run it from a dos box

What I'm trying to do is read a txt file that is saved in unicode. Converting the file is not difficult, reading the original file is the problem.

When I read the first line of 'Example.log' which contains "Sample text." it returns ÿþS.

Posted Image

Doing the same thing with 'Example.txt' saved as Ansi produces "Sample text." as it should.

Posted Image

I really hope I'm making sense. ( Just for the record the extensions .log and .txt do not effect the encoding)


Agreement is not necessary - thinking for one's self is!

My-Colors.jpg

cuniform2.gif

Share this post


Link to post
Share on other sites

OK, after a few hours of pulling my hair out (and I ain't got much) I came up with this little trick to strip the Unicode out. There are only 3 characters that I wanted removed... Null , ÿ and þ

In the Unicode file ÿþ are the first 2 characters of the file and every second character after that is a NULL character.

You need to make 2 files first.... C:\Example.log Saved with UNICODE and C:\Example.txt Saved with ANSI. Fill 'Example.log' with any text you want and leave the 'Example.txt' empty.

I ran this on a 500K file and it only took a minute. If anyone can make it faster I would appreciate the info.

Ultimately I would like to read an entire line in and strip the codes and then write the entire line.... I'll keep working on that.

Enjoy

; Open Files
$file = FileOpen("C:\Example.log", 0)  ; Open for Reading
$file1 = FileOpen("C:\Example.txt", 1) ; Open for Writing

; Check if file is opened for reading
If $file = -1 Then
    MsgBox(0, "Error", "Unable to open file.")
    Exit
EndIf

; Check if file is opened for writing
If $file1 = -1 Then
    MsgBox(0, "Error", "Unable to open file.")
    Exit
EndIf

; Read in 2 characters at a time until the EOF is reached
While 1
    $chars = FileRead($file, 2)
    If @error = -1 Then ExitLoop
        If ($chars) <> "ÿþ" then
            FileWrite ($file1 , StringLeft ($chars , 1))
 
EndIf

Wend

FileClose($file)
FileClose($file1)

Agreement is not necessary - thinking for one's self is!

My-Colors.jpg

cuniform2.gif

Share this post


Link to post
Share on other sites

#14 ·  Posted (edited)

Do you have a sample text to play around with?

Edit:

Nevermind, guess it would help not reading these post so late!!

Edited by SmOke_N

[center]Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.[/center]

Share this post


Link to post
Share on other sites

Well this option doesn't look so good if you are using it other than personal use, but it does take about 2 seconds to complete.

$Timer = TimerInit()
Opt('SendKeyDelay', 0)
Opt('WinTextMatchMode', 2)
Run('Notepad c:\Example.Log')
WinWaitActive('Example.Log - Notepad')
ControlSend('Example.Log - Notepad', '', 'Edit1', Chr(94)&Chr(97))
ControlSend('Example.Log - Notepad', '', 'Edit1', Chr(94)&Chr(99))
WinKill('Example.Log - Notepad')
Run('Notepad.exe')
WinWaitActive('Untitled - Notepad')
ControlSend('Untitled - Notepad', '', 'Edit1', Chr(94)&Chr(118))
ControlSend('Untitled - Notepad', '', 'Edit1', Chr(94)&Chr(115))
WinWaitActive('Save As')
ControlSend('Save As', '', 'Edit1', 'Example')
ControlClick('Save As', '', 'Button2')
ControlClick('Save As', 'Yes', 'Button1')
WinKill('Example.txt - Notepad')
MsgBox(0, "Done!", "It took " & TimerDiff($Timer) / 1000 & " seconds.")

[center]Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.[/center]

Share this post


Link to post
Share on other sites

Well this option doesn't look so good if you are using it other than personal use, but it does take about 2 seconds to complete.

$Timer = TimerInit()
Opt('SendKeyDelay', 0)
Opt('WinTextMatchMode', 2)
Run('Notepad c:\Example.Log')
WinWaitActive('Example.Log - Notepad')
ControlSend('Example.Log - Notepad', '', 'Edit1', Chr(94)&Chr(97))
ControlSend('Example.Log - Notepad', '', 'Edit1', Chr(94)&Chr(99))
WinKill('Example.Log - Notepad')
Run('Notepad.exe')
WinWaitActive('Untitled - Notepad')
ControlSend('Untitled - Notepad', '', 'Edit1', Chr(94)&Chr(118))
ControlSend('Untitled - Notepad', '', 'Edit1', Chr(94)&Chr(115))
WinWaitActive('Save As')
ControlSend('Save As', '', 'Edit1', 'Example')
ControlClick('Save As', '', 'Button2')
ControlClick('Save As', 'Yes', 'Button1')
WinKill('Example.txt - Notepad')
MsgBox(0, "Done!", "It took " & TimerDiff($Timer) / 1000 & " seconds.")

Can you explain what this is supposed to do? I ran it and Notepad opened with the Example.log file and then nothing. I closed notepad (after waiting a few seconds) and still nothing. I finally exited from the script without a clue as to what it did (or was doing).

Sorry if that sounds bad - no offense


Agreement is not necessary - thinking for one's self is!

My-Colors.jpg

cuniform2.gif

Share this post


Link to post
Share on other sites

Well if you go down the list one by one:

1. Opens Example.Log

2. Waits for Example.Log to be active

3. Sends ^a (ctrl+a) or Chr(94)&Chr(97) to Example.Log to highlight

4. Sends ^c (ctrl+c) or Chr(94)&Chr(99) to Example.Log to copy

5. Winkill = close the Example.Log

6. Opens notepad

7. Waits for not pad to be active

8. Sends ^v (ctrl+v) or Chr(94)&Chr(118) to Untitled Notepad to paste

9. Sends ^s (ctrl+s) or Chr(94)&Chr(115) to Untitled Notepad to save

10. Sends the name 'Example' to be saved as 'Example.txt'

11. Control Clicks Save

12. Control Clicks yes to overwrite if file already exists

13. Closes the window.

14. Gives you how long it took

You're computer is a bit slower than the one I wrote this on, I did a test with the script that you wrote, and a 500kb .log file, and it only took 16 seconds... My suggestion is to put sleeps like so in the one I sent you.

$Timer = TimerInit()
Opt('SendKeyDelay', 0)
Opt('WinTextMatchMode', 2)
Run('Notepad c:\Example.Log')
WinWaitActive('Example.Log - Notepad')
ControlSend('Example.Log - Notepad', '', 'Edit1', Chr(94)&Chr(97))
Sleep(500)
ControlSend('Example.Log - Notepad', '', 'Edit1', Chr(94)&Chr(99))
Sleep(500)
WinKill('Example.Log - Notepad')
Run('Notepad.exe')
WinWaitActive('Untitled - Notepad')
ControlSend('Untitled - Notepad', '', 'Edit1', Chr(94)&Chr(118))
Sleep(1000)
ControlSend('Untitled - Notepad', '', 'Edit1', Chr(94)&Chr(115))
WinWaitActive('Save As')
Sleep(300)
ControlSend('Save As', '', 'Edit1', 'Example')
Sleep(700)
ControlClick('Save As', '', 'Button2')
Sleep(500)
ControlClick('Save As', 'Yes', 'Button1')
Sleep(1000)
WinKill('Example.txt - Notepad')
MsgBox(0, "Done!", "It took " & TimerDiff($Timer) / 1000 & " seconds.")

Or adjust them as need be...


[center]Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.[/center]

Share this post


Link to post
Share on other sites

#18 ·  Posted (edited)

Give this a shot....

$File1 = "Example.log"; Saved in UNICODE
$File2 = "Example.txt"; Saved in ASNI
; TYPE command converts UNICODE to ANSI
Run(@ComSpec & ' /c Type "' & $File1 & '" > "' & $File2 & '"',"",@SW_HIDE)

Posted Image

Thanks 'Danny35d' for helping with the code.

Edited by Fossil Rock

Agreement is not necessary - thinking for one's self is!

My-Colors.jpg

cuniform2.gif

Share this post


Link to post
Share on other sites

Ha!! and much nicer :P!


[center]Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.[/center]

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0