UNICODE TO ANSI conversion

[Fossil Rock]

I have a DOS based program that outputs a txt file using UNICODE encoding. When I try to read a line of text in is comes back as gibberish. If I save the file using ANSI encoding I can read it fine. I can not do anything about the output file while the program is running so I was wondering if anyone new a way to convert the Unicode to Ansi when I read it into my script. Or possibly read the Unicode but it not show as jibberish.

Thanks

Edited December 7, 2005 by Fossil Rock

[webmedic]

google for stingconverter. If you know the file name just pump it into this program and give it an output name and then read the output file. Maybe not optimal but it works.

[Fossil Rock]

google for stingconverter. If you know the file name just pump it into this program and give it an output name and then read the output file. Maybe not optimal but it works.

Already thought of that, thanks. But the optimal part is what I'm after. I can convert the file just by using the 'TYPE' command i.e

TYPE Program.log > Program.txt

but I need to constantly be reading the file so this would probably add alot of cpu time or memory consumption.

Another idea would be to strip the encoding, but I don't know how to do that.

[ChrisL]

I had to do this too, I needed to read in the file and also save it.

What I did was used notepad in a hidden window and used ControlRead to read the text from notepad's hidden window. Once I had the text I closed the hidden window.

I did te reverse when I wanted to save it.

[Fossil Rock]

I had to do this too, I needed to read in the file and also save it.

What I did was used notepad in a hidden window and used ControlRead to read the text from notepad's hidden window. Once I had the text I closed the hidden window.

I did te reverse when I wanted to save it.

What kind of performance hit did you take? Do you read/convert a lot of lines?

[Fossil Rock]

What kind of performance hit did you take? Do you read/convert a lot of lines?

This seems to be the only way to do this. Do you have any sample code. I need to know how to open notepad with a specific file.

Thanks

[JSThePatriot]

This seems to be the only way to do this. Do you have any sample code. I need to know how to open notepad with a specific file.

Thanks

nopade.exe "somefilename.txt"

I think that is how it goes.

JS

[Fossil Rock]

nopade.exe "somefilename.txt"

I think that is how it goes.

JS

Thanks, after experimenting, the actual command is Run("Notepad.exe somefilename.txt") - path optional to needs

I appreciate the help.

Edited December 11, 2005 by Fossil Rock

[JSThePatriot]

Thanks, after experimenting, the actual command is Run("Notepad.exe somefilename.txt") - path optional to needs

I appreciate the help.

My apologies. I thought you realised I meant for use with the Run() command.

JS

[Micha1405]

Hello,

this little CMD File can convert all files you want. The sample is for registry files

@Echo Off

echo UniCode to ANSI converter for Registry files.

md ~ANSI

for %%a in (*.reg *.?reg?) do type "%%a">"~ANSI\%%a"

for %%a in (*.reg *.?reg?) do attrib -r -h -s "%%a"

cd ~ANSI

for %%a in (*.reg *.?reg?) do move /y "%%a" ..

cd..

rd ~ANSI

:end of script

Copy it to yout temp directory an run it from a dos box

Edited December 11, 2005 by Micha1405

[ChrisL]

Hello,

this little CMD File can convert all files you want. The sample is for registry files

@Echo Off

echo UniCode to ANSI converter for Registry files.

md ~ANSI

for %%a in (*.reg *.?reg?) do type "%%a">"~ANSI\%%a"

for %%a in (*.reg *.?reg?) do attrib -r -h -s "%%a"

cd ~ANSI

for %%a in (*.reg *.?reg?) do move /y "%%a" ..

cd..

rd ~ANSI

:end of script

Copy it to yout temp directory an run it from a dos box

Interesting.. so can you convert them back to Unicode in the same way?? if so how?

[Fossil Rock]

Hello,

this little CMD File can convert all files you want. The sample is for registry files

@Echo Off

echo UniCode to ANSI converter for Registry files.

md ~ANSI

for %%a in (*.reg *.?reg?) do type "%%a">"~ANSI\%%a"

for %%a in (*.reg *.?reg?) do attrib -r -h -s "%%a"

cd ~ANSI

for %%a in (*.reg *.?reg?) do move /y "%%a" ..

cd..

rd ~ANSI

:end of script

Copy it to yout temp directory an run it from a dos box

What I'm trying to do is read a txt file that is saved in unicode. Converting the file is not difficult, reading the original file is the problem.

When I read the first line of 'Example.log' which contains "Sample text." it returns ÿþS.

Doing the same thing with 'Example.txt' saved as Ansi produces "Sample text." as it should.

I really hope I'm making sense. ( Just for the record the extensions .log and .txt do not effect the encoding)

[Fossil Rock]

OK, after a few hours of pulling my hair out (and I ain't got much) I came up with this little trick to strip the Unicode out. There are only 3 characters that I wanted removed... Null , ÿ and þ

In the Unicode file ÿþ are the first 2 characters of the file and every second character after that is a NULL character.

You need to make 2 files first.... C:\Example.log Saved with UNICODE and C:\Example.txt Saved with ANSI. Fill 'Example.log' with any text you want and leave the 'Example.txt' empty.

I ran this on a 500K file and it only took a minute. If anyone can make it faster I would appreciate the info.

Ultimately I would like to read an entire line in and strip the codes and then write the entire line.... I'll keep working on that.

Enjoy

; Open Files
$file = FileOpen("C:\Example.log", 0)  ; Open for Reading
$file1 = FileOpen("C:\Example.txt", 1) ; Open for Writing

; Check if file is opened for reading
If $file = -1 Then
    MsgBox(0, "Error", "Unable to open file.")
    Exit
EndIf

; Check if file is opened for writing
If $file1 = -1 Then
    MsgBox(0, "Error", "Unable to open file.")
    Exit
EndIf

; Read in 2 characters at a time until the EOF is reached
While 1
    $chars = FileRead($file, 2)
    If @error = -1 Then ExitLoop
        If ($chars) <> "ÿþ" then
            FileWrite ($file1 , StringLeft ($chars , 1))
 
EndIf

Wend

FileClose($file)
FileClose($file1)

[SmOke_N]

Do you have a sample text to play around with?

Edit:

Nevermind, guess it would help not reading these post so late!!

Edited December 14, 2005 by SmOke_N

[SmOke_N]

Well this option doesn't look so good if you are using it other than personal use, but it does take about 2 seconds to complete.

$Timer = TimerInit()
Opt('SendKeyDelay', 0)
Opt('WinTextMatchMode', 2)
Run('Notepad c:\Example.Log')
WinWaitActive('Example.Log - Notepad')
ControlSend('Example.Log - Notepad', '', 'Edit1', Chr(94)&Chr(97))
ControlSend('Example.Log - Notepad', '', 'Edit1', Chr(94)&Chr(99))
WinKill('Example.Log - Notepad')
Run('Notepad.exe')
WinWaitActive('Untitled - Notepad')
ControlSend('Untitled - Notepad', '', 'Edit1', Chr(94)&Chr(118))
ControlSend('Untitled - Notepad', '', 'Edit1', Chr(94)&Chr(115))
WinWaitActive('Save As')
ControlSend('Save As', '', 'Edit1', 'Example')
ControlClick('Save As', '', 'Button2')
ControlClick('Save As', 'Yes', 'Button1')
WinKill('Example.txt - Notepad')
MsgBox(0, "Done!", "It took " & TimerDiff($Timer) / 1000 & " seconds.")

[Fossil Rock]

Well this option doesn't look so good if you are using it other than personal use, but it does take about 2 seconds to complete.

$Timer = TimerInit()
Opt('SendKeyDelay', 0)
Opt('WinTextMatchMode', 2)
Run('Notepad c:\Example.Log')
WinWaitActive('Example.Log - Notepad')
ControlSend('Example.Log - Notepad', '', 'Edit1', Chr(94)&Chr(97))
ControlSend('Example.Log - Notepad', '', 'Edit1', Chr(94)&Chr(99))
WinKill('Example.Log - Notepad')
Run('Notepad.exe')
WinWaitActive('Untitled - Notepad')
ControlSend('Untitled - Notepad', '', 'Edit1', Chr(94)&Chr(118))
ControlSend('Untitled - Notepad', '', 'Edit1', Chr(94)&Chr(115))
WinWaitActive('Save As')
ControlSend('Save As', '', 'Edit1', 'Example')
ControlClick('Save As', '', 'Button2')
ControlClick('Save As', 'Yes', 'Button1')
WinKill('Example.txt - Notepad')
MsgBox(0, "Done!", "It took " & TimerDiff($Timer) / 1000 & " seconds.")

Can you explain what this is supposed to do? I ran it and Notepad opened with the Example.log file and then nothing. I closed notepad (after waiting a few seconds) and still nothing. I finally exited from the script without a clue as to what it did (or was doing).

Sorry if that sounds bad - no offense

[SmOke_N]

Well if you go down the list one by one:

1. Opens Example.Log

2. Waits for Example.Log to be active

3. Sends ^a (ctrl+a) or Chr(94)&Chr(97) to Example.Log to highlight

4. Sends ^c (ctrl+c) or Chr(94)&Chr(99) to Example.Log to copy

5. Winkill = close the Example.Log

6. Opens notepad

7. Waits for not pad to be active

8. Sends ^v (ctrl+v) or Chr(94)&Chr(118) to Untitled Notepad to paste

9. Sends ^s (ctrl+s) or Chr(94)&Chr(115) to Untitled Notepad to save

10. Sends the name 'Example' to be saved as 'Example.txt'

11. Control Clicks Save

12. Control Clicks yes to overwrite if file already exists

13. Closes the window.

14. Gives you how long it took

You're computer is a bit slower than the one I wrote this on, I did a test with the script that you wrote, and a 500kb .log file, and it only took 16 seconds... My suggestion is to put sleeps like so in the one I sent you.

$Timer = TimerInit()
Opt('SendKeyDelay', 0)
Opt('WinTextMatchMode', 2)
Run('Notepad c:\Example.Log')
WinWaitActive('Example.Log - Notepad')
ControlSend('Example.Log - Notepad', '', 'Edit1', Chr(94)&Chr(97))
Sleep(500)
ControlSend('Example.Log - Notepad', '', 'Edit1', Chr(94)&Chr(99))
Sleep(500)
WinKill('Example.Log - Notepad')
Run('Notepad.exe')
WinWaitActive('Untitled - Notepad')
ControlSend('Untitled - Notepad', '', 'Edit1', Chr(94)&Chr(118))
Sleep(1000)
ControlSend('Untitled - Notepad', '', 'Edit1', Chr(94)&Chr(115))
WinWaitActive('Save As')
Sleep(300)
ControlSend('Save As', '', 'Edit1', 'Example')
Sleep(700)
ControlClick('Save As', '', 'Button2')
Sleep(500)
ControlClick('Save As', 'Yes', 'Button1')
Sleep(1000)
WinKill('Example.txt - Notepad')
MsgBox(0, "Done!", "It took " & TimerDiff($Timer) / 1000 & " seconds.")

Or adjust them as need be...

[Fossil Rock]

Give this a shot....

$File1 = "Example.log"; Saved in UNICODE
$File2 = "Example.txt"; Saved in ASNI
; TYPE command converts UNICODE to ANSI
Run(@ComSpec & ' /c Type "' & $File1 & '" > "' & $File2 & '"',"",@SW_HIDE)

Thanks 'Danny35d' for helping with the code.

Edited December 19, 2005 by Fossil Rock

[SmOke_N]

Ha!! and much nicer !