create text file with windows-1255 encode

[shai 0]

hi

i try to create text file with _FileCreate function that encode of this file will be "windows-1255" but i can't.

 

the file always created in utf8 encode.

 

i also tryed to do it with FileOpen($sFilePath, BitOR(2, 8))

 

and still is created in utf8 encode.

 

thanks for all your solutions.

Edited March 1, 2015 by shai

[VIP 19]

FileWrite("FileName.txt","ANSI Content")

[Jos 1,430]

and still is created in utf8 encode.

What are you writing to the file?

How do you determine it's encoding eand what is the encoding exactly.. UTF8 with or without BOM?

Jos

[shai 0]

What are you writing to the file?

How do you determine it's encoding eand what is the encoding exactly.. UTF8 with or without BOM?

Jos

 

i writing simple text to the file

when i open the txt file in notepad++ i see that file is utf8 WO BOM

[shai 0]

FileWrite("FileName.txt","ANSI Content")

 

its also create file on utf8 WO BOM

[Jos 1,430]

i writing simple text to the file

when i open the txt file in notepad++ i see that file is utf8 WO BOM

So it is a regular file and Notead++ opens it as UTF8 without bom... the new SciTE will do the same to support multibyte chaacters by default.

What is the simply text you write?

Does it contain multi-byte characters?

Show some code you use to create the file.

Jos

[shai 0]

even if i try this code:

FileWrite("c:\FileName.txt","ANSI Content")

the FileName.txt file is in utf8

[Jos 1,430]

No it is not....  it is both ansi and utf8 without BOM as there are no special characters in there to indicate either one.

Jos

[shai 0]

how i can check what is the real encoding of output file?

Edited March 1, 2015 by shai

[VIP 19]

a/ă

expandcollapse popup

Local $sFile = "FileToCheckEncoding.txt"
Local $hfOpen = FileOpen($sFile, 2 + 8)
FileWrite($hfOpen, "Đào Van Trong")
ConsoleWrite("Encoding of file " & $sFile & " is: " & _GetFileEncoding($sFile) & " - " & @extended & @CRLF)

$hfOpen = FileOpen($sFile, 2 + 8 + 32)
FileWrite($hfOpen, "UTF16 Little Endian: Đào Văn Trong")
ConsoleWrite("Encoding of file " & $sFile & " is: " & _GetFileEncoding($sFile) & " - " & @extended & @CRLF)

$hfOpen = FileOpen($sFile, 2 + 8 + 64)
FileWrite($hfOpen, "UTF16 Big Endian: Đào Văn Trong")
ConsoleWrite("Encoding of file " & $sFile & " is: " & _GetFileEncoding($sFile) & " - " & @extended & @CRLF)

$hfOpen = FileOpen($sFile, 2 + 8 + 128)
FileWrite($hfOpen, "UTF8 (with BOM): Đào Văn Trong")
ConsoleWrite("Encoding of file " & $sFile & " is: " & _GetFileEncoding($sFile) & " - " & @extended & @CRLF)

$hfOpen = FileOpen($sFile, 2 + 8 + 256)
FileWrite($hfOpen, "UTF8 (without BOM): Đào Văn Trong")
ConsoleWrite("Encoding of file " & $sFile & " is: " & _GetFileEncoding($sFile) & " - " & @extended & @CRLF)

; * -----:|
Func _GetFileEncoding($sFile)
    ; /*********************************************************************** * Trong
    ; * ---:| Function......: _GetFileEncoding($sFile)
    ; * ---:| Description...: Get File Encoding!
    ; ************************************************************************ *
    Local $sEndoding = FileGetEncoding($sFile), $sEncode
    $sEncode = $sEndoding
    If $sEndoding = 0 Then $sEndoding = "ANSI"
    If $sEndoding = 32 Then $sEndoding = "UTF16 Little Endian"
    If $sEndoding = 64 Then $sEndoding = "UTF16 Big Endian"
    If $sEndoding = 128 Then $sEndoding = "UTF8 (with BOM)"
    If $sEndoding = 256 Then $sEndoding = "UTF8 (without BOM)"
    If $sEndoding = -1 Then SetError(1, 0, -1)
    Return SetError(0, $sEncode, $sEndoding)
EndFunc   ;==>_GetFileEncoding
; * -----:|

Edit:

In Notepad++

Edited March 1, 2015 by Trong

[shai 0]

ok now i see that if i writing simple text is works

but if i writing text from HTTP.ResponseText object the file swich to utf8.

how i can keep the file in ansi format?

[VIP 19]

Local $sFile = "FileToCheckEncoding.txt"
FileClose(FileOpen($sFile, 2 + 8))
Local $hfOpen = FileOpen($sFile, 1 + 8)
FileWrite($hfOpen, "UTF8 to ANSI")
FileClose($hfOpen)

Edited March 1, 2015 by Trong

[shai 0]

Local $sFile = "FileToCheckEncoding.txt"
FileClose(FileOpen($sFile, 2 + 8))
Local $hfOpen = FileOpen($sFile, 1 + 8)
FileWrite($hfOpen, "UTF8 to ANSI")
FileClose($hfOpen)

 

i try this but after FileWrite the new file is still on utf8

[VIP 19]

>#10 

[shai 0]

i create this function:

Func utf8towindows1255($hFileorg, $hFiledest)
    Local $hFileOpen = FileOpen($hFileorg, 0)
    Local $sFileRead = FileRead($hFileOpen)
    FileClose($hFileOpen)
    FileClose(FileOpen($hFiledest, 2 + 8))
    Local $hfOpen = FileOpen($hFiledest, 1 + 8)
    FileWrite($hfOpen, $sFileRead)
    FileClose($hfOpen)
EndFunc

and i run it on file that is encode in utf8.

 

if i run it in my full script the output file is in utf8

but if i run it on new script with only this function the output file is in ANSI.

 

why?

Edited March 3, 2015 by shai

[VIP 19]

Your OS Version? XP/7/8/10/x32/x64 ?
Your OS Language? English/China?
Your AutoIt Version? 3.3.6.1/3.3.8.1/3.3.10.2/3.3.12.0/3.3.13.19/AutoitCN-3.3.9.21 ?

It worked fine.

MsgBox(0,"","File Encoding is: "&utf8towindows1255("UTFFile.txt", "ansi.txt"))


Func utf8towindows1255($hFileorg, $hFiledest)
    Local $hFileOpen = FileOpen($hFileorg, 0)
    Local $sFileRead = FileRead($hFileOpen)
    FileClose($hFileOpen)
    FileClose(FileOpen($hFiledest, 2 + 8))
    Local $hfOpen = FileOpen($hFiledest, 1 + 8)
    FileWrite($hfOpen, $sFileRead)
    FileClose($hfOpen)
    Return _GetFileEncoding($hFiledest)
EndFunc

; * -----:|
Func _GetFileEncoding($sFile)
    ; /*********************************************************************** * Trong
    ; * ---:| Function......: _GetFileEncoding($sFile)
    ; * ---:| Description...: Get File Encoding!
    ; ************************************************************************ *
    Local $sEndoding = FileGetEncoding($sFile), $sEncode
    $sEncode = $sEndoding
    If $sEndoding = 0 Then $sEndoding = "ANSI"
    If $sEndoding = 32 Then $sEndoding = "UTF16 Little Endian"
    If $sEndoding = 64 Then $sEndoding = "UTF16 Big Endian"
    If $sEndoding = 128 Then $sEndoding = "UTF8 (with BOM)"
    If $sEndoding = 256 Then $sEndoding = "UTF8 (without BOM)"
    If $sEndoding = -1 Then SetError(1, 0, -1)
    Return SetError(0, $sEncode, $sEndoding)
EndFunc   ;==>_GetFileEncoding

[AlienStar 0]

On 3/1/2015 at 8:55 PM, VIP said:

a/ă

 

expandcollapse popup

Local $sFile = "FileToCheckEncoding.txt"
Local $hfOpen = FileOpen($sFile, 2 + 8)
FileWrite($hfOpen, "Đào Van Trong")
ConsoleWrite("Encoding of file " & $sFile & " is: " & _GetFileEncoding($sFile) & " - " & @extended & @CRLF)

$hfOpen = FileOpen($sFile, 2 + 8 + 32)
FileWrite($hfOpen, "UTF16 Little Endian: Đào Văn Trong")
ConsoleWrite("Encoding of file " & $sFile & " is: " & _GetFileEncoding($sFile) & " - " & @extended & @CRLF)

$hfOpen = FileOpen($sFile, 2 + 8 + 64)
FileWrite($hfOpen, "UTF16 Big Endian: Đào Văn Trong")
ConsoleWrite("Encoding of file " & $sFile & " is: " & _GetFileEncoding($sFile) & " - " & @extended & @CRLF)

$hfOpen = FileOpen($sFile, 2 + 8 + 128)
FileWrite($hfOpen, "UTF8 (with BOM): Đào Văn Trong")
ConsoleWrite("Encoding of file " & $sFile & " is: " & _GetFileEncoding($sFile) & " - " & @extended & @CRLF)

$hfOpen = FileOpen($sFile, 2 + 8 + 256)
FileWrite($hfOpen, "UTF8 (without BOM): Đào Văn Trong")
ConsoleWrite("Encoding of file " & $sFile & " is: " & _GetFileEncoding($sFile) & " - " & @extended & @CRLF)

; * -----:|
Func _GetFileEncoding($sFile)
    ; /*********************************************************************** * Trong
    ; * ---:| Function......: _GetFileEncoding($sFile)
    ; * ---:| Description...: Get File Encoding!
    ; ************************************************************************ *
    Local $sEndoding = FileGetEncoding($sFile), $sEncode
    $sEncode = $sEndoding
    If $sEndoding = 0 Then $sEndoding = "ANSI"
    If $sEndoding = 32 Then $sEndoding = "UTF16 Little Endian"
    If $sEndoding = 64 Then $sEndoding = "UTF16 Big Endian"
    If $sEndoding = 128 Then $sEndoding = "UTF8 (with BOM)"
    If $sEndoding = 256 Then $sEndoding = "UTF8 (without BOM)"
    If $sEndoding = -1 Then SetError(1, 0, -1)
    Return SetError(0, $sEncode, $sEndoding)
EndFunc   ;==>_GetFileEncoding
; * -----:|

Edit:

In Notepad++

it works fine but I wanna set ini file in "UTF8 (without BOM)" encoding

but after I use iniwrite it backs as "ANSI" !!!!!!!

$hfOpen = FileOpen($sFile, 2 + 8 + 256)
iniWrite($sfile, "General", "Title", "AutoIt")
ConsoleWrite("Encoding of file " & $sFile & " is: " & _GetFileEncoding($sFile) & " - " & @extended & @CRLF)

output:

Encoding of file FileToCheckEncoding.txt is: ANSI - 0

please help

Edited November 21, 2017 by AlienStar

[VIP 19]

INI files by design are not Unicode compatible. There are workarounds to get them to be able to utilize Unicode characters, but it's non-standard to use them with Unicode characters. 

So what you need to do is encode your section, key and value strings in UTF-8 and store that as if it was ANSI. That can't make the standard function behave strangely UNLESS you share this ini file with application not aware that sections, keys and values actually are UTF-8.

Note that the ini file is itself an ANSI file (no UTF-8 BOM).

 

jchd:

Here's the secret sauce behind the hood:

Local $sText = "中國 한국어 ไทย"
Local $sUTF8 = BinaryToString(StringToBinary($sText, 4), 1)
IniWrite("my.ini", "test", "key", $sUTF8)
Local $sRaw = IniRead("my.ini", "test", "key", "I'm not there")
Local $sValue = BinaryToString(StringToBinary($sRaw, 1), 4)
MsgBox(0, "Round-trip check", $sValue)