Jump to content

File Encoding for Arrays


Recommended Posts

I need to read log files into an array to search for errors. However when I display the array I get garbage or "chinese characters". Our developers say they are using UTF-8, but FileGetEncoding says the logs are "2048" or $FO_UTF16_BE_NOBOM (2048) = Use Unicode UTF16 Big Endian (without BOM) from the Encoding codes in FileOpen().

There is an app called Detenc that detects the encoding used by files. You have to guess, but it returns correctly when I set the Encoder for UTF-8. I understand Encoding is not etched in stone, but the first character of the file is a capital B, using HxD Hex Editor.

I even have another  topic here about running PowerShell to reencode the file so AutoIt will store the file properly in the array - See:

So I am trying to figure out why AutoIt thinks my logs are not UTF-8.

Here is sample code:

#include <array.au3>
#include <File.au3>

Local $aRetArrayFile
    _FileReadToArray("C:\Logs\Myplayer1.log", $aRetArrayFile)
    _ArrayDisplay($aRetArrayFile)

I won't post the results as it is illegible, but I did attach a screenshot of the _ArrayDisplay results, and this is the first line of the Log file:

BANNER 10/10/2017 15:56:00 ======================================================================

And the Hex from the beginning of the file:

42 41 4E 4E 45 52 20 31 30 2F 31 30 2F 32 30 31 37 20 31 34 3A 33 31 3A 33 35 20 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 0D 0A 42 41 4E 4E 45 52 20

So I don't understand why AutoIt thinks the file is UTF16 BE.

If I can get the Powershell script running, I have a workaround.

BTW none of my other arrays display as garbage, just the log files.

Weird.

Rereading my post, what seems to be missing is the question. I guess my question is, does anyone know why these logs are being displayed incorrectly?

Cheers

Jibs

logjunk.png

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...