[Solved] How to check a file is a Plain text file?

[Trong]

My script shall have to converting all text files to UTF-8 and Line Breaks into CRLF!
The problem is that if the file is not a text file it will be broken!

#include <File.au3>
Local $hOpen, $sNewContent, $sLine
Local $sFileList = _FileListToArrayRec(@ScriptDir, "*.*", 1, 1, 1, 2)
If IsArray($sFileList) Then
    For $i = 1 To $sFileList[0] - 1
        ConsoleWrite($sFileList[$i] & " >" & FileSetAttrib($sFileList[$i], "-RASHNOT") & @CRLF)
        $sNewContent = ""
        $sLine = FileReadToArray($sFileList[$i])
        If IsArray($sLine) Then
            For $x = 0 To UBound($sLine) - 1
                $sNewContent &= $sLine[$x] & @CRLF
            Next
        EndIf
        $hOpen = FileOpen($sFileList[$i], 2 + 256)
        FileWrite($hOpen, $sNewContent)
        FileClose($hOpen)
    Next
EndIf

The file is treated as plain text like: .txt, .ini, .inf, .csv, .css, .js, .html, .cpp, .h, .sln, .rc, .cs .......
The files are not regarded as plain text like: .doc, .rtf, .obj, .res,  pch, .suo, .lib, .pdf, .jpg, .........

The problem is that the content can not be determined through the extension of the file!

Any ideas? Thanks

Edited June 6, 2016 by Trong

[AutoBert]

Have a look at https://en.wikipedia.org/wiki/List_of_file_signatures but i think it's very expencive (time) to verify each file if signature is same as extension says.

[Trong]

I've realized that the text file does not contain signatures!

I checked and the only signatures of coding!

Edited June 3, 2016 by Trong

[Trong]

Edited June 6, 2016 by Trong

[iamtheky]

By reading it in binary and testing the bytes for characters outside of parameters? My eof thread from last week was solved with something similar.

[kadanefewi]

34 minutes ago, Trong said:

bump 

Try reading all files in binary (will work for binary and text).

 

Local $hfile = FileOpen($filePath, 16)
    Local $data = FileRead($hfile)
    FileClose($filePath)

 

[Trong]

Check char NULL FileOpen("", 16)

Solved!

FileGetEncoding() 

https://en.wikipedia.org/wiki/Byte_order_mark#Representations_of_byte_order_marks_by_encoding

 

Edited June 6, 2016 by Trong