Jump to content

Search the Community

Showing results for tags 'encoding'.

More search options

  • Search By Tags

    Type tags separated by commas.
  • Search By Author

Content Type


  • General
    • Announcements and Site News
    • Administration
  • AutoIt v3
    • AutoIt Help and Support
    • AutoIt Technical Discussion
    • AutoIt Example Scripts
  • Scripting and Development
    • Developer General Discussion
    • Language Specific Discussion
  • IT Administration
    • Operating System Deployment
    • Windows Client
    • Windows Server
    • Office


  • AutoIt Team
    • Beta
    • MVP
  • AutoIt
    • Automation
    • Databases and web connections
    • Data compression
    • Encryption and hash
    • Games
    • GUI Additions
    • Hardware
    • Information gathering
    • Internet protocol suite
    • Maths
    • Media
    • PDF
    • Security
    • Social Media and other Website API
    • Windows
  • Scripting and Development
  • IT Administration
    • Operating System Deployment
    • Windows Client
    • Windows Server
    • Office


  • Forum
  • AutoIt


  • Community Calendar

Found 11 results

  1. I need to read log files into an array to search for errors. However when I display the array I get garbage or "chinese characters". Our developers say they are using UTF-8, but FileGetEncoding says the logs are "2048" or $FO_UTF16_BE_NOBOM (2048) = Use Unicode UTF16 Big Endian (without BOM) from the Encoding codes in FileOpen(). There is an app called Detenc that detects the encoding used by files. You have to guess, but it returns correctly when I set the Encoder for UTF-8. I understand Encoding is not etched in stone, but the first character of the file is a capital B, using HxD Hex Editor. I even have another topic here about running PowerShell to reencode the file so AutoIt will store the file properly in the array - See: So I am trying to figure out why AutoIt thinks my logs are not UTF-8. Here is sample code: #include <array.au3> #include <File.au3> Local $aRetArrayFile _FileReadToArray("C:\Logs\Myplayer1.log", $aRetArrayFile) _ArrayDisplay($aRetArrayFile) I won't post the results as it is illegible, but I did attach a screenshot of the _ArrayDisplay results, and this is the first line of the Log file: BANNER 10/10/2017 15:56:00 ====================================================================== And the Hex from the beginning of the file: 42 41 4E 4E 45 52 20 31 30 2F 31 30 2F 32 30 31 37 20 31 34 3A 33 31 3A 33 35 20 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 3D 0D 0A 42 41 4E 4E 45 52 20 So I don't understand why AutoIt thinks the file is UTF16 BE. If I can get the Powershell script running, I have a workaround. BTW none of my other arrays display as garbage, just the log files. Weird. Rereading my post, what seems to be missing is the question. I guess my question is, does anyone know why these logs are being displayed incorrectly? Cheers Jibs
  2. Hi all, I need to read a log file into an array, but the log file is encoded as $FO_UTF16_BE_NOBOM (2048) = Use Unicode UTF16 Big Endian (without BOM) per FileGetEncoding (it returns 2048). I have searched how to convert these log files to UTF-8 and finally found a Powershell command. Since then I have been racking my brain trying to get the function to work. The command itself works from a Powerscript prompt: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -Command Get-Content C:\Logs\Myplayer_10-10-17-02-31.log | Set-Content -Encoding utf8 C:\Logs\Myplayer1.log This is my sandbox; #include <array.au3> #include <File.au3> Local $aArrayLogFile Local $sLogDir = "C:\Logs\" Local $sLogFile = "Myplayer_10-10-17-02-31.log" Local $sConvertedLog = "ConvertedLog.log" Local $sLogDirFile = $sLogDir&$sLogFile RunWait("C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -Command Get-Content "&$sLogDirFile&" | Set-Content -Encoding utf8 "&$sConvertedLog,$sLogDir) _FileReadToArray($sLogDirFile, $aArrayLogFile) _ArrayDisplay($aArrayLogFile) Also tried RunWait("C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -Command Get-Content "&$sLogDirFile&" | Set-Content -Encoding utf8 "&$sConvertedLog,$sLogDir) and ShellExecuteWait("C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe"," -Command Get-Content "&$sLogDirFile&" | Set-Content -Encoding utf8 "&$sConvertedLog,$sLogDir) Tried without -Command and a bunch of other parameters that were sprinkled throughout the internet from people trying to get this to work. Thanks Jibs
  3. Hey, I've coma accross a very odd problem. I want to download a CSV-File, and process the contents. This is the extremely simplyfied AutoIT-Code, given the file is already downloaded: $f= @ScriptDir & "\TestFile.csv" $file = FileOpen($f,0) Local $line = FileReadLine($file) MsgBox(0,"",$line) The downloaded CSV file I want to process contains something like this: Artikelnummer;EAN-Code;Artikelname;Artikelgewicht;Beschreibung;Kurzbeschreibung;Eigenschaften;Technische-Daten;Bild1;Bild2;Bild3;Bild4;Bild5;Bild6;Bild7;Bild8;Lieferbar;"Lieferbar Ab";Versandzeit;"UVP-Preis inkl. MwSt.";"Preis1";"Preis2";Hersteller L7335272;5420025602129; Mini Light XLR ;0.1000; JB Systems Schwanenhalsleuchte mit XLR Anschluss. ;;;;http://www.example.com/media/images/org/pic20070114153500a.jpg;;;;;;;;JA;; 1-3 Tage ;12,90;12,90;6,57; JB Systems L3320502;540207025601636; Mini Light LED BNC ;0.1000; JB Systems LED Schwanenhalsleuchte mit BNC Anschluss. ;;;;http://www.example.com/media/images/org/pic20061231171705a.jpg;;;;;;;;JA;; 1-3 Tage ;29,90;25,89;15,26; JB Systems L1332254;542002556023143; Mini Light LED XLR ;0.1000; JB Systems LED Schwanenhalsleuchte mit XLR Anschluss. ;;;;http://www.example.com/media/images/org/pic20061231171728a.jpg;;;;;;;;JA;; 1-3 Tage ;29,90;25,89;15,26; JB Systems L8302591;504200256280277; Spiegelkugel 10cm ;0.5000; JB Systems Spiegelkugel 10cm Durchmesser mit einer hohen Dichte durch 10 x 10 mm Echtglasspiegel. ;;;;http://www.example.com/media/images/org/pic20060324214825a.jpg;;;;;;;;JA;; 1-3 Tage ;5,50;4,90;2,81; JB Systems L7302932;542000256510222; Spiegelkugel 20cm ;0.8400; JB Systems Spiegelkugel 20cm Durchmesser mit einer hohen Dichte durch 10 x 10 mm Echtglasspiegel. ;;;;http://www.example.com/media/images/org/pic20060324214907a.jpg;;;;;;;;JA;; 1-3 Tage ;12,90;11,50;6,58; JB Systems L2350293;534200562064239; Spiegelkugel 30cm ;2.1300; JB Systems Spiegelkugel 30 cm Durchmesser mit einer hohen Dichte durch 10 x 10 mm Echtglasspiegel. ;;;;http://www.example.com/media/images/org/pic20060324214956a.jpg;;;;;;;;JA;; 1-3 Tage ;26,90;23,00;13,72; JB Systems L3302984;545200252024246; Spiegelkugel 40cm ;3.5000; JB Systems Spiegelkugel 40cm Durchmesser mit Sicherungsring und einer hohen Dichte durch 10 x 10 mm Echtglasspiegel. ;;;;http://www.example.com/media/images/org/pic20060324215050a.jpg;;;;;;;;JA;; 1-3 Tage ;54,90;49,00;28,00; JB Systems L9302495;542205056225600; Spiegelkugel 50cm ;5.3900; JB Systems Spiegelkugel 50cm Durchmesser mit Sicherungsring und einer hohen Dichte durch 10 x 10 mm Echtglasspiegel. ;;;;http://www.example.com/media/images/org/pic20060324215122a.jpg;;;;;;;;JA;; 1-3 Tage ;89,00;79,00;45,39; JB Systems But the message box that pops up after FileReadLine contains this: 䅲瑩步汮畭浥爻䕁中䍯摥㭁牴楫敬湡浥㭁牴楫敬来睩捨琻䉥獣桲敩扵湧㭋畲穢敳捨牥楢畮朻䕩来湳捨慦瑥渻呥捨湩獣桥ⵄ慴敮㭂楬搱㭂楬搲㭂楬搳㭂楬搴㭂楬搵㭂楬搶㭂楬搷㭂楬搸㭌楥晥牢慲㬢䱩敦敲扡爠䅢∻噥牳慮摺敩琻≕噐ⵐ牥楳⁩湫氮⁍睓琮∻≐牥楳ㄢ㬢偲敩猲∻䡥牳瑥汬敲ੌ㜳㌵㈷㈻㔴㈰〲㔶〲ㄲ㤻M楮椠䱩杨琠塌刀㬰⸱〰〻J䈠卹獴敭猠卣桷慮敮桡汳汥畣桴攠浩琠塌删䅮獣桬畳献;㬻㭨瑴瀺⼯睷眮數慭灬攮捯洯浥摩愯業慧敳⽯牧⽰楣㈰〷〱ㄴㄵ㌵〰愮橰朻㬻㬻㬻㭊䄻㬀ㄭ㌠呡来;ㄲⰹ〻ㄲⰹ〻㘬㔷㬀䩂⁓祳瑥浳 I've attached both files I use. My guess is, that there's something wrong with the encoding, but I'm not sure how to fix it. Thank you very much for your help - virhonestum Encodingtester.au3 TestFile.csv
  4. Hello, i need to save files with ANSI-Encoding. Since Auto-It it doesn't work in any direction. I tried the following: #include <FileConstants.au3> FileDelete(@ScriptDir&"\Test.txt") $o = FileOpen(@ScriptDir&"\Test.txt", BitOR($FO_BINARY,$FO_ANSI,$FO_OVERWRITE)) FileWrite($o, "Test") FileClose($o) Or #include <FileConstants.au3> FileDelete(@ScriptDir&"\Test.txt") $o = FileOpen(@ScriptDir&"\Test.txt", 514) FileWrite($o, "Test") FileClose($o) Both create UTF-8 encoded files. What am i doing wrong? Thank you!
  5. I want to take a binary data from any source (string,number,files,etc..) and iterate over each X bits of it in a loop, say take bits 1-5, then 6-10, etc.. Then I want to convert these bits to their corresponding decimal value. but all the binary functions I found in autoit only work with full bytes, and do not let me get smaller sections of bits, like "BinaryMid()" that "Extracts a number of bytes from a binary variant" Can anyone tell me if this is possible to do and how? and also if there is a function to convert those bits to/from decimals? I'm not that familiar with dealing directly with binary, so any help or tips will be very appreciated Context: I'm trying to make a function to encode/decode any given binary data into/from a given array of characters. just like Base64 but using different bases then [a-z A-Z 0-9 +/= ]. My approach currently is to figure out how many bits of binary I can encode with each character, read those bits and convert them to a decimal number, then I will use that number as an index and take the character at that index from the character array and add it to the result string. I'm aware that there may be some padding needed.
  6. Hey fellas, I'm pulling data from web with multiple languages. I get a lot of ü and other special characters which show correctly in Scite when I change the encoding. However, I need the special characters to be displayed in an array. For example I get ' ĂĽ ' in an array which should translate ' ü ' . How do I do this? Thanks, Semiko
  7. Hi guys. I am working on Windows 10. I installed autoit and SciTE4AutoIt3. I tried to load a script I have which includes hebrew letters. The problem is that I cannot see these letters. I see something like this: ëãåøâì Also, when I print $oLink.innertext to console I can see only question marks. How can I fix this situation? -------------------------------------- Guys, after "solving" the problem above, I now get exclamation marks in the SciTE output window instead of Hebrew letters. Now, how do I solve this one?? --------------------------------------- Thank you!
  8. Ciao I need to save a file with ISO 8859-1 encoding Now my script is: $contenuti_header = "Configuration file <> O:\test.txt <> Last update: 28/09/2015 09:46:36 ð ñ" $record_part1 = "ò1ò0ò1òLayer_01ïï1ï0ï0ï0ï0ï0ï0ïï0ï255ï0ï0ï0ï255ï255ï125ï0ï0ï3ï0ï0ï0ï0ï0ï0ï0ï0ï0ï3ï150ï5ï102ï82ï10ï4ï0ï0ïTahoma;16777215;22;1;ï0ï0ï0ïïïï0ï0ï33ï500ï200ï4ï1ï720ï10ï5ï100ï63ï300ï850ï16777215ï0ï0ï0ï0ï0ï0ï0ï0ï0ï0ï" $record_part2 ="ï0ïTahoma;16777215;22;1;ïïhh.nnïïï0ïï0ï0ï0ï4ï65793ï255ï0ï2ï0ï0ïï5ï5ï0ï0ï0ï0ï0ï100ï100ï0ï0ï8421504ï16777215ïïï0ï0ïïï0ï720ï576ï0ï0ï0ï0ï0ï0ï0ï0ï2ï0ï6ï0ï0ï1ï0ï1ï0ï0ïîò0ò0òòòòòòó" for $i = 1 to $num $contenuti_record = $record_part1 & "ò" & $num & $record_part2 Next $contenuti_end = "òòòòòòòòòòòòòó" $fileTemplate = FileOpen("O:\file.txt", 2) FileWrite($fileTemplate, $contenuti_header&$contenuti_record&$contenuti_end) FileClose($fileTemplate)Autoit writes the file with UTF-8 encoding but the programs that read that file can't read it correctly. If I open file.txt with SublimeText and save it with ISO 8859-1 encoding, the program works well Thanks Marco
  9. Hello i have lots of text like this i have no idea what does this \u stuff means, after googling it i found that it is some sort of utf encoding how could i write this text to a file without loosing any characters and getting rid of \u ? EDIT: after more digging i found that i need "Converting Unicode Entities to Unicode Text" any ideas hoe to do that with autoit ?
  10. First, the value of my $LineArray[0] is "歌词制作 生态。破坏".. And whenever I tried to run this code: FileWriteLine($PathName, $LineArray[0] & @CRLF) The line of the created file produced the "???? ?????" null characters.. Now, what can I do to make it write the original (I think) Chinese characters?
  11. Hi, i'm looking for some functionality that can unescape strings that has been encoded. First and foremost i'm looking for a plain alternative to javascripts unescape() Func _URIDecode($sData) ; Prog@ndy Local $aData = StringSplit(StringReplace($sData,"+"," ",0,1),"%") $sData = "" For $i = 2 To $aData[0] $aData[1] &= Chr(Dec(StringLeft($aData[$i],2))) & StringTrimLeft($aData[$i],2) Next Return BinaryToString(StringToBinary($aData[1],1),4) EndFunc It only works if all characters are escaped..it can not be mixed. for an example, it fails if the string contains a unescaped %.. $myres1 = _URIDecode('Jo%C3%A3o%20Guimar%C3%A3es%20Rosa') ConsoleWrite($myres1 & @CRLF) ; Outputs: "João Guimarães Rosa" - (Works fine!) $myres2 = _URIDecode("I was about 99% done!") ConsoleWrite($myres2 & @CRLF) ; Outputs: "I was about 99" So..before is start improving the above snippet, do a library exist that can handle these things? (HTML en/decoding would be nice) EDIT: Hmm..i just tried javascripts method.. it also fails when single %... offcourse.. hmm.. nevermind. My question is answered.. ill leave the post