Getting garbage in front of "+>AutoIt3.exe ended.rc:0" line when used unicode characters

[VAN0]

When unicode character used in console, sometimes it shows some additional random characters in front of `+>AutoIt3.exe ended.rc:0` line:

It doesn't happen every time, maybe 1/10 tries. But it always on this line.

ConsoleWrite('this: " " is a non-breaking space (0x00A0)' & @CRLF)

 

Anyone know why this happening?

[Jos]

Any Unicode characters in that code?

[VAN0]

Yes, between the quotes (non-breaking space). I know we have to convert string into binary for it to be displayed correctly, but it's not related to this issue - even if unicode characters displayed correctly, it still shows some garbage sometimes.

The amount of garbage is related to type/amount of unicode characters. For example

ConsoleWrite('test中文测试test' & @CRLF & @CRLF)

produces more garbage than a single character:

Edited October 8, 2022 by VAN0

[VAN0]

it looks like the garbage is being added at the end of chunk of text:

ConsoleWrite('test中文测试test' & @CRLF)
sleep(1000)
ConsoleWrite(@CRLF & '----' & @CRLF)

(it adds garbage on empty line)

[Jos]

I always get this in the output pane for that code:

test????test

----

 

[VAN0]

To narrow down where the issue is originated, I run the script in CMD with:

.\autoit3.exe "C:\test.au3" > out.txt

it creates out.txt file with clean text, no garbage.

When I executed script through autoit3Wrapper.au3;

.\autoit3.exe "SciTE4AutoIt3_Portable\AutoIt3Wrapper\AutoIt3Wrapper.au3" /run /prod /ErrorStdOut /in "C:\test.au3" > out.txt

100% it shows something where empty line supposed to be:

+>13:13:00 Starting AutoIt3Wrapper (21.316.1639.1) from:cmd.exe (10.0.19041.746)  Keyboard:A0000409  OS:WIN_10/2009  CPU:X64 OS:X64  Environment(Language:0409)
>Running AU3Check (3.3.16.1)  from:E:\Program Files\AutoIt3  input:C:\test.au3
+>13:13:00 AU3Check ended.rc:0
>Running:(3.3.16.1):E:\Program Files\AutoIt3\autoit3.exe "C:\test.au3"  /errorstdout
+>Setting Hotkeys...--> Press Ctrl+Alt+Break to Restart or Ctrl+BREAK to Stop.
test中文测试test
敒瑳牡⁴牯 O
----
+>13:13:01 AutoIt3.exe ended.rc:0
+>13:13:01 AutoIt3Wrapper Finished.

From CMD tests I've noticed that the "garbage" string more often than not is word "PROCESS", which corresponds with approximately how often SciTe shows no garbage, as if it's hides word "PROCESS".

[argumentum]

..to peer review/troubleshoot, we should check the beta folder and duly stay updated. Just saying. Not accusing but should be verified.
This I request to attend to, is based in my inability to reproduce the mishap.

[VAN0]

That's fair.

Attached test files I'm testing it with, autoit and wrapper are all latest/beta versions. Just unpack and run .cmd files (check their content prior executing, obviously)

It seems this issue only reproduceable on Windows 10, Windows 11 doesn't have this issue, however it also doesn't output unicode characters at all, replacing them with "???"

autoit.zip

Edited October 8, 2022 by VAN0

[argumentum]

1 hour ago, VAN0 said:

however it also doesn't output unicode characters at all, replacing them with "???"

yes, I see the same. Find attached my setup and results ( and basically, all you needed to share are the 1 au3 and 2 cmd ) 

208969-getting-garbage-in-front-of-autoit3exe-endedrc0.zip

[VAN0]

Yeah, I just wanted to be thorough and make sure no other possible configurations interfere.

result_win11.zipresult_win10.zip

[argumentum]

got the same ???? in Server2022. I guess this is as much as I can contribute. All this is above by capacity to aid this happening.  =/ 

[VAN0]

On a side note, not related to the issue at hand, on Windows 11 when string converted to binary, it's correctly output into file, but if script executed in SciTe, console doesn't show it correctly, SciTe on windows 10 shows it correctly.

ConsoleWrite(BinaryToString(StringToBinary('test中文测试test', 4), 1))

 

[VAN0]

There is got to be something wrong with StdoutRead() on windows 10... first it stops setting @error flag if executed in a loop, now it seems to be chocking on unicode...

Here is what I've discovered so far.

In AutoItWrapper under

; Read STDOUT
        $Line = StdoutRead($l_Handle)

I added:

if $Line <> "" then $line = '>begin<' & StringToBinary($Line & "reference text") & '>end<' & @CRLF & $Line & ">end of chunk<" & @CRLF

On Windows 10 it produces something like this:

+>21:06:55 Starting AutoIt3Wrapper (22.611.2153.12) from:cmd.exe (10.0.19041.746)  Keyboard:A0000409  OS:WIN_10/2009  CPU:X64 OS:X64  Environment(Language:0409)
! *** AU3CHECK Error: *** Skipping AU3Check: c:\autoit\au3check.exe Not Found !

>Running:(3.3.16.1):c:\autoit\autoit3.exe "test.au3"  /errorstdout
+>Setting Hotkeys...--> Press Ctrl+Alt+Break to Restart. --> Press Ctrl+BREAK to Stop.
>begin<0x74657374E4B8ADE69687E6B58BE8AF95746573740D0A4974332E657865007265666572656E63652074657874>end<
test中文测试test
It3.exe>begin<0x0D0A2D2D2D2D0D0A7265666572656E63652074657874>end<

----
>end of chunk<
+>21:06:57 AutoIt3 ended. rc:0
+>21:06:57 AutoIt3Wrapper Finished.

Note, there are two >begin< and only one >end of chunk<

After running this multiple times here is collection of binary strings:

0x74657374E4B8ADE69687E6B58BE8AF95746573740D0A 5F50524F43455353                 7265666572656E63652074657874
0x74657374E4B8ADE69687E6B58BE8AF95746573740D0A E69592E791B3E789A1E2B9B420464F00 7265666572656E63652074657874 <- no >end of chunk<
0x74657374E4B8ADE69687E6B58BE8AF95746573740D0A 4150504552007865                 7265666572656E63652074657874 <- no >end of chunk<
0x74657374E4B8ADE69687E6B58BE8AF95746573740D0A 4974332E65786500                 7265666572656E63652074657874 <- no >end of chunk<
0x74657374E4B8ADE69687E6B58BE8AF95746573740D0A 20402C2026262626                 7265666572656E63652074657874

0x746573743F3F3F3F746573740D0A                                                  7265666572656E63652074657874 <- win11
0x74657374E4B8ADE69687E6B58BE8AF95746573740D0A                                  7265666572656E63652074657874 <- win11 with utf8 conversion

The middle section is the "garbage" data.

 

And just for comparison here is output on Windows 11:

+>21:05:57 Starting AutoIt3Wrapper (22.611.2153.12) from:cmd.exe (10.0.22621.608)  Keyboard:A0000409  OS:WIN_11/2009  CPU:X64 OS:X64  Environment(Language:0409)
! *** AU3CHECK Error: *** Skipping AU3Check: C:\autoit\au3check.exe Not Found !

>Running:(3.3.16.1):C:\autoit\autoit3.exe "test.au3"  /errorstdout
+>Setting Hotkeys...--> Press Ctrl+Alt+Break to Restart. --> Press Ctrl+BREAK to Stop.
>begin<0x746573743F3F3F3F746573740D0A7265666572656E63652074657874>end<
test????test
>end of chunk<
>begin<0x0D0A2D2D2D2D0D0A7265666572656E63652074657874>end<

----
>end of chunk<
+>21:05:58 AutoIt3 ended. rc:0
+>21:05:58 AutoIt3Wrapper Finished.

 

AutoIt3Wrapper.au3

Edited October 9, 2022 by VAN0