Please help me find/replace a character

[leuce]

Hello everyone

In the attachment is a plain text file in UTF8 (without byte order mark) with a troublesome character between <seg> and </seg>.  I'm trying to get an AutoIt script to find that character in a text file and replace it with something else, but it resists all my efforts.  This character is so elusive that I can't even see it in my text editor, and my cursor doesn't stop at the character when I use the left/right arrow keys -- however, if I open the file in MS Word, then I can see it.  I have no idea how to encode this character so that AutoIt can find and replace it.

I hope someone here can help identify it and tell me how to find/replace it.

In MS Word, the character is called an "optional hyphen" and is displayed in MS Word as "¬", but the character itself is not "¬" (that's just MS Word's way of displaying it).

Thanks

Samuel

PS. I zipped the file to ensure that it doesn't break when I upload it (sorry, I'm not sure if the current forum software will read the attachment as text or binary, so I zipped it to ensure it's binary).

character.zip

Edited September 2, 2015 by leuce

[leuce]

I found a text editor that can see the character, namely the clunky old favourite, Babelpad.  Babelpad identified the character as "U+001F INFORMATION SEPARATOR ONE".  So, my question has become: how can I tell AutoIt to find U+001F and replace it with e.g. "ASDF"?  Thanks.

[leuce]

Found it.  Babelpad gives me the hex Unicode code, so I have to use Google to find the decimal Unicode code, and then add it to a variable using ChrW().  In the case of 001F, the decimal code is 31, so I was able to "find" the character in my AutoIt script using ChrW (31).

[TheDcoder]

Wow! Great! I got the info which I need for my project

 

Thanks @leuce! TD

[Skysnake]

How about simply stripping the non-printing characters from between those two tags?  Looks like a bugger.

https://en.wikipedia.org/wiki/Delimiter

https://en.wikipedia.org/wiki/ANSI_escape_code

See the StringRegExp and StringInStr, StringRegExpReplace functions in the HelpFile?

[jchd]

leuce,

Both SciTE and NotePad++ show the control character in its abbreviated form (US = Unit separator). Also you can as well do the following:

#include <Array.au3>
Local $s = "<seg></seg>"  ; your text file
Local $a = StringToASCIIArray($s)
_ArrayDisplay($a)

 

[leuce]

Both SciTE and NotePad++ show the control character in its abbreviated form (US = Unit separator).

Thanks.  Does SciTE or Notepad++ also tell you that the decimal Unicode code is 31?

[jchd]

You just have to use AutoIt help: Appendix > ASCII characters > Control characters and there > US : 31 / 1F / 037 / Unit separator

[BrewManNH]

Found it.  Babelpad gives me the hex Unicode code, so I have to use Google to find the decimal Unicode code,

No you don't, ChrW will accept a hex number as well as a decimal number, they're just numbers they aren't special.

ConsoleWrite(ChrW(0x41) & @CRLF) ; decimal 65, letter A