Convert ANSI CSV to Unicode CSV Issues

[KenNichols]

I need help converting an ANSI CSV file to a Unicode CSV file.

The ANSI CSV file may contain a line like:

18,"Alpha, Beta & Capa","1234567","","",0,0,"","","","","",0,0,0,"",0,0,"","","","",0,0,0

When I use the below code to create the Unicode CSV file it converts ALL commas to chr(9) aka Horizontal Tab.

I need the commas within the "" to remain.

What can I do to fix this?

Code Sample:

$ANSI = FileRead($FileToConvert)
    $ANSI2 = StringReplace($ANSI,'"','')
    $Unicode = FileOpen($Converted,32+2) ;32=Unicode
    FileWrite($Unicode, $ANSI2)
    FileClose($Unicode)

Any Help is appreciated.

[jchd]

I don't understand your question and I'm tempted to say you don't get it either.

The example line you give shouldn't change between ANSI and Unicode, since it doesn't contain any character outside lower ANSI (which map identically to Unicode codepoints).

ANSI is a character set and implies a 8-bit encoding. Unicode is a (much larger) character set and does _not_ define an encoding. Unicode can be encoded into UTF-7 (deprecated) UTF-8 (recommended for most uses), UTF-16LE, UTF-16BE, UTF-32, all of them with or without BOM. There are other encoding for special/localied purposes.

Why do you think you need to remove all double-quotes like you're doing? That destroys the structure of your records.

Remove the StringReplace, select the Unicode encoding you need (is it _really_ UTF-16LE?) and you should be fine.