Getting rid of a line break(?) in txt file

[JustinN]

I have a text file I need to open up and remove these characters from:

I have tried many different scripts and lots of tweaking but I'm not a polished coder and it was all just guessing and testing. Need some help.

Thanks.

[JohnOne]

Post an unpolished test script.

[JustinN]

Hey John. Basically I've been messing around rather unsuccessfully with these:

#Include <File.au3>
Global $success = False
$filename = "file.csv"
$line_text_input = '@CR'
$file_count_lines = _FileCountLines($filename)
for $i = 0 to $file_count_lines
$Lines_text_output = FileReadLine($filename, $i)
if StringInStr($Lines_text_output, $line_text_input) then
StringRegExpReplace($i, "rn|r|n", " ")
     $success = True
EndIf
Next

#Include <File.au3>
lobal $success = False
$filename = "file.csv"
$line_text_input = "nr"
$file_count_lines = _FileCountLines($filename)
for $i = 0 to $file_count_lines
$Lines_text_output = FileReadLine($filename, $i)
if StringInStr($Lines_text_output, $line_text_input) then
_FileWriteToLine($filename, $i, "", 1)
$success = True
EndIf
Next

Oh also as you can see the files are .CSV, but they don't contain any commas or anything just data in between quotes like so:

"line one"

"line two"

"line three"

Edited June 23, 2012 by JustinN

[JohnOne]

I'm not a great knower of this stuff, but in lieu of someone else who

knows I will offer a suggestion.

It is possible that it is some sort of UTF problem.

Try taking a look at using FileOpen() in conjunction with FileRead/FileReadLine, remembering to use the handle returned by FileOpen, in your calls to FileRead/FileReadLine instead of filename.

But when opening the file, try some different modes.

Sorry I could not be of more help.

[Robjong]

Hi,

give this a try:

Local $hFile = FileOpen("file.csv", 0), $sLine
If $hFile == -1 Then
ConsoleWrite("Unable to open file." & @CRLF)
Else
While 1
$sLine = FileReadLine($hFile)
If @error == -1 Then ExitLoop
$sLine = StringRegExpReplace($sLine, "[\v\t]+", " ") ; \v = vertical space, \t = tab
ConsoleWrite($sLine & @CRLF)
; do something with $sLine
WEnd
FileClose($hFile)
EndIf

...if this does not work you should post an example CSV for testing.

[water]

This code will read the first 50 bytes of your file and display it in two lines. So we know which special characters we are talking about:

$sFilename = "file.csv"
Global $sText
$sText = FileRead($sFilename, 50)
For $i = 1 To StringLen($sText)
    $Char = StringMid($sText, $i, 1)
    If Asc($Char) >= 32 Then
        ConsoleWrite(" " & $Char & " ")
    Else
        ConsoleWrite(" . ")
    EndIf
Next
ConsoleWrite(@LF)
For $i = 1 To StringLen($sText)
    ConsoleWrite(Hex(Asc(StringMid($sText, $i, 1)), 2) & " ")
Next
ConsoleWrite(@LF)

"ab" & @CRLF & "cde"

will be displayed as:

a b . . c d e

61 62 0D 0A 63 64 65

Edited June 23, 2012 by water

[Robjong]

@water, great idea! Hope you don't mind if I borrow it.

[water]

@Robjon,

I've extended the code a bit and put it into a separate function. Pass the string to translate (shown as character, hex and decimal) and the length of each output line.

Func _String2AscHex($sString, $iLength)

    Local $sStringAsc, $sStringDec, $sStringHex, $sChar, $iIndex, $iPos = 1
    For $iIndex = 1 To StringLen($sString)
        $sChar = StringMid($sString, $iIndex, 1)
        If Asc($sChar) >= 32 Then
            $sStringAsc = $sStringAsc & "  " & $sChar & " "
        Else
            $sStringAsc = $sStringAsc & "  . "
        EndIf
        $sStringHex = $sStringHex & " " & Hex(Asc(StringMid($sString, $iIndex, 1)), 2) & " "
        $sStringDec = $sStringDec & Stringright("00" & Asc(StringMid($sString, $iIndex, 1)), 3) & " "
    Next
    While $iPos < StringLen($sString)
        ConsoleWrite(StringStripWs(StringMid($sStringAsc, ($iPos * 4) - 3, $iLength * 4), 2) & @LF)
        ConsoleWrite(StringStripWS(StringMid($sStringHex, ($iPos * 4) - 3, $iLength * 4), 2) & @LF)
        ConsoleWrite(StringStripWS(StringMid($sStringDec, ($iPos * 4) - 3, $iLength * 4), 2) & @LF & @LF)
        $iPos += $iLength
    WEnd

EndFunc

[AZJIO]

#include <Array.au3>

Global $Chr[33] = ['NUL', 'SOH', 'STX', 'ETX', 'EOT', 'ENQ', 'ACK', 'BEL', 'BS', 'HT', 'LF', 'VT', 'FF', 'CR', 'SO', 'SI', _
        'DLE', 'DC1', 'DC2', 'DC3', 'DC4', 'NAK', 'SYN', 'ETB', 'CAN', 'EM', 'SUB', 'ESC', 'FS', 'GS', 'RS', 'US', 'Space']
; $sText = FileRead(@ScriptDir&'file.txt')
$sText = FileRead('C:ntldr')
If StringLen($sText) > 4000 Then
    MsgBox(0, 'Long', 'Cut off')
    $sText = StringLeft($sText, 4000)
EndIf

$aText = StringSplit($sText, '')
Global $aView[$aText[0] + 1][4] = [[$aText[0]]]

For $i = 1 To $aText[0]
    $aView[$i][0] = $aText[$i]
    $aView[$i][1] = Asc($aText[$i])
    $aView[$i][2] = Hex(Int($aView[$i][1]), 2)
    $aView[$i][3] = StringFormat("%03o", $aView[$i][1])

    Switch $aView[$i][1]
        Case 0 To 32
            $aView[$i][0] = $Chr[$aView[$i][1]]
        Case 127
            $aView[$i][0] = 'DEL'
        Case 160
            $aView[$i][0] = 'Space2'
    EndSwitch
Next

_ArrayDisplay($aView, 'View = Txt|Dec|Hex|Oct')

; Delete characters using the octal number
$sText = FileRead('C:ntldr')
$sText=StringRegExpReplace($sText, '[000-037]', ' ')
MsgBox(0, 'Message', $sText)

Edited June 23, 2012 by AZJIO

[Robjong]

I've extended the code a bit and put it into a separate function. Pass the string to translate (shown as character, hex and decimal) and the length of each output line.

Cool, I like the 'line wrapping'. I did the same but just a little bit different, it splits the string on line endings and returns a multidimensional array. (also included the decimals)

expandcollapse popup

Local $sString = "foo " & @TAB & " bar" & @CR
$sString &= "blah " & @TAB & '' & @TAB & " blah" & @LF
$sString &= "123... " & @TAB & '...456...' & @TAB & " ...789" & @CRLF

Local $aCharListStr = _StringCCLS($sString)
If @error Then Exit ConsoleWrite("ERROR: ????" & @LF)
For $i = 1 To $aCharListStr[0][0]
    ConsoleWrite('   ' & $aCharListStr[$i][1] & @LF)
    ConsoleWrite('+> ' & $aCharListStr[$i][2] & @LF)
    ConsoleWrite('-> ' & $aCharListStr[$i][3] & @LF)
Next


Func _StringCCLS($sString)
    Local $aLines = StringRegExp($sString, "V*R?", 3)
    If Not IsString($sString) Or StringLen($sString) < 1 Then Return SetError(1, 0, 0)
    Local $iUBound = UBound($aLines) - 1, $aReturn[$iUBound + 1][4] = [[$iUBound]], $sChar
    For $n = 1 To $iUBound
        $sString = $aLines[$n - 1]
        $aReturn[$n][0] = $sString
        For $i = 1 To StringLen($sString)
            $sChar = StringMid($sString, $i, 1)
            Switch $sChar
                Case @LF
                    $aReturn[$n][1] &= "n  "
                Case @CR
                    $aReturn[$n][1] &= "r  "
                Case @TAB
                    $aReturn[$n][1] &= "t  "
;~                 Case ' '
;~                     $aReturn[$n][1] &= "h  "
                Case Else
                    If Asc($sChar) < 32 Then
                        $aReturn[$n][1] &= "c  "
                    Else
                        $aReturn[$n][1] &= " " & $sChar & "  "
                    EndIf
            EndSwitch
        Next
        For $i = 1 To StringLen($sString)
            $sChar = Asc(StringMid($sString, $i, 1))
            $aReturn[$n][2] &= Hex($sChar, 2) & "  "
            $aReturn[$n][3] &= StringFormat("%03d ", $sChar)
        Next
    Next
    Return SetError(0, 0, $aReturn)
EndFunc   ;==>_StringCharListSplit

Edited June 23, 2012 by Robjong