tab delimited file + other delimeters > array

[gcue]

hello =)

i have this script that builds an array from a very dirty csv file (courtesy of psaltyds!)..

#include <array.au3>

$DELIM = @TAB; Change delimiter here
Global $avLines[1]

$sFile = "\\carnie_server\extract.txt"

    ; Read file in binary mode
    $hFile = FileOpen($sFile, 16)
    $binFileData = FileRead($hFile)
    FileClose($hFile)

    ; Change nulls
    $sFileData = BinaryToString($binFileData, 1)
    $sFileData = StringReplace($sFileData, Chr(0), "<null>")

    ; Split lines
    $avLines = StringSplit($sFileData, @CRLF, 1)

    Global $avOut[$avLines[0] + 1][2] = [[$avLines[0], ""]]; Start out with minimum two columns

    ; First loop - for each row
    For $n = 1 To $avLines[0]
        ; Split the line
        $avSplit = StringSplit($avLines[$n], $DELIM)

        ; Test if more columns are needed
        If $avSplit[0] > UBound($avOut, 2) Then ReDim $avOut[$avLines[0] + 1][$avSplit[0]]

        ; Second loop for each column
        For $x = 1 To $avSplit[0]
            $avOut[$n][$x - 1] = $avSplit[$x]
        Next
    Next

_ArrayDisplay($avOut)

problem is that it takes a while to build the array. this file is generated through a server batch job every night. then i run a scheduled script that copies it to a server where my script users access it from.

what i would like to do, is run the first part (where the formatting takes place - stringreplaces,stringsplits) then save the output to a new text file. so that the when my script users have to build the array they wont have to do that so itd be faster.

so the formatting script that creates the new text file would start off with something like this:

$DELIM = @TAB; Change delimiter here
Global $avLines[1]

$sFile = "\\carnie_server\extract.txt"

    ; Read file in binary mode
    $hFile = FileOpen($sFile, 16)
    $binFileData = FileRead($hFile)
    FileClose($hFile)

    ; Change nulls
    $sFileData = BinaryToString($binFileData, 1)
    $sFileData = StringReplace($sFileData, Chr(0), "<null>")

.......

but i get lossed with the stringsplitting part...

then in the end what does the script look like to build the array from the new text file?

ive also included a small exerpt of the text file.

Edited August 10, 2009 by gcue

[enaiman]

No offense but IMO, you're chasing your own tail here.

Let's get this clear: you want to get the info from a csv file (where the info is separated by commas) and put the same info in a text file (where the info is separated by TABs) and think that this will speed up the process ...

Wouldn't you end up in the same situation as before?? but with a different file format?

I would understand this if some info from the csv file is filtered so the amount of info in the new file is reduced.

Unfortunately I don't have a sample of your csv file and I don't have any idea about what do you want to achieve in the end.

Anyway, good luck,

[gcue]

oops sorry the first file is tab delimited as well (changed title to avoid confusion-thanks!)

the original file is a tab delimited file. but it also has other characters like @CRLFs that need to be stripped out of it to make it ONLY a TAB delimited..

so i want to do the clean up and save the text file as a new text file with the changes..

THEN a seperate process to build the array.. right now the script is doing both processes. just trying to seperate the two.

hope this makes sense

[WolfWorld]

See

StringReplace({Your file input}, @CRLF, @TAB))

I don't know how this will work but there is the solution.

[gcue]

yep i know about stringreplace... but did u see the first script i noted? its a bit more complicated than a one liner.. =)

[enaiman]

In order to have this done properly you need first to understand what are the script steps.

Your original file, being TAB separated will have a format like this:

123 @TAB qwe @TAB 12345 @TAB qwerty @CRLF

a23 @TAB awe @TAB a2345 @TAB awerty @CRLF

...

So, you have values delimited by TAB and lines delimited by @CRLF

First: you need to do a StringSplit by @CRLF and the result will be a 1 dimensional array containing the lines.

$lines[0] = 2 $lines[1] = "123 @TAB qwe @TAB 12345 @TAB qwerty @CRLF" and $lines[2] = "a23 @TAB awe @TAB a2345 @TAB awerty @CRLF"

Second: every element of the first array (lines) will be StringSplit by @TAB and the result will be a 1 dimensional array of values present in a line

$elements[0] = 4 $elements[1] = "123" $elements[2] = "qwe" $elements[3] = "12345" $elements[4] = "qwerty"

...

In order to not "over-complicate" things, you will need to do the "Second" step right after the first and the result to be put in your "final" array.

If you want a text file saved from your original file, you will need to write the array obtained at the first step ($lines) in that text file and you're done with.

[gcue]

so the first part is like this?

$DELIM = @TAB; Change delimiter here
Global $avLines[1]

$sFile = "\\carnie_server\extract.txt"

    ; Read file in binary mode
    $hFile = FileOpen($sFile, 16)
    $binFileData = FileRead($hFile)
    FileClose($hFile)

    ; Change nulls
    $sFileData = BinaryToString($binFileData, 1)
    $sFileData = StringReplace($sFileData, Chr(0), "<null>")

[enaiman]

The code you've posted is ok.

If you get right the next line of code it means that you read my post and tried to understand it - if that's the case I will help you further.

If you get it wrong ... I'll be out of this thread.

Good luck,

[gcue]

wow i feel like im in one of those game shows get it right or die!

$sFileData = stringsplit($sfiledata, @crlf) ???

this is where im confused... $sfiledata[1] how can that contain every record in the text file?

[Authenticity]

@CRLF are two joined characters, not one. StringSplit() has another parameter, the third, to specify whether it should split the content on delimiter as a whole. Right now, it seems like it'll split on @CR, then split on @LF, each and every line. Read the function description, it's very descriptive with a few examples. >_

Edited August 11, 2009 by Authenticity

[enaiman]

wow i feel like im in one of those game shows get it right or die!

$sFileData = stringsplit($sfiledata, @crlf) ???

this is where im confused... $sfiledata[1] how can that contain every record in the text file?

It's correct

Nothing personal but I hate when I try to help people and they don't even try to understand what I'm telling them.

As Authenticity said, look at StringSplit function and use the correct parameter with more than 1 character. After that, you're on the good way >_<