Got an interesting problem for you guys

[Sorian]

Hello guys,

It has been a while since I have been on. I still write an occasional script but this one new one has stumped me.

I have a co-worker that is having to take a text file containing number separated by commas:

1,1,1
2,2,2
3,3,3
4,4,4
5,5,5

And needs it to convert it to:

X1 Y1 Z1
X2 Y2 Z2
X3 Y3 Z3
X4 Y4 Z4
X5 Y5 Z5

Now, this is where it gets fun, the text file can contain anywhere upwards of 100,000 lines of numbers.

Any ideas where to start?

[UEZ]

What it is the size of the huge file? Can you provide more details about the output format?

Really only x,y,z appended as prefix?

Can you provide an example of the file?

But you can try this if file is not too huge:

$sHugefile = FileRead("huge.txt")
$sNew = StringRegExpReplace($sHugefile, "(\d+),(\d+),(\d+)", "X$1 Y$2 Z$3")
$hFile = FileOpen("new.txt", 2)
FileWrite($hFile, $sNew)
FileClose($hFile)

Br,

UEZ

Edited June 7, 2011 by UEZ

[SmOke_N]

Assuming that there are only 3 columns as your example provided:

Global $gs_fileToOpen = "SomeFile.txt"
Global $gs_fileToWrite = "SomeOutFile.txt"

Global $gh_fileOpen = FileOpen($gs_fileToOpen)
Global $gh_fileWriteTo = FileOpen($gs_fileToWrite, 2)

Global $gs_line, $gs_convert
While 1
    $gs_line = FileReadLine($gs_fileToOpen)
    $gs_convert = StringRegExpReplace($gs_line, "(\d+)(,)(\d+)(,)(\d+)", "X$1 Y$3 Z$5")
    FileWrite($gh_fileWriteTo, $gs_convert & @CRLF)
WEnd
FileClose($gh_fileOpen)
FileClose($gh_fileWriteTo)

Edit:

The above example was to make sure you didn't run into a memory issue with a large file.

Edited June 7, 2011 by SmOke_N

[Sorian]

UEZ,

It is a plain text file, only numbers and commas and 1 string of numbers per line.

So the file size can very on number of lines recorded.

Truly, it is taking the string of numbers and adding the prefix, removing the comma, and one space between sets of numbers like the example on first post.

pieeater,

The numbers are differ like this 9,5,4|5,7,4|6,3,5 but always 3 sets of numbers

Edited June 7, 2011 by Sorian

[pieeater]

i might be wrong but this is what i made:

$file = FileOpenDialog("txt file", @DesktopDir, "Text (*.txt)", 1)
If @error Then Exit
$savelocation = FileSaveDialog("Save location and file", @DesktopDir, "(*.txt)", 16)
If @error Then Exit
FileWrite($savelocation & ".txt", "")
For $i = 1 To 10
    $fileRead = FileReadLine($file, $i)
    $string = StringSplit($fileRead, ",")
    For $a = 1 To $string[0]
        $newValue = "X" & $string[$a] & " " & "Y" & $string[$a] & " " & "Z" & $string[$a]
        FileWriteLine($savelocation, $newValue)
    Next
Next
FileClose($file)
Exit

however if its anything longer than 3 numbers per comma per line it wont work

Edited June 7, 2011 by pieeater

[Sorian]

$sHugefile = FileRead("huge.txt")$sNew = StringRegExpReplace($sHugefile, "(\d+),(\d+),(\d+)", "X$1 Y$2 Z$3")$hFile = FileOpen("new.txt", 2)FileWrite($hFile, $sNew)FileClose($hFile)

Br,UEZ

That worked, thanks!

Assuming that there are only 3 columns as your example provided:

Global $gs_fileToOpen = "SomeFile.txt"Global $gs_fileToWrite = "SomeOutFile.txt"Global $gh_fileOpen = FileOpen($gs_fileToOpen)Global $gh_fileWriteTo = FileOpen($gs_fileToWrite, 2)Global $gs_line, $gs_convertWhile 1  $gs_line = FileReadLine($gs_fileToOpen) $gs_convert = StringRegExpReplace($gs_line, "(\d+)(,)(\d+)(,)(\d+)", "X$1 Y$3 Z$5") FileWrite($gh_fileWriteTo, $gs_convert & @CRLF)WEndFileClose($gh_fileOpen)FileClose($gh_fileWriteTo)

Edit:The above example was to make sure you didn't run into a memory issue with a large file.

Tried this, got the following error:

C:\Documents and Settings\user\Desktop\test2.au3(4,46) : ERROR: FileOpen() [built-in] called with wrong number of args.

Global $gh_fileOpen = FileOpen($gs_fileToOpen)

i might be wrong but this is what i made:

$file = FileOpenDialog("txt file", @DesktopDir, "Text (*.txt)", 1)If @error Then Exit$savelocation = FileSaveDialog("Save location and file", @DesktopDir, "(*.txt)", 16)If @error Then ExitFileWrite($savelocation & ".txt", "")For $i = 1 To 10   $fileRead = FileReadLine($file, $i) $string = StringSplit($fileRead, ",")   For $a = 1 To $string[0]        $newValue = "X" & $string[$a] & " " & "Y" & $string[$a] & " " & "Z" & $string[$a]       FileWriteLine($savelocation, $newValue) NextNextFileClose($file)Exit

however if its anything longer than 3 numbers per comma per line it wont work

Yours ran and gave options to find actual file and save it somewhere else (which I really like), but the output file is not readable, saved as .File.

When I did open the file as test, It gave x y z to each number. Also it gave me a second file as .txt with no data.

Original text:

1,3,5
2,4,6
3,6,9
4,8,12
5,10,15.221

End result:

X1,Y1,Z1
X3,Y3,Z3
X5,Y5,Z5
X2,Y2,Z2
X4,Y4,Z4
X6,Y6,Z6
X3,Y3,Z3
X6,Y6,Z6
X9,Y9,Z9
X4,Y4,Z4
X8,Y8,Z8
X12,Y12,Z12
X5,Y5,Z5
X10,Y10,Z10
X15.221,Y15.221,Z15.221
X,Y,Z
X,Y,Z
X,Y,Z
X,Y,Z
X,Y,Z

I am going to mess around with both SmOke_N's and pieeater's so I can learn more about doing this.

Edited June 7, 2011 by Sorian

[pieeater]

oops sorry i can fix that

ok here is my fixed code. problem with my script is that you need to know the exact amount of lines in the file.

$file = FileOpenDialog("txt file", @DesktopDir, "Text (*.txt)", 1)
If @error Then Exit
$savelocation = FileSaveDialog("Save location and file", @DesktopDir, "(*.txt)", 16)
If @error Then Exit
$lines = InputBox("how many lines in the txt file?", "line numbers here:") ;added a customizable range of number of loops
If $lines <= 0 Then Exit ;took out the file write that had no purpose
For $i = 1 To $lines
    $fileRead = FileReadLine($file, $i)
    $string = StringSplit($fileRead, ",")
    $newValue = "X" & $string[1] & "," & "Y" & $string[2] & "," & "Z" & $string[3] ;took out the 2nd for...to...next loop
    FileWriteLine($savelocation & ".txt", $newValue) ;forgot to add the .txt part, which caused the blank file extension
Next
FileClose($file)
Exit

Edit: showed what i changed

Edited June 7, 2011 by pieeater

[Sorian]

oops sorry i can fix that

ok here is my fixed code. problem with my script is that you need to know the exact amount of lines in the file.

$file = FileOpenDialog("txt file", @DesktopDir, "Text (*.txt)", 1)
If @error Then Exit
$savelocation = FileSaveDialog("Save location and file", @DesktopDir, "(*.txt)", 16)
If @error Then Exit
$lines = InputBox("how many lines in the txt file?", "line numbers here:") ;added a customizable range of number of loops
If $lines <= 0 Then Exit ;took out the file write that had no purpose
For $i = 1 To $lines
    $fileRead = FileReadLine($file, $i)
    $string = StringSplit($fileRead, ",")
    $newValue = "X" & $string[1] & "," & "Y" & $string[2] & "," & "Z" & $string[3] ;took out the 2nd for...to...next loop
    FileWriteLine($savelocation & ".txt", $newValue) ;forgot to add the .txt part, which caused the blank file extension
Next
FileClose($file)
Exit

Edit: showed what i changed

Thanks, this ran good. The only problem is the Number of lines can change from file to file, anywhere as 10 lines to 10,000 lines (max lines per file).

Edited June 7, 2011 by Sorian

[pieeater]

The only problem is the Number of lines can change from file to file, anywhere as 10 lines to 10,000 lines (max lines per file).

thats why i added the inputbox so you can tell it how many times to loop

but i just thought of somthing to make it more reliable so here

$file = FileOpenDialog("txt file", @DesktopDir, "Text (*.txt)", 1)
If @error Then Exit
$savelocation = FileSaveDialog("Save location and file", @DesktopDir, "(*.txt)", 16)
if not stringright($savelocation, 4) = ".txt" then $savelocation = $savelocation & ".txt"
If @error Then Exit
Do
    $lines = InputBox("how many lines in the txt file?", "line numbers here:")
    If @error = 1 Then Exit
Until $lines >= 10  And $lines <= 10000
For $i = 1 To $lines
    $fileRead = FileReadLine($file, $i)
    $string = StringSplit($fileRead, ",")
    $newValue = "X" & $string[1] & "," & "Y" & $string[2] & "," & "Z" & $string[3]
    FileWriteLine($savelocation, $newValue)
Next
FileClose($file)
Exit

Edit: fixed bug

Edited June 8, 2011 by pieeater

[Sorian]

Tried this, got the following error:

C:\Documents and Settings\user\Desktop\test2.au3(4,46) : ERROR: FileOpen() [built-in] called with wrong number of args.

Global $gh_fileOpen = FileOpen($gs_fileToOpen)

Ok, found first problem, was missing mode as 0.

Ran, then hung somewhere. Closed script output file with first line only duplicated 83 times :head scratch:

Edited June 7, 2011 by Sorian

[Malkey]

This is based on SmOke_N's example.

Global $gs_fileToOpen = "SomeFile.txt"
Global $gs_fileToWrite = "SomeOutFile.txt"

Global $gh_fileOpen = FileOpen($gs_fileToOpen) ; Read from file.
Global $gh_fileWriteTo = FileOpen($gs_fileToWrite, 10) ; Write to file.

Global $gs_line, $gs_convert, $iCount = 0

While 1
    $iCount += 1
    $gs_line = FileReadLine($gh_fileOpen)
    If @error = -1 Then ExitLoop
    If StringRegExp($gs_line, "[^0-9., ]") = 0 Then ;If character other than 0 to 9, coma, dot, or space then exit.
        $gs_convert = StringRegExpReplace($gs_line, "\h*([0-9.]+)\h*,\h*([0-9.]+)\h*,\h*([0-9.]+)\h*", "X$1 Y$2 Z$3")
        FileWrite($gh_fileWriteTo, $gs_convert & @CRLF)
    Else
        MsgBox(16, "Error", "   Aborted!" & @CRLF & @CRLF & _
                "Invalid character on line #" & $iCount & @CRLF & @CRLF & '"' & $gs_line & '"')
        ExitLoop
    EndIf
WEnd

FileClose($gh_fileOpen)
FileClose($gh_fileWriteTo)
;ShellExecute("SomeOutFile.txt")

Edit: As per BrewManNH next post #12. "$iCount" removed from FileReadLine().

Edited June 8, 2011 by Malkey

[BrewManNH]

@Malkey

I would change this line

$gs_line = FileReadLine($gh_fileOpen, $iCount)

to this:

$gs_line = FileReadLine($gh_fileOpen)

Otherwise the script will have to read from the start of the text file to the $iCount line number every time it reads a line. Without the line to read it defaults to reading the next line after the last line read. It can be a huge performance hit with large files to read it line by line over and over.

Edited June 8, 2011 by BrewManNH

[Malkey]

I now realize why

$gs_line = FileReadLine($gh_fileOpen)

does not work in SmOke_N's example post #3.

"$gs_line = FileReadLine($gs_fileToOpen)" is being used. Where "$gs_fileToOpen" is the path and file name of the file to be opened, and not the handle of the opened file.

So what is happening is each time through the loop, the file is opened and closed, and the file pointer is pointing at the first line.

I used the line number parameter because I thought the FileWrite() later in the loop was interfering with the file pointer. I was mistaken.

@BrewManNH

You are absolutely correct.

"$gs_line = FileReadLine($gh_fileOpen)" should be used in my example of post #11.

Edit: Note: My use of the term "file pointer", I mean the current file position within the file as set with the FileSetPos function.

Edited June 8, 2011 by Malkey

[Sorian]

This is based on SmOke_N's example.

Global $gs_fileToOpen = "SomeFile.txt"
Global $gs_fileToWrite = "SomeOutFile.txt"

Global $gh_fileOpen = FileOpen($gs_fileToOpen) ; Read from file.
Global $gh_fileWriteTo = FileOpen($gs_fileToWrite, 10) ; Write to file.

Global $gs_line, $gs_convert, $iCount = 0

While 1
    $iCount += 1
    $gs_line = FileReadLine($gh_fileOpen)
    If @error = -1 Then ExitLoop
    If StringRegExp($gs_line, "[^0-9., ]") = 0 Then ;If character other than 0 to 9, coma, dot, or space then exit.
        $gs_convert = StringRegExpReplace($gs_line, "\h*([0-9.]+)\h*,\h*([0-9.]+)\h*,\h*([0-9.]+)\h*", "X$1 Y$2 Z$3")
        FileWrite($gh_fileWriteTo, $gs_convert & @CRLF)
    Else
        MsgBox(16, "Error", "   Aborted!" & @CRLF & @CRLF & _
                "Invalid character on line #" & $iCount & @CRLF & @CRLF & '"' & $gs_line & '"')
        ExitLoop
    EndIf
WEnd

FileClose($gh_fileOpen)
FileClose($gh_fileWriteTo)
;ShellExecute("SomeOutFile.txt")

Edit: As per BrewManNH next post #12. "$iCount" removed from FileReadLine().

Thanks for posting your example for me to learn. Yours has the same problem as SmOke_N's example:

Global $gh_fileOpen = FileOpen($gs_fileToOpen) ; Read from file.

C:\Documents and Settings\user\Desktop\test2.au3(4,46) : ERROR: FileOpen() [built-in] called with wrong number of args.

Global $gh_fileOpen = FileOpen($gs_fileToOpen).

Should be this:

Global $gh_fileOpen = FileOpen($gs_fileToOpen, 0) ; Read from file. Missing mode number.

Fixing the error did ran with no other errors.

Edited June 8, 2011 by Sorian

[SmOke_N]

Eek, that's what I get for not testing the code. Sorry for the confusion.

So I assume this does in fact work:

Global $gs_fileToOpen = "SomeFile.txt"
Global $gs_fileToWrite = "SomeOutFile.txt"

Global $gh_fileOpen = FileOpen($gs_fileToOpen, 0)
Global $gh_fileWriteTo = FileOpen($gs_fileToWrite, 2)

Global $gs_line, $gs_convert
While 1
    $gs_line = FileReadLine($gs_fileOpen)
    If @error Then Exitloop
    $gs_convert = StringRegExpReplace($gs_line, "(\d+),(\d+),(\d+)", "X$1 Y$3 Z$5")
    FileWrite($gh_fileWriteTo, $gs_convert & @CRLF)
WEnd
FileClose($gh_fileOpen)
FileClose($gh_fileWriteTo)

Anyway, there are multiple solutions. Good luck.

Edited June 8, 2011 by SmOke_N

[Sorian]

thats why i added the inputbox so you can tell it how many times to loop

but i just thought of somthing to make it more reliable so here

$file = FileOpenDialog("txt file", @DesktopDir, "Text (*.txt)", 1)
If @error Then Exit
$savelocation = FileSaveDialog("Save location and file", @DesktopDir, "(*.txt)", 16)
If @error Then Exit
Do
    $lines = InputBox("how many lines in the txt file?", "line numbers here:")
    If @error = 1 Then Exit
Until $lines >= 10  And $lines <= 10000
For $i = 1 To $lines
    $fileRead = FileReadLine($file, $i)
    $string = StringSplit($fileRead, ",")
    $newValue = "X" & $string[1] & "," & "Y" & $string[2] & "," & "Z" & $string[3]
    FileWriteLine($savelocation & ".txt", $newValue)
Next
FileClose($file)
Exit

Found a small problem with this, it doesn't rewrite over the file if it exists, it ands another .txt to the name. So an example is if I output the file as done.txt the first run; second run is done.txt.txt; third run is done.txt.txt.txt

[Sorian]

Eek, that's what I get for not testing the code. Sorry for the confusion.

So I assume this does in fact work:

Global $gs_fileToOpen = "SomeFile.txt"
Global $gs_fileToWrite = "SomeOutFile.txt"

Global $gh_fileOpen = FileOpen($gs_fileToOpen, 0)
Global $gh_fileWriteTo = FileOpen($gs_fileToWrite, 2)

Global $gs_line, $gs_convert
While 1
    $gs_line = FileReadLine($gs_fileOpen)
    If @error Then Exitloop
    $gs_convert = StringRegExpReplace($gs_line, "(\d+),(\d+),(\d+)", "X$1 Y$3 Z$5")
    FileWrite($gh_fileWriteTo, $gs_convert & @CRLF)
WEnd
FileClose($gh_fileOpen)
FileClose($gh_fileWriteTo)

Anyway, there are multiple solutions. Good luck.

Actually, that brought up more errors.

C:\Documents and Settings\user\Desktop\test2.au3(9,41) : WARNING: $gs_fileOpen: possibly used before declaration.

$gs_line = FileReadLine($gs_fileOpen)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~^

C:\Documents and Settings\user\Desktop\test2.au3(15,11) : ERROR: syntax error

FileClose($gh_fileWriteTo

~~~~~~~~~~~~~~~~~~^

C:\Documents and Settings\user\Desktop\test2.au3(9,41) : ERROR: $gs_fileOpen: undeclared global variable.

$gs_line = FileReadLine($gs_fileOpen)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~^

C:\Documents and Settings\user\Desktop\test2.au3 - 2 error(s), 1 warning(s)

Edited June 8, 2011 by Sorian

[BrewManNH]

The Mode setting for FileOpen is now optional, it shouldn't have given you an error message like that. What version of AutoIT3 are you using because it was changed in version 3.3.6.0?

[Sorian]

The Mode setting for FileOpen is now optional, it shouldn't have given you an error message like that. What version of AutoIT3 are you using because it was changed in version 3.3.6.0?

I am actually using SciTE Version 1.79 to run these... I think I need to update some software.

[SmOke_N]

Awesome

At least the syntax errors tell you what to fix.

This time I did run Au3Check:

Global $gs_fileToOpen = "SomeFile.txt"
Global $gs_fileToWrite = "SomeOutFile.txt"

Global $gh_fileOpen = FileOpen($gs_fileToOpen, 0)
Global $gh_fileWriteTo = FileOpen($gs_fileToWrite, 2)

Global $gs_line, $gs_convert
While 1
    $gs_line = FileReadLine($gh_fileOpen)
    If @error Then Exitloop
    $gs_convert = StringRegExpReplace($gs_line, "(\d+),(\d+),(\d+)", "X$1 Y$3 Z$5")
    FileWrite($gh_fileWriteTo, $gs_convert & @CRLF)
WEnd
FileClose($gh_fileOpen)
FileClose($gh_fileWriteTo)

However I did not run it.

Edit:

BTW, I'm running the same version of SciTe; however I am using the 3.3.6.1 release version of AutoIt

Edited June 8, 2011 by SmOke_N