Sign in to follow this  
Followers 0
Sorian

Got an interesting problem for you guys

29 posts in this topic

Hello guys,

It has been a while since I have been on. I still write an occasional script but this one new one has stumped me.

I have a co-worker that is having to take a text file containing number separated by commas:

1,1,1
2,2,2
3,3,3
4,4,4
5,5,5

And needs it to convert it to:

X1 Y1 Z1
X2 Y2 Z2
X3 Y3 Z3
X4 Y4 Z4
X5 Y5 Z5

Now, this is where it gets fun, the text file can contain anywhere upwards of 100,000 lines of numbers.

Any ideas where to start?

Share this post


Link to post
Share on other sites



#2 ·  Posted (edited)

What it is the size of the huge file? Can you provide more details about the output format?

Really only x,y,z appended as prefix?

Can you provide an example of the file?

But you can try this if file is not too huge:

$sHugefile = FileRead("huge.txt")
$sNew = StringRegExpReplace($sHugefile, "(\d+),(\d+),(\d+)", "X$1 Y$2 Z$3")
$hFile = FileOpen("new.txt", 2)
FileWrite($hFile, $sNew)
FileClose($hFile)

Br,

UEZ

Edited by UEZ

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

Assuming that there are only 3 columns as your example provided:

Global $gs_fileToOpen = "SomeFile.txt"
Global $gs_fileToWrite = "SomeOutFile.txt"

Global $gh_fileOpen = FileOpen($gs_fileToOpen)
Global $gh_fileWriteTo = FileOpen($gs_fileToWrite, 2)

Global $gs_line, $gs_convert
While 1
    $gs_line = FileReadLine($gs_fileToOpen)
    $gs_convert = StringRegExpReplace($gs_line, "(\d+)(,)(\d+)(,)(\d+)", "X$1 Y$3 Z$5")
    FileWrite($gh_fileWriteTo, $gs_convert & @CRLF)
WEnd
FileClose($gh_fileOpen)
FileClose($gh_fileWriteTo)

Edit:

The above example was to make sure you didn't run into a memory issue with a large file.

Edited by SmOke_N

Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.

Share this post


Link to post
Share on other sites

#4 ·  Posted (edited)

UEZ,

It is a plain text file, only numbers and commas and 1 string of numbers per line.

So the file size can very on number of lines recorded.

Truly, it is taking the string of numbers and adding the prefix, removing the comma, and one space between sets of numbers like the example on first post.

pieeater,

The numbers are differ like this 9,5,4|5,7,4|6,3,5 but always 3 sets of numbers

Edited by Sorian

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

i might be wrong but this is what i made:

$file = FileOpenDialog("txt file", @DesktopDir, "Text (*.txt)", 1)
If @error Then Exit
$savelocation = FileSaveDialog("Save location and file", @DesktopDir, "(*.txt)", 16)
If @error Then Exit
FileWrite($savelocation & ".txt", "")
For $i = 1 To 10
    $fileRead = FileReadLine($file, $i)
    $string = StringSplit($fileRead, ",")
    For $a = 1 To $string[0]
        $newValue = "X" & $string[$a] & " " & "Y" & $string[$a] & " " & "Z" & $string[$a]
        FileWriteLine($savelocation, $newValue)
    Next
Next
FileClose($file)
Exit

however if its anything longer than 3 numbers per comma per line it wont work

Edited by pieeater

[spoiler]My UDFs: Login UDF[/spoiler]

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

$sHugefile = FileRead("huge.txt")$sNew = StringRegExpReplace($sHugefile, "(\d+),(\d+),(\d+)", "X$1 Y$2 Z$3")$hFile = FileOpen("new.txt", 2)FileWrite($hFile, $sNew)FileClose($hFile)
Br,UEZ

That worked, thanks!

Assuming that there are only 3 columns as your example provided:

Global $gs_fileToOpen = "SomeFile.txt"Global $gs_fileToWrite = "SomeOutFile.txt"Global $gh_fileOpen = FileOpen($gs_fileToOpen)Global $gh_fileWriteTo = FileOpen($gs_fileToWrite, 2)Global $gs_line, $gs_convertWhile 1  $gs_line = FileReadLine($gs_fileToOpen) $gs_convert = StringRegExpReplace($gs_line, "(\d+)(,)(\d+)(,)(\d+)", "X$1 Y$3 Z$5") FileWrite($gh_fileWriteTo, $gs_convert & @CRLF)WEndFileClose($gh_fileOpen)FileClose($gh_fileWriteTo)
Edit:The above example was to make sure you didn't run into a memory issue with a large file.

Tried this, got the following error:

C:\Documents and Settings\user\Desktop\test2.au3(4,46) : ERROR: FileOpen() [built-in] called with wrong number of args.

Global $gh_fileOpen = FileOpen($gs_fileToOpen)

i might be wrong but this is what i made:

$file = FileOpenDialog("txt file", @DesktopDir, "Text (*.txt)", 1)If @error Then Exit$savelocation = FileSaveDialog("Save location and file", @DesktopDir, "(*.txt)", 16)If @error Then ExitFileWrite($savelocation & ".txt", "")For $i = 1 To 10   $fileRead = FileReadLine($file, $i) $string = StringSplit($fileRead, ",")   For $a = 1 To $string[0]        $newValue = "X" & $string[$a] & " " & "Y" & $string[$a] & " " & "Z" & $string[$a]       FileWriteLine($savelocation, $newValue) NextNextFileClose($file)Exit
however if its anything longer than 3 numbers per comma per line it wont work

Yours ran and gave options to find actual file and save it somewhere else (which I really like), but the output file is not readable, saved as .File.

When I did open the file as test, It gave x y z to each number. Also it gave me a second file as .txt with no data.

Original text:

1,3,5
2,4,6
3,6,9
4,8,12
5,10,15.221

End result:

X1,Y1,Z1
X3,Y3,Z3
X5,Y5,Z5
X2,Y2,Z2
X4,Y4,Z4
X6,Y6,Z6
X3,Y3,Z3
X6,Y6,Z6
X9,Y9,Z9
X4,Y4,Z4
X8,Y8,Z8
X12,Y12,Z12
X5,Y5,Z5
X10,Y10,Z10
X15.221,Y15.221,Z15.221
X,Y,Z
X,Y,Z
X,Y,Z
X,Y,Z
X,Y,Z

I am going to mess around with both SmOke_N's and pieeater's so I can learn more about doing this.

Edited by Sorian

Share this post


Link to post
Share on other sites

#7 ·  Posted (edited)

oops sorry i can fix that :huh2:

ok here is my fixed code. problem with my script is that you need to know the exact amount of lines in the file.

$file = FileOpenDialog("txt file", @DesktopDir, "Text (*.txt)", 1)
If @error Then Exit
$savelocation = FileSaveDialog("Save location and file", @DesktopDir, "(*.txt)", 16)
If @error Then Exit
$lines = InputBox("how many lines in the txt file?", "line numbers here:") ;added a customizable range of number of loops
If $lines <= 0 Then Exit ;took out the file write that had no purpose
For $i = 1 To $lines
    $fileRead = FileReadLine($file, $i)
    $string = StringSplit($fileRead, ",")
    $newValue = "X" & $string[1] & "," & "Y" & $string[2] & "," & "Z" & $string[3] ;took out the 2nd for...to...next loop
    FileWriteLine($savelocation & ".txt", $newValue) ;forgot to add the .txt part, which caused the blank file extension
Next
FileClose($file)
Exit

Edit: showed what i changed

Edited by pieeater

[spoiler]My UDFs: Login UDF[/spoiler]

Share this post


Link to post
Share on other sites

#8 ·  Posted (edited)

oops sorry i can fix that :huh2:

ok here is my fixed code. problem with my script is that you need to know the exact amount of lines in the file.

$file = FileOpenDialog("txt file", @DesktopDir, "Text (*.txt)", 1)
If @error Then Exit
$savelocation = FileSaveDialog("Save location and file", @DesktopDir, "(*.txt)", 16)
If @error Then Exit
$lines = InputBox("how many lines in the txt file?", "line numbers here:") ;added a customizable range of number of loops
If $lines <= 0 Then Exit ;took out the file write that had no purpose
For $i = 1 To $lines
    $fileRead = FileReadLine($file, $i)
    $string = StringSplit($fileRead, ",")
    $newValue = "X" & $string[1] & "," & "Y" & $string[2] & "," & "Z" & $string[3] ;took out the 2nd for...to...next loop
    FileWriteLine($savelocation & ".txt", $newValue) ;forgot to add the .txt part, which caused the blank file extension
Next
FileClose($file)
Exit

Edit: showed what i changed

Thanks, this ran good. The only problem is the Number of lines can change from file to file, anywhere as 10 lines to 10,000 lines (max lines per file).

Edited by Sorian

Share this post


Link to post
Share on other sites

#9 ·  Posted (edited)

The only problem is the Number of lines can change from file to file, anywhere as 10 lines to 10,000 lines (max lines per file).

thats why i added the inputbox so you can tell it how many times to loop ;)

but i just thought of somthing to make it more reliable so here :huh2:

$file = FileOpenDialog("txt file", @DesktopDir, "Text (*.txt)", 1)
If @error Then Exit
$savelocation = FileSaveDialog("Save location and file", @DesktopDir, "(*.txt)", 16)
if not stringright($savelocation, 4) = ".txt" then $savelocation = $savelocation & ".txt"
If @error Then Exit
Do
    $lines = InputBox("how many lines in the txt file?", "line numbers here:")
    If @error = 1 Then Exit
Until $lines >= 10  And $lines <= 10000
For $i = 1 To $lines
    $fileRead = FileReadLine($file, $i)
    $string = StringSplit($fileRead, ",")
    $newValue = "X" & $string[1] & "," & "Y" & $string[2] & "," & "Z" & $string[3]
    FileWriteLine($savelocation, $newValue)
Next
FileClose($file)
Exit

Edit: fixed bug

Edited by pieeater

[spoiler]My UDFs: Login UDF[/spoiler]

Share this post


Link to post
Share on other sites

#10 ·  Posted (edited)

Tried this, got the following error:

C:\Documents and Settings\user\Desktop\test2.au3(4,46) : ERROR: FileOpen() [built-in] called with wrong number of args.

Global $gh_fileOpen = FileOpen($gs_fileToOpen)

Ok, found first problem, was missing mode as 0.

Ran, then hung somewhere. Closed script output file with first line only duplicated 83 times :head scratch:

Edited by Sorian

Share this post


Link to post
Share on other sites

#11 ·  Posted (edited)

This is based on SmOke_N's example.

Global $gs_fileToOpen = "SomeFile.txt"
Global $gs_fileToWrite = "SomeOutFile.txt"

Global $gh_fileOpen = FileOpen($gs_fileToOpen) ; Read from file.
Global $gh_fileWriteTo = FileOpen($gs_fileToWrite, 10) ; Write to file.

Global $gs_line, $gs_convert, $iCount = 0

While 1
    $iCount += 1
    $gs_line = FileReadLine($gh_fileOpen)
    If @error = -1 Then ExitLoop
    If StringRegExp($gs_line, "[^0-9., ]") = 0 Then ;If character other than 0 to 9, coma, dot, or space then exit.
        $gs_convert = StringRegExpReplace($gs_line, "\h*([0-9.]+)\h*,\h*([0-9.]+)\h*,\h*([0-9.]+)\h*", "X$1 Y$2 Z$3")
        FileWrite($gh_fileWriteTo, $gs_convert & @CRLF)
    Else
        MsgBox(16, "Error", "   Aborted!" & @CRLF & @CRLF & _
                "Invalid character on line #" & $iCount & @CRLF & @CRLF & '"' & $gs_line & '"')
        ExitLoop
    EndIf
WEnd

FileClose($gh_fileOpen)
FileClose($gh_fileWriteTo)
;ShellExecute("SomeOutFile.txt")

Edit: As per BrewManNH next post #12. "$iCount" removed from FileReadLine().

Edited by Malkey

Share this post


Link to post
Share on other sites

#12 ·  Posted (edited)

@Malkey

I would change this line

$gs_line = FileReadLine($gh_fileOpen, $iCount)

to this:

$gs_line = FileReadLine($gh_fileOpen)

Otherwise the script will have to read from the start of the text file to the $iCount line number every time it reads a line. Without the line to read it defaults to reading the next line after the last line read. It can be a huge performance hit with large files to read it line by line over and over.

Edited by BrewManNH

If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

#13 ·  Posted (edited)

I now realize why

$gs_line = FileReadLine($gh_fileOpen)

does not work in SmOke_N's example post #3.

"$gs_line = FileReadLine($gs_fileToOpen)" is being used. Where "$gs_fileToOpen" is the path and file name of the file to be opened, and not the handle of the opened file.

So what is happening is each time through the loop, the file is opened and closed, and the file pointer is pointing at the first line.

I used the line number parameter because I thought the FileWrite() later in the loop was interfering with the file pointer. I was mistaken.

@BrewManNH

You are absolutely correct.

"$gs_line = FileReadLine($gh_fileOpen)" should be used in my example of post #11.

Edit: Note: My use of the term "file pointer", I mean the current file position within the file as set with the FileSetPos function.

Edited by Malkey

Share this post


Link to post
Share on other sites

#14 ·  Posted (edited)

This is based on SmOke_N's example.

Global $gs_fileToOpen = "SomeFile.txt"
Global $gs_fileToWrite = "SomeOutFile.txt"

Global $gh_fileOpen = FileOpen($gs_fileToOpen) ; Read from file.
Global $gh_fileWriteTo = FileOpen($gs_fileToWrite, 10) ; Write to file.

Global $gs_line, $gs_convert, $iCount = 0

While 1
    $iCount += 1
    $gs_line = FileReadLine($gh_fileOpen)
    If @error = -1 Then ExitLoop
    If StringRegExp($gs_line, "[^0-9., ]") = 0 Then ;If character other than 0 to 9, coma, dot, or space then exit.
        $gs_convert = StringRegExpReplace($gs_line, "\h*([0-9.]+)\h*,\h*([0-9.]+)\h*,\h*([0-9.]+)\h*", "X$1 Y$2 Z$3")
        FileWrite($gh_fileWriteTo, $gs_convert & @CRLF)
    Else
        MsgBox(16, "Error", "   Aborted!" & @CRLF & @CRLF & _
                "Invalid character on line #" & $iCount & @CRLF & @CRLF & '"' & $gs_line & '"')
        ExitLoop
    EndIf
WEnd

FileClose($gh_fileOpen)
FileClose($gh_fileWriteTo)
;ShellExecute("SomeOutFile.txt")

Edit: As per BrewManNH next post #12. "$iCount" removed from FileReadLine().

Thanks for posting your example for me to learn. Yours has the same problem as SmOke_N's example:

Global $gh_fileOpen = FileOpen($gs_fileToOpen) ; Read from file.

C:\Documents and Settings\user\Desktop\test2.au3(4,46) : ERROR: FileOpen() [built-in] called with wrong number of args.

Global $gh_fileOpen = FileOpen($gs_fileToOpen).

Should be this:

Global $gh_fileOpen = FileOpen($gs_fileToOpen, 0) ; Read from file. Missing mode number.

Fixing the error did ran with no other errors.

Edited by Sorian

Share this post


Link to post
Share on other sites

#15 ·  Posted (edited)

Eek, that's what I get for not testing the code. Sorry for the confusion.

So I assume this does in fact work:

Global $gs_fileToOpen = "SomeFile.txt"
Global $gs_fileToWrite = "SomeOutFile.txt"

Global $gh_fileOpen = FileOpen($gs_fileToOpen, 0)
Global $gh_fileWriteTo = FileOpen($gs_fileToWrite, 2)

Global $gs_line, $gs_convert
While 1
    $gs_line = FileReadLine($gs_fileOpen)
    If @error Then Exitloop
    $gs_convert = StringRegExpReplace($gs_line, "(\d+),(\d+),(\d+)", "X$1 Y$3 Z$5")
    FileWrite($gh_fileWriteTo, $gs_convert & @CRLF)
WEnd
FileClose($gh_fileOpen)
FileClose($gh_fileWriteTo)

Anyway, there are multiple solutions. Good luck.

Edited by SmOke_N

Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.

Share this post


Link to post
Share on other sites

thats why i added the inputbox so you can tell it how many times to loop ;)

but i just thought of somthing to make it more reliable so here :huh2:

$file = FileOpenDialog("txt file", @DesktopDir, "Text (*.txt)", 1)
If @error Then Exit
$savelocation = FileSaveDialog("Save location and file", @DesktopDir, "(*.txt)", 16)
If @error Then Exit
Do
    $lines = InputBox("how many lines in the txt file?", "line numbers here:")
    If @error = 1 Then Exit
Until $lines >= 10  And $lines <= 10000
For $i = 1 To $lines
    $fileRead = FileReadLine($file, $i)
    $string = StringSplit($fileRead, ",")
    $newValue = "X" & $string[1] & "," & "Y" & $string[2] & "," & "Z" & $string[3]
    FileWriteLine($savelocation & ".txt", $newValue)
Next
FileClose($file)
Exit

Found a small problem with this, it doesn't rewrite over the file if it exists, it ands another .txt to the name. So an example is if I output the file as done.txt the first run; second run is done.txt.txt; third run is done.txt.txt.txt

Share this post


Link to post
Share on other sites

#17 ·  Posted (edited)

Eek, that's what I get for not testing the code. Sorry for the confusion.

So I assume this does in fact work:

Global $gs_fileToOpen = "SomeFile.txt"
Global $gs_fileToWrite = "SomeOutFile.txt"

Global $gh_fileOpen = FileOpen($gs_fileToOpen, 0)
Global $gh_fileWriteTo = FileOpen($gs_fileToWrite, 2)

Global $gs_line, $gs_convert
While 1
    $gs_line = FileReadLine($gs_fileOpen)
    If @error Then Exitloop
    $gs_convert = StringRegExpReplace($gs_line, "(\d+),(\d+),(\d+)", "X$1 Y$3 Z$5")
    FileWrite($gh_fileWriteTo, $gs_convert & @CRLF)
WEnd
FileClose($gh_fileOpen)
FileClose($gh_fileWriteTo)

Anyway, there are multiple solutions. Good luck.

Actually, that brought up more errors.

C:\Documents and Settings\user\Desktop\test2.au3(9,41) : WARNING: $gs_fileOpen: possibly used before declaration.

$gs_line = FileReadLine($gs_fileOpen)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~^

C:\Documents and Settings\user\Desktop\test2.au3(15,11) : ERROR: syntax error

FileClose($gh_fileWriteTo

~~~~~~~~~~~~~~~~~~^

C:\Documents and Settings\user\Desktop\test2.au3(9,41) : ERROR: $gs_fileOpen: undeclared global variable.

$gs_line = FileReadLine($gs_fileOpen)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~^

C:\Documents and Settings\user\Desktop\test2.au3 - 2 error(s), 1 warning(s)

Edited by Sorian

Share this post


Link to post
Share on other sites

The Mode setting for FileOpen is now optional, it shouldn't have given you an error message like that. What version of AutoIT3 are you using because it was changed in version 3.3.6.0?


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

The Mode setting for FileOpen is now optional, it shouldn't have given you an error message like that. What version of AutoIT3 are you using because it was changed in version 3.3.6.0?

I am actually using SciTE Version 1.79 to run these... I think I need to update some software.

Share this post


Link to post
Share on other sites

#20 ·  Posted (edited)

Awesome :huh2:

At least the syntax errors tell you what to fix.

This time I did run Au3Check:

Global $gs_fileToOpen = "SomeFile.txt"
Global $gs_fileToWrite = "SomeOutFile.txt"

Global $gh_fileOpen = FileOpen($gs_fileToOpen, 0)
Global $gh_fileWriteTo = FileOpen($gs_fileToWrite, 2)

Global $gs_line, $gs_convert
While 1
    $gs_line = FileReadLine($gh_fileOpen)
    If @error Then Exitloop
    $gs_convert = StringRegExpReplace($gs_line, "(\d+),(\d+),(\d+)", "X$1 Y$3 Z$5")
    FileWrite($gh_fileWriteTo, $gs_convert & @CRLF)
WEnd
FileClose($gh_fileOpen)
FileClose($gh_fileWriteTo)
However I did not run it.

Edit:

BTW, I'm running the same version of SciTe; however I am using the 3.3.6.1 release version of AutoIt

Edited by SmOke_N

Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0