wisem2540

Slow Array Build. Can someone help speed this up?

16 posts in this topic

#1 ·  Posted (edited)

I know this is not efficient, but I do not know how to fix it.  The file should always have 11 columns, but the number of lines will change.  Currently the file is over 6,000 lines, and 813k in file size

My Current process is to read the file into a 1D aray, then split each CSV like into seperate elements.  It takes about 1min to complete, which is too long for the GUI I want to build.

#include <array.au3>
#include <file.au3>
Dim $oneDarray
_FileReadToArray("C:\file.csv", $oneDarray)


Local $Array[0][11]

For $x = 1 to $OneDArray[0]
   _ArrayAdd ($Array,$OneDArray[$x],0,",")
   ;MsgBox (0, "", @Error)
Next

_ArrayDisplay ($Array)
Edited by wisem2540

Share this post


Link to post
Share on other sites



#2 ·  Posted (edited)

wisem2540,

The _FileReadToArray function in the latest Beta versions will read your file directly into a 2D array. ;)

M23

Edited by Melba23
Wrote the wrong function name!

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

Why not try out:

#include <File.au3>
 
_FileReadToArray("C:\file.csv", $oneDarray, 1, ",")

? :)

Edited by MikahS

Snips & Scripts


My Snips: graphCPUTemp ~ getENVvars
My Scripts: Short-Order Encrypter - message and file encryption V1.6.1 ~ AuPad - Notepad written entirely in AutoIt V1.9.4

Feel free to use any of my code for your own use.                                                                                                                                                           Forum FAQ

 

Share this post


Link to post
Share on other sites

#4 ·  Posted (edited)

wisem2540,

The _FileListToArray function in the latest Beta versions will read your file directly into a 2D array. ;)

M23

That will only list files and folders in an array, not the contents of the file. :D

EDIT: Unless the beta version has this capability, if so disregard this. ;)

Edited by MikahS

Snips & Scripts


My Snips: graphCPUTemp ~ getENVvars
My Scripts: Short-Order Encrypter - message and file encryption V1.6.1 ~ AuPad - Notepad written entirely in AutoIt V1.9.4

Feel free to use any of my code for your own use.                                                                                                                                                           Forum FAQ

 

Share this post


Link to post
Share on other sites

MikahS,

Brain failure! :D

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

Why not try out:

#include <File.au3>
 
_FileReadToArray("C:\file.csv", $oneDarray, 1, ",")

? :)

 

I thought this would work too, but when I do an _Arraydisplay, nothing happened.

Share this post


Link to post
Share on other sites

wisem2540,

How about posting a copy of the file so we can test? :)

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

I am using autoit 3.3.12.  I will upgrade based on M23's suggestion and see how it goes....

Share this post


Link to post
Share on other sites

wisem2540,

How about posting a copy of the file so we can test? :)

M23

 

Sure, Ill need to take some data out of it first...  gimmie a sec

Share this post


Link to post
Share on other sites

I thought this would work too, but when I do an _Arraydisplay, nothing happened.

 

Try this:

_FileReadToArray("C:\file.csv", $oneDarray, 2, ",")

 

Brain failure! :D

 

I know I have my moments! yay, for old age! :)


Snips & Scripts


My Snips: graphCPUTemp ~ getENVvars
My Scripts: Short-Order Encrypter - message and file encryption V1.6.1 ~ AuPad - Notepad written entirely in AutoIt V1.9.4

Feel free to use any of my code for your own use.                                                                                                                                                           Forum FAQ

 

Share this post


Link to post
Share on other sites

I am using autoit 3.3.12.  I will upgrade based on M23's suggestion and see how it goes....

see post #5


Snips & Scripts


My Snips: graphCPUTemp ~ getENVvars
My Scripts: Short-Order Encrypter - message and file encryption V1.6.1 ~ AuPad - Notepad written entirely in AutoIt V1.9.4

Feel free to use any of my code for your own use.                                                                                                                                                           Forum FAQ

 

Share this post


Link to post
Share on other sites

MikahS,

The Beta _FileReadToArray does indeed have the required functionality - I just wrote the wrong function name. ;)

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

Mikah,

Your original solution worked...my Array was just the wrong size.  Thanks for the help.  I misinterpreted the meaning of the help file.  My mistake.

Share this post


Link to post
Share on other sites

MikahS,

The Beta _FileReadToArray does indeed have the required functionality - I just wrote the wrong function name. ;)

M23

Good deal, thank you for clarifying Melba. ^_^

 

Mikah,

Your original solution worked...my Array was just the wrong size.  Thanks for the help.  I misinterpreted the meaning of the help file.  My mistake.

 

Glad to have helped. ;)


Snips & Scripts


My Snips: graphCPUTemp ~ getENVvars
My Scripts: Short-Order Encrypter - message and file encryption V1.6.1 ~ AuPad - Notepad written entirely in AutoIt V1.9.4

Feel free to use any of my code for your own use.                                                                                                                                                           Forum FAQ

 

Share this post


Link to post
Share on other sites

Actually, after more testing, I had one weird characters in one of the columns.  When I edited the file to post for you guys, it masked the issue.  Anyway, still fixed

Share this post


Link to post
Share on other sites

Actually, after more testing, I had one weird characters in one of the columns.  When I edited the file to post for you guys, it masked the issue.  Anyway, still fixed

 

No problem, if you have anymore questions relating to reading the file contents into an array post here, otherwise you can start a new topic if you hit anymore snags. :D


Snips & Scripts


My Snips: graphCPUTemp ~ getENVvars
My Scripts: Short-Order Encrypter - message and file encryption V1.6.1 ~ AuPad - Notepad written entirely in AutoIt V1.9.4

Feel free to use any of my code for your own use.                                                                                                                                                           Forum FAQ

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now