Jump to content
civilcalc

1d array -> 2d array

Recommended Posts

civilcalc

ok so I have an array...

[0] somedata1 somedata2 somedata3

[1] moredata1 moredata2 moredata3

[2] ......... ......... .........

each seperated by " " (ws)

Can I easily convert this to 2d array?

first dimension varies a lot, second dim is never bigger than 6.

Share this post


Link to post
Share on other sites
Melba23

civilcalc,

This should do it for you: :blink:

#include <Array.au3>

Global $aArray_1D[2] = ["somedata1 somedata2 somedata3", "moredata1 moredata2 moredata3"]

Global $aArray_2D[2][3] ; Set these dimensions as required

For $i = 0 To UBound($aArray_1D) - 1
    $aTemp = StringSplit($aArray_1D[$i], " ") ; Split the 1D elements assuming the delimiter is always a space
    For $j = 0 To UBound($aArray_2D, 2) - 1
        $aArray_2D[$i][$j] = $aTemp[$j + 1] ; Get the elements into the 2D array
    Next
Next

_ArrayDisplay($aArray_2D)

M23


Public_Domain.png.2d871819fcb9957cf44f4514551a2935.png Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind

Open spoiler to see my UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites
Funtime60

I know this is old, but is this the fastest possible way?

I came up with something similar that doesn't account for varying number of items in each row. My problem is that it takes well over 6 hours  to run through 230,400 lines at 4MB of an array with each line containing 3 sections. The only thing that I can see slowing it down more than yours is the ProgressSet that I have in the loop.

If anyone reads this I have included my work below which is at the start of a larger project. Sorry for the mess that it likely is, I started this months ago and have be working on it whenever I have a mental break through.

#Region Include
#include <ArrayMultiColSort.au3>    ;Non-standard .au3 added to my Include folder, not applicable to this section
#include <String.au3>
#include <BMP3.au3>                 ;Non-standard .au3 added to my Include folder, not applicable to this section
#EndRegion


#Region Open
Local $testpath = FileOpenDialog("Select A Data File",@ScriptDir & "\","All (*.*)")
Local $testpathpart = StringSplit($testpath, "\")
Local $testfilenamefull = $testpathpart[$testpathpart[0]]
Local $testfilenamepart = StringSplit($testfilenamefull, ".")
Local $testfilename = $testfilenamepart[1]
Local $test = FileOpen($testpath)
Local $testfile = FileReadToArray($test)
#EndRegion


#Region Main Init
Local $aSortData[][] = [[1, 0], [2, 0]]
Local $progress = "N/A"
ProgressOn("Conversion", "Part 1 of 9", "Progress: " & $progress, -1, -1, 16)
#EndRegion


#Region Pack Crct
_ArrayDisplay($testfile, "$testfile")
Local $progress = 0
    ProgressSet($progress, "Progress: " & Round($progress, 2) & "%", "Part 2 of 9")
Local $testsize = UBound($testfile)-1
$testfilerowdata = StringSplit($testfile[0],",")
Local $testfilearray1[1][3] = [[$testfilerowdata[1], $testfilerowdata[2], $testfilerowdata[3]]]
For $testfilerownum = 1 To $testsize Step 1
    $testfilerowdata = StringSplit($testfile[$testfilerownum],",")
    Local $testfilearray2[1][3] = [[$testfilerowdata[1], $testfilerowdata[2], $testfilerowdata[3]]]
    _ArrayConcatenate($testfilearray1,$testfilearray2)
    Local $progress = ((($testfilerownum+1)/($testsize+2))*100)
    ProgressSet($progress, "Progress: " & Round($progress, 2) & "%", "Part 2 of 9")
Next
_ArrayDisplay($testfilearray1, "$testfilearray1")
Local $progress = 100
ProgressSet($progress, "Progress: " & Round($progress, 2) & "%", "Part 2 of 9")
#EndRegion

It should turn this:

15791353,419,294
15791353,420,294
15791353,421,294
15791353,422,294
15791353,423,294
15791353,419,295
15791353,420,295
15791353,421,295
15791353,422,295
15791353,423,295
15791353,419,296
15791353,420,296
15791353,421,296
15791353,422,296
15791353,423,296
15791353,419,297
15791353,420,297
15791353,421,297
15791353,422,297
15791353,423,297
15791353,419,298
15791353,420,298
15791353,421,298
15791353,422,298
15791353,423,298

Into this:  The "|" delimit a column for anyone who hasn't pasted out of _ArrayDisplay before

15791353|419|294
15791353|420|294
15791353|421|294
15791353|422|294
15791353|423|294
15791353|419|295
15791353|420|295
15791353|421|295
15791353|422|295
15791353|423|295
15791353|419|296
15791353|420|296
15791353|421|296
15791353|422|296
15791353|423|296
15791353|419|297
15791353|420|297
15791353|421|297
15791353|422|297
15791353|423|297
15791353|419|298
15791353|420|298
15791353|421|298
15791353|422|298
15791353|423|298

Test File that generated these is below

Thank you for your time. even if you just read this.

TiniTmp.csv

Share this post


Link to post
Share on other sites
czardas

I suggest you use @Melba23's example but I had to test an idea. This automatically resizes the array (on the fly) as required, so you don't need to know the size of the output array in advance. To run the code you need ArrayWorkshop.au3 (found in my signature below). Such code is total overkill for a simple conversion like this (with only two dimensions), but an example of this nature lends itself to testing my own functions and may interest some readers.
 

#include <Array.au3>
#include <StringConstants.au3>
#include <ArrayWorkshop.au3>

; The Array
Local $aTestArray[7] = [ _
"Data Data Data Data", _
"", _
"Data Data Data Data Data",  _
"Data Data Data Data Data Data", _
"Data", _
"Data Data Data Data Data Data Data Data Data Data Data Data", _
"Data Data Data Data Data Data Data Data"]
_ArrayDisplay($aTestArray, "1D Array")

; The Code
Local $aOutputArray = StringSplit($aTestArray[0], " ", $STR_NOCOUNT)
For $i = 1 To UBound($aTestArray) -1
    _ArrayAttach($aOutputArray, StringSplit($aTestArray[$i], " ", $STR_NOCOUNT), 2) ; Add columns
Next

_ArrayTransform($aOutputArray) ; Transform: Rows => Columns

; Display output
_ArrayDisplay($aOutputArray, "2D Array")

 

Share this post


Link to post
Share on other sites
Funtime60

I appreciate the help, however in my inexperienced view it seems you are doing nearly the same thing as me except that your method of connecting the stringsplit pieces togetther causes a 90 rotation of the data which you then correct with another function. I did limited testing, but have found that when converting 128,417 characters on 8160 lines in a .csv file  your code takes ~0.4 seconds longer at ~36.5 seconds which is the best speed difference. This file size is at the smaller end of potential files with mid size files being 1-4 MB which can take multiple hours so your 1% slower, while probably much better written is too much of a cost. Still thank you for your time and effort. I hope this scenario was a good test of your code.

With that said, I'm looking for anything that can beet ~36.1 seconds with this file below. If you think you know how to do it feel free to test it yourself or send it to me.

tmp4.csv

  • Like 1

Share this post


Link to post
Share on other sites
mikell
Posted (edited)

Melba's code applied to your tmp4.csv takes 0.1 second. I don't understand the huge delay you mentioned

 

Edited by mikell

Share this post


Link to post
Share on other sites
iamtheky

0.8 seconds with array UDF, and thats about as slow as it gets.

#include<array.au3>
$s = stringtrimright(fileread("Tmp4.csv") , 2)

local $a[0][3]

_ArrayAdd($a , $s , "," , ",")

_ArrayDisplay($a)

 


,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites
Funtime60

well thanks, not sure why I didn't think to test melba's code or why his is so fast.

Share this post


Link to post
Share on other sites
Funtime60
Posted (edited)

I have failed to understand how the heck your code works iamtheky, I would be eternally grateful if you could offer any insight.

Never mind, I forgot that you were the author of a UDF that I already had so it looked like you pulled off some black magic. You have saved my code!

Edited by Funtime60
I solved my own problem

Share this post


Link to post
Share on other sites
jchd

All the work is done by _ArrayAdd(). Reading the help file will bring light in the darkness.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites
Funtime60

Turns out as I expected, after I fixed this 12 hours became 1.2 minutes. out of 9 sections numbers 2 and 3 were the longest with #2 being hours.

Share this post


Link to post
Share on other sites
iamtheky
1 hour ago, Funtime60 said:

I have failed to understand how the heck your code works iamtheky

i have the same issue, it's all a blur.  Glad you found myriad ways to skin your cat.


,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites
czardas
Posted (edited)
14 hours ago, Funtime60 said:

I appreciate the help, however in my inexperienced view it seems you are doing nearly the same thing as me except that your method of connecting the stringsplit pieces togetther causes a 90 rotation of the data which you then correct with another function. I did limited testing, but have found that when converting 128,417 characters on 8160 lines in a .csv file  your code takes ~0.4 seconds longer at ~36.5 seconds which is the best speed difference.

I had expected a time lag and that Melba23's example would run faster. The functions I wrote are intended to accommodate miscellaneous scenarios including auto-determination of the number output dimensions with _ArrayAttach(). Dealing with three, or more, dimensions leads to horrendous syntax and these functions are an attempt to avoid that.

Edited by czardas

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×