Jump to content
sigil

fast way to read excel range into array?

Recommended Posts

sigil

I have a 400 x 250 Excel sheet, and I'd like to read its contents into an AutoIt array (using COM of course). But making individual _ExcelReadCell calls in nested For..Next loops, which is what _ExcelReadSheetToArray() does, is really slow. I'm hoping there's a way I can just read the whole sheet as a range "R1C1:R400C250" and give that range to AutoIt as a 2D array.

I'm having a little trouble finding what I need in VBA, though. Just calling excelObject.range("cellA:cellB") doesn't return an array, it returns a Range object. Is there a way I can return an array of cell values, or do I have to loop through them all to build that array in a way that will let me pass it to AutoIt?

Share this post


Link to post
Share on other sites
jchd

It seems to me that _ExcelReadSheetToArray() is close to your best bet. Each value grabbed from Excel has to be instanciated in an AutoIt variant anyway and it's possible that's where cycles go. Remember that you can store anything in an AutoIt array, so variant ctor has to be called on the fly. I may be overlooking a faster way, but that's how I see things.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites
dani

If you don't want to use _ExcelReadSheetToArray() you can try using the COM model of Excel yourself in one way or another. Dig through the Excel Object Model Reference, it contains everything you need.

Share this post


Link to post
Share on other sites
PsaltyDS

Hmm... don't have Excel here to test with, but I'm curious if something like this might return an array variant directly from Excel:

$aArray = $oExcel.Activesheet.Range("A1:B20").Resize(2,20)
If IsArray($aArray) Then _ArrayDisplay($aArray, "$aArray")

:mellow:


Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law

Share this post


Link to post
Share on other sites
sigil

Hmm... don't have Excel here to test with, but I'm curious if something like this might return an array variant directly from Excel:

$aArray = $oExcel.Activesheet.Range("A1:B20").Resize(2,20)
If IsArray($aArray) Then _ArrayDisplay($aArray, "$aArray")

:mellow:

IsArray() returned False for this code. Resize returns a Range object, not a variant.

Share this post


Link to post
Share on other sites
PsaltyDS

How about value of the range?

$aArray = $oExcel.Activesheet.Range("A1:B20").Value
If IsArray($aArray) Then _ArrayDisplay($aArray, "$aArray")

I found several VBA examples on Google where a variable is declared as a variant and set to the value(s) of a range, which should be an array.

(Sorry, I would answer my own question, but I'm happily living in OOo-land now and don't have Excel on this box.)

:mellow:


Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law

Share this post


Link to post
Share on other sites
sigil

How about value of the range?

$aArray = $oExcel.Activesheet.Range("A1:B20").Value
If IsArray($aArray) Then _ArrayDisplay($aArray, "$aArray")

I found several VBA examples on Google where a variable is declared as a variant and set to the value(s) of a range, which should be an array.

(Sorry, I would answer my own question, but I'm happily living in OOo-land now and don't have Excel on this box.)

:mellow:

Wow.

Hours of coding have officially been saved.

Many thanks.

Share this post


Link to post
Share on other sites
PsaltyDS

Wow.

Hours of coding have officially been saved.

Many thanks.

I can't tell if that's sarcastic or it worked... :(

Saving a big "Whoot!" only for if it worked.

:mellow:


Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law

Share this post


Link to post
Share on other sites
Tvern

I can't tell if that's sarcastic or it worked... :(

Saving a big "Whoot!" only for if it worked.

:mellow:

You get a cookie! Can't believe how well that works. :lol:

Share this post


Link to post
Share on other sites
jchd

Can you elaborate and post a small working example?

If there's a way to have the standard UDF work significantly better, that would be useful. Also, perhaps a smart solution could benefit other COM applications as well, and their correcponding UDF here.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites
PsaltyDS

You get a cookie! Can't believe how well that works. :(

Whoo Hoo! :mellow:

When I get somewhere with Excel loaded, I'll look into using that technique to tweak _ExcelReadArray(), _ExcelWriteArray(), _ExcelReadSheetToArray() and _ExcelWriteSheetFromArray() for the performance improvement.

:lol:


Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law

Share this post


Link to post
Share on other sites
Tvern

Can you elaborate and post a small working example?

If there's a way to have the standard UDF work significantly better, that would be useful. Also, perhaps a smart solution could benefit other COM applications as well, and their correcponding UDF here.

example:

#Include <Excel.au3>
#include <Array.au3>
$oExcel = _ExcelBookOpen(@ScriptDir & "\test.xls",0)
$aArray = $oExcel.Activesheet.Range("A1:B20").Value
If IsArray($aArray) Then _ArrayDisplay($aArray, "$aArray")
_ExcelBookClose($oExcel)

Create test.xls in the same folder and put some data in it.

Note that colums are returned as rows and vice versa.

Infact the UDF might be slowed down by reformatting the array.

Share this post


Link to post
Share on other sites
PsaltyDS

Note that colums are returned as rows and vice versa.

Infact the UDF might be slowed down by reformatting the array.

I believe there is a .Transpose method for that in the Excel scripting interface. More stuff to play with when I get a chance.

:mellow:


Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law

Share this post


Link to post
Share on other sites
jchd

How nice. Could be transposed to several COM stuff as well.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites
sigil

I can't tell if that's sarcastic or it worked... :(

Saving a big "Whoot!" only for if it worked.

:mellow:

Sorry that wasn't clear. It worked very well. Thanks very much for helping me solve that problem.

Share this post


Link to post
Share on other sites
antonioj84

here is the proper way folks if i can save someone time

#include <Excel.au3>
#include <File.au3>
#include <Array.au3>



Local $oExcel =_Excel_Open()


$datawb = _Excel_BookOpen($oExcel,@ScriptDir & "\test.xls")
$datawb.worksheets("info").select
$LastRow = $datawb.ActiveSheet.UsedRange.Rows.Count
$mydata = _Excel_RangeRead($datawb, Default, "A1:AI" & $LastRow)
If IsArray($mydata) Then _ArrayDisplay($mydata)


_Excel_BookClose($datawb)

 

Share this post


Link to post
Share on other sites
water

Most of the participating users have been offline for years ;) And a lot has changed in AutoIt - especially since I have rewritten the Excel UDF :)
Your example needs a few comment lines so that users can see that you "only" read a few columns from a spcific worksheet.

  • Like 1

My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (2018-06-01 - Version 1.4.9.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2018-09-01 - Version 1.3.4.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
PowerPoint (2017-06-06 - Version 0.0.5.0) - Download - General Help & Support
Excel - Example Scripts - Wiki
Word - Wiki
 
Tutorials:

ADO - Wiki

 

Share this post


Link to post
Share on other sites
antonioj84

 hey Water, actually I can read everything, " I am not limited  to a few columns"my range is from column A1 to AI

Local $oExcel =_Excel_Open(false)  ; instantiate and hide excel

$datawb = _Excel_BookOpen($oExcel,@ScriptDir & "\s5315.xlsm", 0) ;assign data to $datawb
$datawb.worksheets("info").select                                 ; select info tab
$LastRow = $datawb.ActiveSheet.UsedRange.Rows.Count               ; determine last row
$mydata = _Excel_RangeRead($datawb, Default, "A1:AI" & $LastRow)   ; read range from A1 to AI
;If IsArray($mydata) Then _ArrayDisplay($mydata)                  ; display spreadsheet data in the array


_Excel_BookClose($datawb,True)

A1  to AI  until the last row

  • Like 1

Share this post


Link to post
Share on other sites
water

Then set the range to "Default" and the function returns all used cells.


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (2018-06-01 - Version 1.4.9.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2018-09-01 - Version 1.3.4.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
PowerPoint (2017-06-06 - Version 0.0.5.0) - Download - General Help & Support
Excel - Example Scripts - Wiki
Word - Wiki
 
Tutorials:

ADO - Wiki

 

Share this post


Link to post
Share on other sites
antonioj84

"Then set the range to "Default" and the function returns all used cells."

Hey Water ,  excellent point actually

I change this   

$mydata = _Excel_RangeRead($datawb, Default, "A1:AI" & $LastRow)   ; read range from A1 to AI

to that

$mydata = _Excel_RangeRead($datawb,Default,Default )

 

Edited by antonioj84

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×