Jump to content
Sign in to follow this  
Bowmore

Fast Array Write To Excel

Recommended Posts

Bowmore

I have some scripts that use Excel to present the result of data validation tests and I was becoming a little frustrated at the length of time it was taking to write the data to excel, using _ExcelWriteSheetFromArray() function in the Excel.au3 UDF. So I thought I'd take a look and see if I could improve on it's performance. This was the result. This new function is 200 to 300 times faster than _ExcelWriteSheetFromArray() for large arrays (20000 plus elements) and in the region of 100 times faster for small arrays.

I have also included the option to write only part of the array to the spreadsheet as sometimes I do not need all of the array.

_ExcelSheetWriteFromArray

; #FUNCTION# ====================================================================================================================
; Name...........: _ExcelSheetWriteFromArray
; Description ...: Writes a 2D array, or part of, to a specified location in the active worksheet
; Syntax.........: _ExcelSheetWriteFromArray($oExcel, ByRef $aArray, $iExcelStartRow = Default, $iExcelStartCol = Default, $iArrayStartRow = Default, $iArrayStartCol = Default, $iArrayEndRow = Default, $iArrayEndCol = Default)
; Parameters ....: $oExcel - Excel object opened by a preceding call to _ExcelBookOpen() or _ExcelBookNew()
;                $aArray - The array ByRef to write data from (array is not modified)
;                $iExcelStartRow - The spreadsheet row to start writing the array to, default is 1
;                $iExcelStartCol - The spreadsheet column to start writing the array to, default is 1
;                $iArrayStartRow - Start row index of array to start writing data from, default is 0
;                $iArrayStartCol - Start column index of array to start writing data from, default is 0
;                $iArrayEndRow - End row index of array to stop writing data from, default is Ubound($aArray)
;                $iArrayEndCol - End column index of array to stop writing data from, default is Ubound($aArray,2)
; Return values .: Success   - Returns 1
;                Failure - Returns 0 and sets @error on errors:
;                |@error=1   - Excel object does not exist
;                |@error=2   - Not a 2D array
;                |@error=3   - End row greater than start row
;                |@error=4   - End col greater than start col
; Author ........: Bowmore
; Modified.......:
; Remarks .......: Only 2D arrays are currently handeled. Maxim cell content length is 255 characters
; Related .......:
; Link ..........:
; Example .......: Yes
; ===============================================================================================================================
Func _ExcelSheetWriteFromArray($oExcel, ByRef $aArray, $iExcelStartRow = Default, $iExcelStartCol = Default, $iArrayStartRow = Default, $iArrayStartCol = Default, $iArrayEndRow = Default, $iArrayEndCol = Default)
Local $iExcelEndRow = 1
Local $iExcelEndCol = 1

If Not IsObj($oExcel) Then Return SetError(1, 0, 0)
If UBound($aArray, 0) <> 2 Then Return SetError(2, 0, 0)
If $iExcelStartRow = Default Then $iExcelStartRow = 1
If $iExcelStartCol = Default Then $iExcelStartCol = 1
If $iArrayStartRow = Default Then $iArrayStartRow = 0
If $iArrayStartCol = Default Then $iArrayStartCol = 0
If $iArrayEndRow = Default Then $iArrayEndRow = UBound($aArray) - 1
If $iArrayEndCol = Default Then $iArrayEndCol = UBound($aArray, 2) - 1
If $iExcelStartRow < 1 Then $iExcelStartRow = 1
If $iExcelStartCol < 1 Then $iExcelStartCol = 1
If $iArrayStartRow < 0 Then $iArrayStartRow = 0
If $iArrayStartCol < 0 Then $iArrayStartCol = 0
If $iArrayEndRow < $iArrayStartRow Then Return SetError(3, 1, 0)
If $iArrayEndCol < $iArrayStartCol Then Return SetError(4, 1, 0)
$iExcelEndRow = $iExcelStartRow + $iArrayEndRow - $iArrayStartRow
$iExcelEndCol = $iExcelStartCol + $iArrayEndCol - $iArrayStartCol

; Check if only part of the array is to written to the speadsheet
If $iArrayStartRow <> 0 Or $iArrayStartCol <> 0 Or $iArrayEndRow <> UBound($aArray) - 1 Or $iArrayEndCol = UBound($aArray, 2) - 1 Then
;Copy specified array range to a temporary array
Local $aTemp[$iArrayEndRow - $iArrayStartRow + 1][$iArrayEndCol - $iArrayStartCol + 1]
Local $iRow = 0
Local $iCol = 0
For $i = $iArrayStartRow To $iArrayEndRow
$iCol = 0
For $j = $iArrayStartCol To $iArrayEndCol
$aTemp[$iRow][$iCol] = $aArray[$i][$j]
$iCol += 1
Next
$iRow += 1
Next
With $oExcel.ActiveSheet
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).Select
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).value = $oExcel.Application.WorksheetFunction.Transpose($aTemp)
EndWith
Else
With $oExcel.ActiveSheet
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).Select
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).value = $oExcel.Application.WorksheetFunction.Transpose($aArray)
EndWith
EndIf

EndFunc ;==>_ExcelSheetWriteFromArray

Simple Speed Test _ExcelSheetWriteFromArray

#include <excel.au3>

Test_Main()

Func Test_Main()
Local $sXlsFile1 = @ScriptDir & "Test1.xls"
Local $sXlsFile2 = @ScriptDir & "Test2.xls"
Local $oExcel = 0
Local $Tim = 0
Local $aData[5000][10]

For $i = 0 To 4999
$aData[$i][0] = "Y"
$aData[$i][1] = "DATA_TEST_ROW_0" & StringFormat("%03i", $i)
$aData[$i][2] = "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vestibulum ultrices est varius elit consequat sollicitudin mollis libero ultrices. Quisque lobortis lorem quis nunc vestibulum tincidunt. Duis eget urna sapien, nec egestas purus. Fusce massa nunc."
$aData[$i][3] = $i * 10
$aData[$i][4] = $i * 100
$aData[$i][5] = $aData[$i][3] / $aData[$i][4]
$aData[$i][6] = "Pass"
$aData[$i][7] = @YEAR & "-" & @MON & "-" & @MDAY & "T" & @HOUR & ":" & @MIN
$aData[$i][8] = "FAIL"
$aData[$i][9] = "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vestibulum ultrices est varius elit consequat sollicitudin mollis libero ultrices. Quisque lobortis lorem quis nunc vestibulum tincidunt. Duis eget urna sapien, nec egestas purus. Fusce massa nunc."
Next

; Speed Test of new function
$oExcel = _ExcelBookNew(0)
$Tim = TimerInit()
_ExcelSheetWriteFromArray($oExcel, $aData)
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : TimerDiff($Tim) = ' & TimerDiff($Tim) & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console
_ExcelBookSaveAs($oExcel, $sXlsFile1)
_ExcelBookClose($oExcel, 0, 0)

; Speed Test of old function
$oExcel = _ExcelBookNew(0)
$Tim = TimerInit()
_ExcelWriteSheetFromArray($oExcel, $aData)
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : TimerDiff($Tim) = ' & TimerDiff($Tim) & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console
_ExcelBookSaveAs($oExcel, $sXlsFile2)
_ExcelBookClose($oExcel, 0, 0)

EndFunc ;==>Test_Main


; #FUNCTION# ====================================================================================================================
; Name...........: _ExcelSheetWriteFromArray
; Description ...: Writes a 2D array, or part of, to a specified location in the active worksheet
; Syntax.........: _ExcelSheetWriteFromArray($oExcel, ByRef $aArray, $iExcelStartRow = Default, $iExcelStartCol = Default, $iArrayStartRow = Default, $iArrayStartCol = Default, $iArrayEndRow = Default, $iArrayEndCol = Default)
; Parameters ....: $oExcel - Excel object opened by a preceding call to _ExcelBookOpen() or _ExcelBookNew()
;                $aArray - The array ByRef to write data from (array is not modified)
;                $iExcelStartRow - The spreadsheet row to start writing the array to, default is 1
;                $iExcelStartCol - The spreadsheet column to start writing the array to, default is 1
;                $iArrayStartRow - Start row index of array to start writing data from, default is 0
;                $iArrayStartCol - Start column index of array to start writing data from, default is 0
;                $iArrayEndRow - End row index of array to stop writing data from, default is Ubound($aArray)
;                $iArrayEndCol - End column index of array to stop writing data from, default is Ubound($aArray,2)
; Return values .: Success   - Returns 1
;                Failure - Returns 0 and sets @error on errors:
;                |@error=1   - Excel object does not exist
;                |@error=2   - Not a 2D array
;                |@error=3   - End row greater than start row
;                |@error=4   - End col greater than start col
; Author ........: Bowmore
; Modified.......:
; Remarks .......: Only 2D arrays are currently handeled. Maxim cell content length is 255 characters
; Related .......:
; Link ..........:
; Example .......: Yes
; ===============================================================================================================================
Func _ExcelSheetWriteFromArray($oExcel, ByRef $aArray, $iExcelStartRow = Default, $iExcelStartCol = Default, $iArrayStartRow = Default, $iArrayStartCol = Default, $iArrayEndRow = Default, $iArrayEndCol = Default)
Local $iExcelEndRow = 1
Local $iExcelEndCol = 1

If Not IsObj($oExcel) Then Return SetError(1, 0, 0)
If UBound($aArray, 0) <> 2 Then Return SetError(2, 0, 0)
If $iExcelStartRow = Default Then $iExcelStartRow = 1
If $iExcelStartCol = Default Then $iExcelStartCol = 1
If $iArrayStartRow = Default Then $iArrayStartRow = 0
If $iArrayStartCol = Default Then $iArrayStartCol = 0
If $iArrayEndRow = Default Then $iArrayEndRow = UBound($aArray) - 1
If $iArrayEndCol = Default Then $iArrayEndCol = UBound($aArray, 2) - 1
If $iExcelStartRow < 1 Then $iExcelStartRow = 1
If $iExcelStartCol < 1 Then $iExcelStartCol = 1
If $iArrayStartRow < 0 Then $iArrayStartRow = 0
If $iArrayStartCol < 0 Then $iArrayStartCol = 0
If $iArrayEndRow < $iArrayStartRow Then Return SetError(3, 1, 0)
If $iArrayEndCol < $iArrayStartCol Then Return SetError(4, 1, 0)
$iExcelEndRow = $iExcelStartRow + $iArrayEndRow - $iArrayStartRow
$iExcelEndCol = $iExcelStartCol + $iArrayEndCol - $iArrayStartCol

; Check if only part of the array is to written to the speadsheet
If $iArrayStartRow <> 0 Or $iArrayStartCol <> 0 Or $iArrayEndRow <> UBound($aArray) - 1 Or $iArrayEndCol = UBound($aArray, 2) - 1 Then
;Copy specified array range to a temporary array
Local $aTemp[$iArrayEndRow - $iArrayStartRow + 1][$iArrayEndCol - $iArrayStartCol + 1]
Local $iRow = 0
Local $iCol = 0
For $i = $iArrayStartRow To $iArrayEndRow
$iCol = 0
For $j = $iArrayStartCol To $iArrayEndCol
$aTemp[$iRow][$iCol] = $aArray[$i][$j]
$iCol += 1
Next
$iRow += 1
Next
With $oExcel.ActiveSheet
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).Select
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).value = $oExcel.Application.WorksheetFunction.Transpose($aTemp)
EndWith
Else
With $oExcel.ActiveSheet
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).Select
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).value = $oExcel.Application.WorksheetFunction.Transpose($aArray)
EndWith
EndIf

EndFunc ;==>_ExcelSheetWriteFromArray
Edited by Bowmore
  • Like 1

"Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to build bigger and better idiots. So far, the universe is winning."- Rick Cook

Share this post


Link to post
Share on other sites
Spiff59

There is a ticket in BugTracker for a new version of _ExcelReadSheetFromArray() that also eliminates the nested loops that individually access Excel cells in favor of using the Range.Cells method. It shows a similar speed increase.

I would think if you reverted back to the production function name, switched both the $iRowBase ($iArrayStartRow) and $iColBase ($iArrayStartCol) parameters back to defaulting to 1, instead of 0, and maintained the existing error return values (with some of your own additions), that you'd have a backward-compatible function (with 2 new parms) that could replace the one in the production UDF?

Share this post


Link to post
Share on other sites
Bowmore

Thanks for the suggestions. Although I wrote it to meet my own specific needs I would make sense to change the version I posted hear to make it compatible with the existing _ExcelWriteSheetFromArray(). I'll change it in the next couple of days when I have half an hour to make the changes and test it properly.


"Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to build bigger and better idiots. So far, the universe is winning."- Rick Cook

Share this post


Link to post
Share on other sites
tempman

Hi,

Your Fast Array Write To Excel Script rocks, but only for 65536 rows, if I put one more 65537 I get an error:

: ==> The requested action with this object has failed.:

.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).value = $oExcel.Application.WorksheetFunction.Transpose($aTemp)

.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).value = $oExcel.Application.WorksheetFunction.Transpose($aTemp)^ ERROR

can that be fixed?

Share this post


Link to post
Share on other sites
water

I'm about to rewrite the Excel UDF that comes with AutoIt. It's an early Alpha version but your problem has been solved by GMX's (it is named _Excel_RangeRead).

If you can test, please post the results in the Excel Rewrite thread.


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2017-04-18 - Version 1.4.8.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2017-02-27 - Version 1.3.1.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2015-04-01 - Version 0.4.0.0) - Download - General Help & Support - Example Scripts
Excel - Example Scripts - Wiki
Word - Wiki
PowerPoint (2015-06-06 - Version 0.0.5.0) - Download - General Help & Support

Tutorials:
ADO - Wiki

 

Share this post


Link to post
Share on other sites
GMK

Actually, I need to make changes to the _Excel_RangeWrite function as well. Will post it in the Excel Rewrite thread as soon as I've completed it.

Share this post


Link to post
Share on other sites
tempman

I agree with GMK, I have tested _ExcelSheetWriteFromArray with Excel Rewrite.au3 and 65536 issue is still present.

Waiting for changes to the _Excel_RangeWrite function...

Share this post


Link to post
Share on other sites
BrewManNH

There appears to be a limit of 65536 cells when using the .transpose method at least up to Excel 2007, might even go beyond that, but I'm not sure.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
tempman

I find this post on mrexcel.com forum: http://www.mrexcel.com/forum/excel-questions/555073-limitation-range-size-transpose-function.html#post2742571

(The Transpose function just doesn't work with >65536, just as SpecialCells or AdvancedFilter etc.methods sometimes don't work with ranges above a certain size. You'd have to ask the Microsoft coders why they haven't updated these to allow for the larger sheet size of Excel 2007 and subsequent versions.)

I really don't know is this comment form mrexcel.com forum true or false, but I find one not so pretty but working solution for example in our topic

#include <excel.au3>

Test_Main()

Func Test_Main()
Local $oExcel = 0
Local $Tim = 0
Local $aData[65536][1]
Local $aData1[65536][1]

For $i = 0 To 65535
$aData[$i][0] = "DATA_TEST_ROW_0" & StringFormat("%03i", $i+1)
Next

For $j = 0 To 65535
$aData1[$j][0] = "DATA_TEST_ROW_0" & StringFormat("%03i", $j+65537)
Next

; Speed Test of new function
$oExcel = _ExcelBookNew()
$Tim = TimerInit()
_ExcelSheetWriteFromArray($oExcel, $aData)
_ExcelSheetWriteFromArray($oExcel, $aData1, 65537, 1)
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : TimerDiff($Tim) = ' & TimerDiff($Tim) & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

EndFunc ;==>Test_Main


; #FUNCTION# ====================================================================================================================
; Name...........: _ExcelSheetWriteFromArray
; Description ...: Writes a 2D array, or part of, to a specified location in the active worksheet
; Syntax.........: _ExcelSheetWriteFromArray($oExcel, ByRef $aArray, $iExcelStartRow = Default, $iExcelStartCol = Default, $iArrayStartRow = Default, $iArrayStartCol = Default, $iArrayEndRow = Default, $iArrayEndCol = Default)
; Parameters ....: $oExcel - Excel object opened by a preceding call to _ExcelBookOpen() or _ExcelBookNew()
;            $aArray - The array ByRef to write data from (array is not modified)
;            $iExcelStartRow - The spreadsheet row to start writing the array to, default is 1
;            $iExcelStartCol - The spreadsheet column to start writing the array to, default is 1
;            $iArrayStartRow - Start row index of array to start writing data from, default is 0
;            $iArrayStartCol - Start column index of array to start writing data from, default is 0
;            $iArrayEndRow - End row index of array to stop writing data from, default is Ubound($aArray)
;            $iArrayEndCol - End column index of array to stop writing data from, default is Ubound($aArray,2)
; Return values .: Success - Returns 1
;            Failure - Returns 0 and sets @error on errors:
;            |@error=1 - Excel object does not exist
;            |@error=2 - Not a 2D array
;            |@error=3 - End row greater than start row
;            |@error=4 - End col greater than start col
; Author ........: Bowmore
; Modified.......:
; Remarks .......: Only 2D arrays are currently handeled. Maxim cell content length is 255 characters
; Related .......:
; Link ..........:
; Example .......: Yes
; ===============================================================================================================================
Func _ExcelSheetWriteFromArray($oExcel, ByRef $aArray, $iExcelStartRow = Default, $iExcelStartCol = Default, $iArrayStartRow = Default, $iArrayStartCol = Default, $iArrayEndRow = Default, $iArrayEndCol = Default)
Local $iExcelEndRow = 1
Local $iExcelEndCol = 1

If Not IsObj($oExcel) Then Return SetError(1, 0, 0)
If UBound($aArray, 0) <> 2 Then Return SetError(2, 0, 0)
If $iExcelStartRow = Default Then $iExcelStartRow = 1
If $iExcelStartCol = Default Then $iExcelStartCol = 1
If $iArrayStartRow = Default Then $iArrayStartRow = 0
If $iArrayStartCol = Default Then $iArrayStartCol = 0
If $iArrayEndRow = Default Then $iArrayEndRow = UBound($aArray) - 1
If $iArrayEndCol = Default Then $iArrayEndCol = UBound($aArray, 2) - 1
If $iExcelStartRow < 1 Then $iExcelStartRow = 1
If $iExcelStartCol < 1 Then $iExcelStartCol = 1
If $iArrayStartRow < 0 Then $iArrayStartRow = 0
If $iArrayStartCol < 0 Then $iArrayStartCol = 0
If $iArrayEndRow < $iArrayStartRow Then Return SetError(3, 1, 0)
If $iArrayEndCol < $iArrayStartCol Then Return SetError(4, 1, 0)
$iExcelEndRow = $iExcelStartRow + $iArrayEndRow - $iArrayStartRow
$iExcelEndCol = $iExcelStartCol + $iArrayEndCol - $iArrayStartCol

; Check if only part of the array is to written to the speadsheet
If $iArrayStartRow <> 0 Or $iArrayStartCol <> 0 Or $iArrayEndRow <> UBound($aArray) - 1 Or $iArrayEndCol = UBound($aArray, 2) - 1 Then
;Copy specified array range to a temporary array
Local $aTemp[$iArrayEndRow - $iArrayStartRow + 1][$iArrayEndCol - $iArrayStartCol + 1]
Local $iRow = 0
Local $iCol = 0
For $i = $iArrayStartRow To $iArrayEndRow
$iCol = 0
For $j = $iArrayStartCol To $iArrayEndCol
$aTemp[$iRow][$iCol] = $aArray[$i][$j]
$iCol += 1
Next
$iRow += 1
Next
With $oExcel.ActiveSheet
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).Select
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).value = $oExcel.Application.WorksheetFunction.Transpose($aTemp)
EndWith
Else
With $oExcel.ActiveSheet
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).Select
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).value = $oExcel.Application.WorksheetFunction.Transpose($aArray)
EndWith
EndIf

EndFunc ;==>_ExcelSheetWriteFromArray

Share this post


Link to post
Share on other sites
BrewManNH

A better way would be to do something like this:

#include <excel.au3>

Test_Main()


; #FUNCTION# ====================================================================================================================
; Name...........: _ExcelSheetWriteFromArray
; Description ...: Writes a 2D array, or part of, to a specified location in the active worksheet
; Syntax.........: _ExcelSheetWriteFromArray($oExcel, ByRef $aArray, $iExcelStartRow = Default, $iExcelStartCol = Default, $iArrayStartRow = Default, $iArrayStartCol = Default, $iArrayEndRow = Default, $iArrayEndCol = Default)
; Parameters ....: $oExcel - Excel object opened by a preceding call to _ExcelBookOpen() or _ExcelBookNew()
;            $aArray - The array ByRef to write data from (array is not modified)
;            $iExcelStartRow - The spreadsheet row to start writing the array to, default is 1
;            $iExcelStartCol - The spreadsheet column to start writing the array to, default is 1
;            $iArrayStartRow - Start row index of array to start writing data from, default is 0
;            $iArrayStartCol - Start column index of array to start writing data from, default is 0
;            $iArrayEndRow - End row index of array to stop writing data from, default is Ubound($aArray)
;            $iArrayEndCol - End column index of array to stop writing data from, default is Ubound($aArray,2)
; Return values .: Success - Returns 1
;            Failure - Returns 0 and sets @error on errors:
;            |@error=1 - Excel object does not exist
;            |@error=2 - Not a 2D array
;            |@error=3 - End row greater than start row
;            |@error=4 - End col greater than start col
; Author ........: Bowmore
; Modified.......:
; Remarks .......: Only 2D arrays are currently handeled. Maxim cell content length is 255 characters
; Related .......:
; Link ..........:
; Example .......: Yes
; ===============================================================================================================================
Func _ExcelSheetWriteFromArray($oExcel, ByRef $aArray, $iExcelStartRow = Default, $iExcelStartCol = Default, $iArrayStartRow = Default, $iArrayStartCol = Default, $iArrayEndRow = Default, $iArrayEndCol = Default)
    Local $iExcelEndRow = 1
    Local $iExcelEndCol = 1

    If Not IsObj($oExcel) Then Return SetError(1, 0, 0)
    If UBound($aArray, 0) <> 2 Then Return SetError(2, 0, 0)
    If $iExcelStartRow = Default Then $iExcelStartRow = 1
    If $iExcelStartCol = Default Then $iExcelStartCol = 1
    If $iArrayStartRow = Default Then $iArrayStartRow = 0
    If $iArrayStartCol = Default Then $iArrayStartCol = 0
    If $iArrayEndRow = Default Then $iArrayEndRow = UBound($aArray) - 1
    If $iArrayEndCol = Default Then $iArrayEndCol = UBound($aArray, 2) - 1
    If $iExcelStartRow < 1 Then $iExcelStartRow = 1
    If $iExcelStartCol < 1 Then $iExcelStartCol = 1
    If $iArrayStartRow < 0 Then $iArrayStartRow = 0
    If $iArrayStartCol < 0 Then $iArrayStartCol = 0
    If $iArrayEndRow < $iArrayStartRow Then Return SetError(3, 1, 0)
    If $iArrayEndCol < $iArrayStartCol Then Return SetError(4, 1, 0)
    If $iArrayEndRow > UBound($aArray) - 1 Then $iArrayEndRow = UBound($aArray) - 1 ; <<<<<<<<<<<<<<< added this line, because there's no check to see if the $iArrayEndRow is larger than the end of the array
    $iExcelEndRow = $iExcelStartRow + $iArrayEndRow - $iArrayStartRow
    $iExcelEndCol = $iExcelStartCol + $iArrayEndCol - $iArrayStartCol

    ; Check if only part of the array is to written to the speadsheet
    If $iArrayStartRow <> 0 Or $iArrayStartCol <> 0 Or $iArrayEndRow <> UBound($aArray) - 1 Or $iArrayEndCol = UBound($aArray, 2) - 1 Then
        ;Copy specified array range to a temporary array
        Local $aTemp[$iArrayEndRow - $iArrayStartRow + 1][$iArrayEndCol - $iArrayStartCol + 1]
        Local $iRow = 0
        Local $iCol = 0
        For $i = $iArrayStartRow To $iArrayEndRow
            $iCol = 0
            For $j = $iArrayStartCol To $iArrayEndCol
                $aTemp[$iRow][$iCol] = $aArray[$i][$j]
                $iCol += 1
            Next
            $iRow += 1
        Next
        With $oExcel.ActiveSheet
            .Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).Select
            .Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).value = $oExcel.Application.WorksheetFunction.Transpose($aTemp)
        EndWith
    Else
        With $oExcel.ActiveSheet
            .Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).Select
            .Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).value = $oExcel.Application.WorksheetFunction.Transpose($aArray)
        EndWith
    EndIf

EndFunc   ;==>_ExcelSheetWriteFromArray

Func Test_Main()
    Local $oExcel = 0
    Local $Tim = 0
    Local $aData[131072][1]

    For $i = 0 To 131071
        $aData[$i][0] = "DATA_TEST_ROW_0" & StringFormat("%03i", $i + 1)
    Next
    ; Speed Test of new function
    $oExcel = _ExcelBookNew()
    $Tim = TimerInit()
    ; This only writes 65535 elements of the array at a time, if your array is larger than that, then it will split it up
    ; for you.
    For $loop = 0 To UBound($aData) - 1 Step 65536 
        _ExcelSheetWriteFromArray($oExcel, $aData, $loop + 1, 0, $loop, 0, $Loop + 65535)
    Next
    ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : TimerDiff($Tim) = ' & TimerDiff($Tim) & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

EndFunc   ;==>Test_Main

This rewrites the test script that tempman posted above and also corrects a bug in the _ExcelSheetWriteFromArray function if you send it an end row of the array that is beyond the UBound dimension of the array.

This will write 65535 rows to the spreadsheet until you've hit the end of the array, and the array can be of any size either larger or smaller than 65535 rows.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
water

Thanks GMK!

I hope to release a new alpha version with your changes quite soon.

Edited by water

My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2017-04-18 - Version 1.4.8.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2017-02-27 - Version 1.3.1.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2015-04-01 - Version 0.4.0.0) - Download - General Help & Support - Example Scripts
Excel - Example Scripts - Wiki
Word - Wiki
PowerPoint (2015-06-06 - Version 0.0.5.0) - Download - General Help & Support

Tutorials:
ADO - Wiki

 

Share this post


Link to post
Share on other sites
water

I posted a revised version of _Excel_RangeWrite for you to test.

Would like to hear from Excel 2003 testers as I only can test with Excel 2010.


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2017-04-18 - Version 1.4.8.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2017-02-27 - Version 1.3.1.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2015-04-01 - Version 0.4.0.0) - Download - General Help & Support - Example Scripts
Excel - Example Scripts - Wiki
Word - Wiki
PowerPoint (2015-06-06 - Version 0.0.5.0) - Download - General Help & Support

Tutorials:
ADO - Wiki

 

Share this post


Link to post
Share on other sites
DeDee

Hi

Is there a script for Fast Array Write from Excel? I see that it takes a lot of time for this as well - when using big Excel files.

Thank you!

Share this post


Link to post
Share on other sites
water

Welcome to AutoIt and the forum,

the rewrite of the Excel UDF I'm currently working on has a _Excel_RangeRead function too.

Please check the for the latest download. But be aware that it is still an early Alpha version!

If you search the forum you will find some other fast _ExcelReadSheetToArray functions too.


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2017-04-18 - Version 1.4.8.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2017-02-27 - Version 1.3.1.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2015-04-01 - Version 0.4.0.0) - Download - General Help & Support - Example Scripts
Excel - Example Scripts - Wiki
Word - Wiki
PowerPoint (2015-06-06 - Version 0.0.5.0) - Download - General Help & Support

Tutorials:
ADO - Wiki

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Similar Content

    • Gowrisankar
      By Gowrisankar
      Dear members of the forum,
      I need to open excel files that may or may not need a password and finally move the files that needs password to manual queue.
      Is there a fastest way to do this?
       
      PS: I have a huge respect for the rules of this forum. I am not asking assistance to override any security measure. I just need to segregate the files that needs passwords.
    • Iceburg
      By Iceburg
      Hi everyone, I'm at best a noobie.  I have read through the Array helps, and specifically the 2D array help file, and I'm struggling to get my code working.
      I have an array that is read from a file, thats working great.  I'm trying to do some math on the array, so I can find the largest, average, lowest, day over day change %, etc.
      The array read working fine, I get 43 lines, line 0 is 44, and then I get data that looks like
      0519 $10,000
      0520 $10,001
      0521 $10,002
      The data in this array is a single 1D array, breaking it out into 2 columns so I can do the math is what I can get to happen.  
      How do I reference the array to store this data?  Second, how do I assign this data to the appropriate row/column?
      Thanks in advance.
      Dim $all_cash_amounts[UBound($aInput)][2] Dim $max_amount_in_account Dim $min_amount_in_account _FileReadToArray($LC_Check_file_path, $aInput) _ArrayDisplay($aInput) local $date = StringRegExp($aInput[1], "(\d\d\d\d)", 1) local $cash = StringRegExp($aInput[1], "\d+\s(-?[0-9\.\,]+)", 1) ConsoleWrite("Date is: " & $date & @CRLF) For $i = 1 To UBound($aInput)-1     $date = StringRegExp($aInput[$i], "(\d\d\d\d)", 1)     $all_cash_amounts[$i][2] = $date[$i][0], $cash[$i][1]      Next _ArrayDisplay($all_cash_amounts)  
    • OldGuyWalking
      By OldGuyWalking
      Given an array with multiple columns that is displayed in a listview,
       ===> What is the fastest/most efficient way to create and manage multiple filters and display results in ListView.
      I have a text file that loads into a listview that has string, numeric, and date columns.  The main file contains about 5100 rows. It's loaded into an array and (in this ListView) it's pre-filtered to display a range of rows based on a start and end date.  On the form I have menu options for various filters. (see below).
      I have options to filter on an "Air Date" column (=Today, >=Today, <=Today) and on a numeric field that is either 1 or 0 that indicate Active or Ended.

      For each filter option I have a prebuilt array that holds a subset of the main array based on a single filter.  For the list above I have the main Array and 5 additional arrays.  None of the arrays are updated since this is for "view only" purposes.  This is a short list and I could have done the filtering "live" but I have several of these forms and so kept the same functionality in each. I have another ListView that displays the complete 5100 row list with 3 filters that, when building the filters live was considerably slower than using prebuilt arrays.
      If I want to expand past simple single column filtering, using an array for each filter becomes cumbersome especially if I want to combine filters using AND & OR.
      The text file I'm working with has 16 columns. If I setup filters for 4 columns and include AND / OR capability that would require prebuilding 24 arrays to cover the various combinations.
      If using the slower method of building a filtered array in real time each time a different filter is selected is the only way to go with this then I'll live with it. It is less overhead. .
      Below is the code I'm currently using to "filter" an array.  My next change was going to add AND / OR functionality (see the info above the header for where I was going with this) .
      ; Description ...: Delete rows from an array and only keep rows that meet the crtieria of identified columns. ; ; Next Change: Add AND/OR to combine filters. Use array to hold multiple criteria and values? ; ; Local $aCriteria[][] = [["",$iColNbr1, $sOperator1, $vValue1], ["AND",$iColNbr2, $sOperator2, $vValue2], ["OR",$iColNbr3, $sOperator3, $vValue3]] ; The first set of criteria ["", $iColNbr1, $sOperator1, $vValue1] must start with a "". ; If anything is entered in that first parameter it will be ignored. ; If the first parameter in any additional criteria set is left blank, or it is not OR, it will default to AND. ; If $aArray is 1 dimension with more than one set of criteria, only the first set will be used. ; Any criteria that uses a column that is less than 0 or higher than the total number of columns in the array will return an error. ; ; Recognized data types for this function are: S (String), D (Date), N (Number). ; ; Recognized Operators are: "EQ", "NEQ", "IN", "GT", "GE", "LT", "LE", "BETWEEN". ; ****** Not all operators work with all data types. ; #FUNCTION# ==================================================================================================================== ; Name ..........: _ArrayFilter ; Description ...: Delete rows from an array and only keep rows that meet the crtieria of identified columns. ; Syntax ........: _ArrayFilter(Byref $aArray[, $iCol = 0[, $sOperator = "EQ"[, $vValue = ""[, $iOptionBase = 0]]]]) ; Parameters ....: $aArray - Array being filtered. ; $iCol - [optional] Column to filter. Default is 0. ; $sOperator - [optional] Operator. Default is "EQ". ; $vValue - [optional] Criteria to compare the column/row value against. ; $iOptionBase - [optional] Starting row. Default is 0. ; Return values .: None ; Author ........: OldGuyWalking ; Modified ......: ; Remarks .......: ; Related .......: ; Link ..........: ; Example .......: No ; =============================================================================================================================== Func _ArrayFilter(ByRef $aArray, $iCol = 0, $sOperator = "EQ", $vValue = "", $iOptionBase = 0) Local $hFunc = _ArrayFilter $vValue = StringStripWS($vValue, 3) If $vValue = "[Today]" Then $vValue = _NowCalcDate() EndIf Local $sMsg Local $sMsgHdr Local $n1 Local $sDeleteIndex Local $aDeleteIndex Local $iCnt = 0 Local $iRows Local $iColMax Local $iDim Local $sData Local $sVType Local $sDType Local $LBound Local $iDiff If $iOptionBase <> 0 Then $iOptionBase = 1 EndIf If _IsValueEmpty($aArray) Then Return SetError(1, 0, "") EndIf $iDim = UBound($aArray, $UBOUND_DIMENSIONS) If $iDim = 1 Then If $iCol <> 0 Then $iCol = 0 EndIf EndIf If $iDim = 2 Then $iColMax = UBound($aArray, $UBOUND_COLUMNS) - 1 If $iCol > $iColMax Or $iCol < 0 Then Return SetError(1, 0, "") EndIf EndIf If Not _IsBetween($iDim, 1, 2) Then ;############### MSG2 - START ############### $sMsgHdr = FuncName($hFunc) & " :Line: " & @ScriptLineNumber & " :Error= " & @error $sMsg = "Invalid Dimensioned Array. Must be a 1 or 2 dimensional array." MsgBox(0, $sMsgHdr, $sMsg) Return SetError(1, 0, "") ;############### MSG2 - END ############### EndIf ; Identify what the value is ; If it is not a String, Int, Number, or Date then skip. Select Case _DateIsValid($vValue) = 1 $sVType = "D" Case IsNumber($vValue) = 1 $sVType = "N" Case IsString($vValue) = 1 $sVType = "S" Case Else ;############### MSG2 - START ############### $sMsgHdr = FuncName($hFunc) & " :Line: " & @ScriptLineNumber & " :Error= " & @error $sMsg = "Comparison value must be a " & @CRLF & _ "1. Date in YYYY/MM/DD format " & @CRLF & _ "2. A string " & @CRLF & _ "3. A number " & @CRLF MsgBox(0, $sMsgHdr, $sMsg) Return SetError(1, 0, "") ;############### MSG2 - END ############### EndSelect $iCnt = 0 For $n1 = UBound($aArray) - 1 To $iOptionBase Step -1 If $iDim = 1 Then $sData = StringStripWS($aArray[$n1], 3) ElseIf $iDim = 2 Then $sData = StringStripWS($aArray[$n1][$iCol], 3) EndIf Select Case _DateIsValid($sData) = 1 $sDType = "D" Case IsNumber($sData) = 1 $sDType = "N" Case IsString($sData) = 1 $sDType = "S" Case Else $sDType = "U" EndSelect If _IsValueEmpty($sData) Then $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop ; $sDType = $sVType EndIf If Not _IsValueEmpty($sData) And $sDType <> $sVType Then $sDeleteIndex = $sDeleteIndex & $n1 & "," $iCnt += 1 ContinueLoop EndIf Select Case $sOperator = "EQ" Switch $sDType Case "D" $iDiff = _DateDiff("D", $vValue, $sData) If $iDiff = 0 Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop Case "S" If $sData = $vValue Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop Case "N" If $sData = $vValue Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop EndSwitch Case $sOperator = "NEQ" Switch $sDType Case "D" If $sData <> $vValue Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop Case "S" If $sData <> $vValue Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop Case "N" If $sData <> $vValue Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop EndSwitch Case $sOperator = "IN" Switch $sDType Case "S" If StringInStr($sData, $vValue) Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop EndSwitch Case $sOperator = "GT" Switch $sDType Case "N" If $sData > $vValue Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop Case "D" $iDiff = _DateDiff("D", $vValue, $sData) If $iDiff > 0 Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop EndSwitch Case $sOperator = "GE" Switch $sDType Case "N" If $sData >= $vValue Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop Case "D" $iDiff = _DateDiff("D", $vValue, $sData) If $iDiff >= 0 Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop EndSwitch Case $sOperator = "LT" Switch $sDType Case "N" If $sData < $vValue Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop Case "D" $iDiff = _DateDiff("D", $vValue, $sData) If $iDiff < 0 Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop EndSwitch Case $sOperator = "LE" Switch $sDType Case "N" If $sData <= $vValue Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop Case "D" $iDiff = _DateDiff("D", $vValue, $sData) If $iDiff <= 0 Then ContinueLoop EndIf $sDeleteIndex &= $n1 & "," $iCnt += 1 ContinueLoop EndSwitch EndSelect Next If $iCnt > 0 Then _DeleteArrayRows($aArray, $sDeleteIndex) EndIf EndFunc ;==>_ArrayFilter Thanks in advance.
      OldGuyWalking
    • MrCheese
      By MrCheese
      Hi guys,
      without including everything (unless you want it)
      I am copying data from a table in chrome and wanting to paste it into excel.
      Copying in Chrome works.
      I can paste it into the field i want by emulating goto -> ctrl V:
      WinActivate($dataload) WinWaitActive($dataload) Sleep(500) $oWorkbook1.Sheets("ItemReturn").Activate Sleep(500) $msg = "Measuring Sheet" conwrite() ttips2() Local Const $xlUp = -4162 With $oWorkbook1.ActiveSheet ; process active sheet $oRangeLast = .UsedRange.SpecialCells($xlCellTypeLastCell) ; get a Range that contains the last used cells $iRowCount = .Range(.Cells(1, 1), .Cells($oRangeLast.Row, "B")).Rows.Count ; get the the row count for the range starting in row/column 1 and ending at the last used row/column $iLastCell = .Cells($iRowCount + 1, "B").End($xlUp).Row ; start in the row following the last used row and move up to the first used cell in column "B" and grab this row number EndWith $NewStartCell = $iLastCell + 2 $msg = "moving to location" conwrite() ttips2() Sleep(250) Send("^g") WinWait("Go To") Sleep(100) Send("B" & $NewStartCell) Sleep(100) Send("{ENTER}") Sleep(500) Send("^v")  
      But, I want to use _excel_rangecopypaste, pasting from the clipboard
      _Excel_RangeCopyPaste($oWorkbook1.ActiveSheet, default, "B" & $NewStartCell,default,$xlPasteValuesAndNumberFormats) If @error Then Exit MsgBox($MB_SYSTEMMODAL, "Excel UDF: _Excel_RangeCopy Example 2", "Error pasting cells." & @CRLF & "@error = " & @error & ", @extended = " & @extended) however, this gives me error 4 , extended@:  -2147352567
      How can i fix this or find out how to debug this error?
       
      Thanks
    • Simpel
      By Simpel
      Hi.
      I try to figure out who is using a excel workbook which I can only open "read only". I use this code:
      #include <Array.au3> #include <Excel.au3> Local $sFile = ; excel file with path on a network drive Local $oExcel = _Excel_Open(True, True) Local $oTabelle = _Excel_BookOpen($oExcel, $sFile) Local $aUsers If IsObj($oTabelle) Then $aUsers = $oTabelle.UserStatus _ArrayDisplay($aUsers) EndIf If I am the one allowed to write to the excel file (I'm the first one who opened it) then I will get an array with myself:

      If my collegue opened the excel file first and I run the code I get the following error message:
      "H:\_Conrad lokal\Downloads\AutoIt3\_COX\Tests\test.au3" (9) : ==> The requested action with this object has failed.: $aUsers = $oTabelle.UserStatus $aUsers = $oTabelle^ ERROR The excel file is on a network drive. Is that's the problem?
      Regards, Conrad
×

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.