Jump to content
Sign in to follow this  
Bowmore

Fast Array Write To Excel

Recommended Posts

Bowmore

I have some scripts that use Excel to present the result of data validation tests and I was becoming a little frustrated at the length of time it was taking to write the data to excel, using _ExcelWriteSheetFromArray() function in the Excel.au3 UDF. So I thought I'd take a look and see if I could improve on it's performance. This was the result. This new function is 200 to 300 times faster than _ExcelWriteSheetFromArray() for large arrays (20000 plus elements) and in the region of 100 times faster for small arrays.

I have also included the option to write only part of the array to the spreadsheet as sometimes I do not need all of the array.

_ExcelSheetWriteFromArray

; #FUNCTION# ====================================================================================================================
; Name...........: _ExcelSheetWriteFromArray
; Description ...: Writes a 2D array, or part of, to a specified location in the active worksheet
; Syntax.........: _ExcelSheetWriteFromArray($oExcel, ByRef $aArray, $iExcelStartRow = Default, $iExcelStartCol = Default, $iArrayStartRow = Default, $iArrayStartCol = Default, $iArrayEndRow = Default, $iArrayEndCol = Default)
; Parameters ....: $oExcel - Excel object opened by a preceding call to _ExcelBookOpen() or _ExcelBookNew()
;                $aArray - The array ByRef to write data from (array is not modified)
;                $iExcelStartRow - The spreadsheet row to start writing the array to, default is 1
;                $iExcelStartCol - The spreadsheet column to start writing the array to, default is 1
;                $iArrayStartRow - Start row index of array to start writing data from, default is 0
;                $iArrayStartCol - Start column index of array to start writing data from, default is 0
;                $iArrayEndRow - End row index of array to stop writing data from, default is Ubound($aArray)
;                $iArrayEndCol - End column index of array to stop writing data from, default is Ubound($aArray,2)
; Return values .: Success   - Returns 1
;                Failure - Returns 0 and sets @error on errors:
;                |@error=1   - Excel object does not exist
;                |@error=2   - Not a 2D array
;                |@error=3   - End row greater than start row
;                |@error=4   - End col greater than start col
; Author ........: Bowmore
; Modified.......:
; Remarks .......: Only 2D arrays are currently handeled. Maxim cell content length is 255 characters
; Related .......:
; Link ..........:
; Example .......: Yes
; ===============================================================================================================================
Func _ExcelSheetWriteFromArray($oExcel, ByRef $aArray, $iExcelStartRow = Default, $iExcelStartCol = Default, $iArrayStartRow = Default, $iArrayStartCol = Default, $iArrayEndRow = Default, $iArrayEndCol = Default)
Local $iExcelEndRow = 1
Local $iExcelEndCol = 1

If Not IsObj($oExcel) Then Return SetError(1, 0, 0)
If UBound($aArray, 0) <> 2 Then Return SetError(2, 0, 0)
If $iExcelStartRow = Default Then $iExcelStartRow = 1
If $iExcelStartCol = Default Then $iExcelStartCol = 1
If $iArrayStartRow = Default Then $iArrayStartRow = 0
If $iArrayStartCol = Default Then $iArrayStartCol = 0
If $iArrayEndRow = Default Then $iArrayEndRow = UBound($aArray) - 1
If $iArrayEndCol = Default Then $iArrayEndCol = UBound($aArray, 2) - 1
If $iExcelStartRow < 1 Then $iExcelStartRow = 1
If $iExcelStartCol < 1 Then $iExcelStartCol = 1
If $iArrayStartRow < 0 Then $iArrayStartRow = 0
If $iArrayStartCol < 0 Then $iArrayStartCol = 0
If $iArrayEndRow < $iArrayStartRow Then Return SetError(3, 1, 0)
If $iArrayEndCol < $iArrayStartCol Then Return SetError(4, 1, 0)
$iExcelEndRow = $iExcelStartRow + $iArrayEndRow - $iArrayStartRow
$iExcelEndCol = $iExcelStartCol + $iArrayEndCol - $iArrayStartCol

; Check if only part of the array is to written to the speadsheet
If $iArrayStartRow <> 0 Or $iArrayStartCol <> 0 Or $iArrayEndRow <> UBound($aArray) - 1 Or $iArrayEndCol = UBound($aArray, 2) - 1 Then
;Copy specified array range to a temporary array
Local $aTemp[$iArrayEndRow - $iArrayStartRow + 1][$iArrayEndCol - $iArrayStartCol + 1]
Local $iRow = 0
Local $iCol = 0
For $i = $iArrayStartRow To $iArrayEndRow
$iCol = 0
For $j = $iArrayStartCol To $iArrayEndCol
$aTemp[$iRow][$iCol] = $aArray[$i][$j]
$iCol += 1
Next
$iRow += 1
Next
With $oExcel.ActiveSheet
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).Select
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).value = $oExcel.Application.WorksheetFunction.Transpose($aTemp)
EndWith
Else
With $oExcel.ActiveSheet
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).Select
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).value = $oExcel.Application.WorksheetFunction.Transpose($aArray)
EndWith
EndIf

EndFunc ;==>_ExcelSheetWriteFromArray

Simple Speed Test _ExcelSheetWriteFromArray

#include <excel.au3>

Test_Main()

Func Test_Main()
Local $sXlsFile1 = @ScriptDir & "Test1.xls"
Local $sXlsFile2 = @ScriptDir & "Test2.xls"
Local $oExcel = 0
Local $Tim = 0
Local $aData[5000][10]

For $i = 0 To 4999
$aData[$i][0] = "Y"
$aData[$i][1] = "DATA_TEST_ROW_0" & StringFormat("%03i", $i)
$aData[$i][2] = "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vestibulum ultrices est varius elit consequat sollicitudin mollis libero ultrices. Quisque lobortis lorem quis nunc vestibulum tincidunt. Duis eget urna sapien, nec egestas purus. Fusce massa nunc."
$aData[$i][3] = $i * 10
$aData[$i][4] = $i * 100
$aData[$i][5] = $aData[$i][3] / $aData[$i][4]
$aData[$i][6] = "Pass"
$aData[$i][7] = @YEAR & "-" & @MON & "-" & @MDAY & "T" & @HOUR & ":" & @MIN
$aData[$i][8] = "FAIL"
$aData[$i][9] = "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vestibulum ultrices est varius elit consequat sollicitudin mollis libero ultrices. Quisque lobortis lorem quis nunc vestibulum tincidunt. Duis eget urna sapien, nec egestas purus. Fusce massa nunc."
Next

; Speed Test of new function
$oExcel = _ExcelBookNew(0)
$Tim = TimerInit()
_ExcelSheetWriteFromArray($oExcel, $aData)
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : TimerDiff($Tim) = ' & TimerDiff($Tim) & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console
_ExcelBookSaveAs($oExcel, $sXlsFile1)
_ExcelBookClose($oExcel, 0, 0)

; Speed Test of old function
$oExcel = _ExcelBookNew(0)
$Tim = TimerInit()
_ExcelWriteSheetFromArray($oExcel, $aData)
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : TimerDiff($Tim) = ' & TimerDiff($Tim) & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console
_ExcelBookSaveAs($oExcel, $sXlsFile2)
_ExcelBookClose($oExcel, 0, 0)

EndFunc ;==>Test_Main


; #FUNCTION# ====================================================================================================================
; Name...........: _ExcelSheetWriteFromArray
; Description ...: Writes a 2D array, or part of, to a specified location in the active worksheet
; Syntax.........: _ExcelSheetWriteFromArray($oExcel, ByRef $aArray, $iExcelStartRow = Default, $iExcelStartCol = Default, $iArrayStartRow = Default, $iArrayStartCol = Default, $iArrayEndRow = Default, $iArrayEndCol = Default)
; Parameters ....: $oExcel - Excel object opened by a preceding call to _ExcelBookOpen() or _ExcelBookNew()
;                $aArray - The array ByRef to write data from (array is not modified)
;                $iExcelStartRow - The spreadsheet row to start writing the array to, default is 1
;                $iExcelStartCol - The spreadsheet column to start writing the array to, default is 1
;                $iArrayStartRow - Start row index of array to start writing data from, default is 0
;                $iArrayStartCol - Start column index of array to start writing data from, default is 0
;                $iArrayEndRow - End row index of array to stop writing data from, default is Ubound($aArray)
;                $iArrayEndCol - End column index of array to stop writing data from, default is Ubound($aArray,2)
; Return values .: Success   - Returns 1
;                Failure - Returns 0 and sets @error on errors:
;                |@error=1   - Excel object does not exist
;                |@error=2   - Not a 2D array
;                |@error=3   - End row greater than start row
;                |@error=4   - End col greater than start col
; Author ........: Bowmore
; Modified.......:
; Remarks .......: Only 2D arrays are currently handeled. Maxim cell content length is 255 characters
; Related .......:
; Link ..........:
; Example .......: Yes
; ===============================================================================================================================
Func _ExcelSheetWriteFromArray($oExcel, ByRef $aArray, $iExcelStartRow = Default, $iExcelStartCol = Default, $iArrayStartRow = Default, $iArrayStartCol = Default, $iArrayEndRow = Default, $iArrayEndCol = Default)
Local $iExcelEndRow = 1
Local $iExcelEndCol = 1

If Not IsObj($oExcel) Then Return SetError(1, 0, 0)
If UBound($aArray, 0) <> 2 Then Return SetError(2, 0, 0)
If $iExcelStartRow = Default Then $iExcelStartRow = 1
If $iExcelStartCol = Default Then $iExcelStartCol = 1
If $iArrayStartRow = Default Then $iArrayStartRow = 0
If $iArrayStartCol = Default Then $iArrayStartCol = 0
If $iArrayEndRow = Default Then $iArrayEndRow = UBound($aArray) - 1
If $iArrayEndCol = Default Then $iArrayEndCol = UBound($aArray, 2) - 1
If $iExcelStartRow < 1 Then $iExcelStartRow = 1
If $iExcelStartCol < 1 Then $iExcelStartCol = 1
If $iArrayStartRow < 0 Then $iArrayStartRow = 0
If $iArrayStartCol < 0 Then $iArrayStartCol = 0
If $iArrayEndRow < $iArrayStartRow Then Return SetError(3, 1, 0)
If $iArrayEndCol < $iArrayStartCol Then Return SetError(4, 1, 0)
$iExcelEndRow = $iExcelStartRow + $iArrayEndRow - $iArrayStartRow
$iExcelEndCol = $iExcelStartCol + $iArrayEndCol - $iArrayStartCol

; Check if only part of the array is to written to the speadsheet
If $iArrayStartRow <> 0 Or $iArrayStartCol <> 0 Or $iArrayEndRow <> UBound($aArray) - 1 Or $iArrayEndCol = UBound($aArray, 2) - 1 Then
;Copy specified array range to a temporary array
Local $aTemp[$iArrayEndRow - $iArrayStartRow + 1][$iArrayEndCol - $iArrayStartCol + 1]
Local $iRow = 0
Local $iCol = 0
For $i = $iArrayStartRow To $iArrayEndRow
$iCol = 0
For $j = $iArrayStartCol To $iArrayEndCol
$aTemp[$iRow][$iCol] = $aArray[$i][$j]
$iCol += 1
Next
$iRow += 1
Next
With $oExcel.ActiveSheet
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).Select
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).value = $oExcel.Application.WorksheetFunction.Transpose($aTemp)
EndWith
Else
With $oExcel.ActiveSheet
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).Select
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).value = $oExcel.Application.WorksheetFunction.Transpose($aArray)
EndWith
EndIf

EndFunc ;==>_ExcelSheetWriteFromArray
Edited by Bowmore
  • Like 1

"Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to build bigger and better idiots. So far, the universe is winning."- Rick Cook

Share this post


Link to post
Share on other sites
Spiff59

There is a ticket in BugTracker for a new version of _ExcelReadSheetFromArray() that also eliminates the nested loops that individually access Excel cells in favor of using the Range.Cells method. It shows a similar speed increase.

I would think if you reverted back to the production function name, switched both the $iRowBase ($iArrayStartRow) and $iColBase ($iArrayStartCol) parameters back to defaulting to 1, instead of 0, and maintained the existing error return values (with some of your own additions), that you'd have a backward-compatible function (with 2 new parms) that could replace the one in the production UDF?

Share this post


Link to post
Share on other sites
Bowmore

Thanks for the suggestions. Although I wrote it to meet my own specific needs I would make sense to change the version I posted hear to make it compatible with the existing _ExcelWriteSheetFromArray(). I'll change it in the next couple of days when I have half an hour to make the changes and test it properly.


"Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to build bigger and better idiots. So far, the universe is winning."- Rick Cook

Share this post


Link to post
Share on other sites
tempman

Hi,

Your Fast Array Write To Excel Script rocks, but only for 65536 rows, if I put one more 65537 I get an error:

: ==> The requested action with this object has failed.:

.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).value = $oExcel.Application.WorksheetFunction.Transpose($aTemp)

.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).value = $oExcel.Application.WorksheetFunction.Transpose($aTemp)^ ERROR

can that be fixed?

Share this post


Link to post
Share on other sites
water

I'm about to rewrite the Excel UDF that comes with AutoIt. It's an early Alpha version but your problem has been solved by GMX's (it is named _Excel_RangeRead).

If you can test, please post the results in the Excel Rewrite thread.


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2018-12-03 - Version 1.4.11.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (2018-10-31 - Version 1.3.4.1) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
PowerPoint (2017-06-06 - Version 0.0.5.0) - Download - General Help & Support
Excel - Example Scripts - Wiki
Word - Wiki
 
Tutorials:

ADO - Wiki

 

Share this post


Link to post
Share on other sites
GMK

Actually, I need to make changes to the _Excel_RangeWrite function as well. Will post it in the Excel Rewrite thread as soon as I've completed it.

Share this post


Link to post
Share on other sites
tempman

I agree with GMK, I have tested _ExcelSheetWriteFromArray with Excel Rewrite.au3 and 65536 issue is still present.

Waiting for changes to the _Excel_RangeWrite function...

Share this post


Link to post
Share on other sites
BrewManNH

There appears to be a limit of 65536 cells when using the .transpose method at least up to Excel 2007, might even go beyond that, but I'm not sure.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
tempman

I find this post on mrexcel.com forum: http://www.mrexcel.com/forum/excel-questions/555073-limitation-range-size-transpose-function.html#post2742571

(The Transpose function just doesn't work with >65536, just as SpecialCells or AdvancedFilter etc.methods sometimes don't work with ranges above a certain size. You'd have to ask the Microsoft coders why they haven't updated these to allow for the larger sheet size of Excel 2007 and subsequent versions.)

I really don't know is this comment form mrexcel.com forum true or false, but I find one not so pretty but working solution for example in our topic

#include <excel.au3>

Test_Main()

Func Test_Main()
Local $oExcel = 0
Local $Tim = 0
Local $aData[65536][1]
Local $aData1[65536][1]

For $i = 0 To 65535
$aData[$i][0] = "DATA_TEST_ROW_0" & StringFormat("%03i", $i+1)
Next

For $j = 0 To 65535
$aData1[$j][0] = "DATA_TEST_ROW_0" & StringFormat("%03i", $j+65537)
Next

; Speed Test of new function
$oExcel = _ExcelBookNew()
$Tim = TimerInit()
_ExcelSheetWriteFromArray($oExcel, $aData)
_ExcelSheetWriteFromArray($oExcel, $aData1, 65537, 1)
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : TimerDiff($Tim) = ' & TimerDiff($Tim) & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

EndFunc ;==>Test_Main


; #FUNCTION# ====================================================================================================================
; Name...........: _ExcelSheetWriteFromArray
; Description ...: Writes a 2D array, or part of, to a specified location in the active worksheet
; Syntax.........: _ExcelSheetWriteFromArray($oExcel, ByRef $aArray, $iExcelStartRow = Default, $iExcelStartCol = Default, $iArrayStartRow = Default, $iArrayStartCol = Default, $iArrayEndRow = Default, $iArrayEndCol = Default)
; Parameters ....: $oExcel - Excel object opened by a preceding call to _ExcelBookOpen() or _ExcelBookNew()
;            $aArray - The array ByRef to write data from (array is not modified)
;            $iExcelStartRow - The spreadsheet row to start writing the array to, default is 1
;            $iExcelStartCol - The spreadsheet column to start writing the array to, default is 1
;            $iArrayStartRow - Start row index of array to start writing data from, default is 0
;            $iArrayStartCol - Start column index of array to start writing data from, default is 0
;            $iArrayEndRow - End row index of array to stop writing data from, default is Ubound($aArray)
;            $iArrayEndCol - End column index of array to stop writing data from, default is Ubound($aArray,2)
; Return values .: Success - Returns 1
;            Failure - Returns 0 and sets @error on errors:
;            |@error=1 - Excel object does not exist
;            |@error=2 - Not a 2D array
;            |@error=3 - End row greater than start row
;            |@error=4 - End col greater than start col
; Author ........: Bowmore
; Modified.......:
; Remarks .......: Only 2D arrays are currently handeled. Maxim cell content length is 255 characters
; Related .......:
; Link ..........:
; Example .......: Yes
; ===============================================================================================================================
Func _ExcelSheetWriteFromArray($oExcel, ByRef $aArray, $iExcelStartRow = Default, $iExcelStartCol = Default, $iArrayStartRow = Default, $iArrayStartCol = Default, $iArrayEndRow = Default, $iArrayEndCol = Default)
Local $iExcelEndRow = 1
Local $iExcelEndCol = 1

If Not IsObj($oExcel) Then Return SetError(1, 0, 0)
If UBound($aArray, 0) <> 2 Then Return SetError(2, 0, 0)
If $iExcelStartRow = Default Then $iExcelStartRow = 1
If $iExcelStartCol = Default Then $iExcelStartCol = 1
If $iArrayStartRow = Default Then $iArrayStartRow = 0
If $iArrayStartCol = Default Then $iArrayStartCol = 0
If $iArrayEndRow = Default Then $iArrayEndRow = UBound($aArray) - 1
If $iArrayEndCol = Default Then $iArrayEndCol = UBound($aArray, 2) - 1
If $iExcelStartRow < 1 Then $iExcelStartRow = 1
If $iExcelStartCol < 1 Then $iExcelStartCol = 1
If $iArrayStartRow < 0 Then $iArrayStartRow = 0
If $iArrayStartCol < 0 Then $iArrayStartCol = 0
If $iArrayEndRow < $iArrayStartRow Then Return SetError(3, 1, 0)
If $iArrayEndCol < $iArrayStartCol Then Return SetError(4, 1, 0)
$iExcelEndRow = $iExcelStartRow + $iArrayEndRow - $iArrayStartRow
$iExcelEndCol = $iExcelStartCol + $iArrayEndCol - $iArrayStartCol

; Check if only part of the array is to written to the speadsheet
If $iArrayStartRow <> 0 Or $iArrayStartCol <> 0 Or $iArrayEndRow <> UBound($aArray) - 1 Or $iArrayEndCol = UBound($aArray, 2) - 1 Then
;Copy specified array range to a temporary array
Local $aTemp[$iArrayEndRow - $iArrayStartRow + 1][$iArrayEndCol - $iArrayStartCol + 1]
Local $iRow = 0
Local $iCol = 0
For $i = $iArrayStartRow To $iArrayEndRow
$iCol = 0
For $j = $iArrayStartCol To $iArrayEndCol
$aTemp[$iRow][$iCol] = $aArray[$i][$j]
$iCol += 1
Next
$iRow += 1
Next
With $oExcel.ActiveSheet
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).Select
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).value = $oExcel.Application.WorksheetFunction.Transpose($aTemp)
EndWith
Else
With $oExcel.ActiveSheet
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).Select
.Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).value = $oExcel.Application.WorksheetFunction.Transpose($aArray)
EndWith
EndIf

EndFunc ;==>_ExcelSheetWriteFromArray

Share this post


Link to post
Share on other sites
BrewManNH

A better way would be to do something like this:

#include <excel.au3>

Test_Main()


; #FUNCTION# ====================================================================================================================
; Name...........: _ExcelSheetWriteFromArray
; Description ...: Writes a 2D array, or part of, to a specified location in the active worksheet
; Syntax.........: _ExcelSheetWriteFromArray($oExcel, ByRef $aArray, $iExcelStartRow = Default, $iExcelStartCol = Default, $iArrayStartRow = Default, $iArrayStartCol = Default, $iArrayEndRow = Default, $iArrayEndCol = Default)
; Parameters ....: $oExcel - Excel object opened by a preceding call to _ExcelBookOpen() or _ExcelBookNew()
;            $aArray - The array ByRef to write data from (array is not modified)
;            $iExcelStartRow - The spreadsheet row to start writing the array to, default is 1
;            $iExcelStartCol - The spreadsheet column to start writing the array to, default is 1
;            $iArrayStartRow - Start row index of array to start writing data from, default is 0
;            $iArrayStartCol - Start column index of array to start writing data from, default is 0
;            $iArrayEndRow - End row index of array to stop writing data from, default is Ubound($aArray)
;            $iArrayEndCol - End column index of array to stop writing data from, default is Ubound($aArray,2)
; Return values .: Success - Returns 1
;            Failure - Returns 0 and sets @error on errors:
;            |@error=1 - Excel object does not exist
;            |@error=2 - Not a 2D array
;            |@error=3 - End row greater than start row
;            |@error=4 - End col greater than start col
; Author ........: Bowmore
; Modified.......:
; Remarks .......: Only 2D arrays are currently handeled. Maxim cell content length is 255 characters
; Related .......:
; Link ..........:
; Example .......: Yes
; ===============================================================================================================================
Func _ExcelSheetWriteFromArray($oExcel, ByRef $aArray, $iExcelStartRow = Default, $iExcelStartCol = Default, $iArrayStartRow = Default, $iArrayStartCol = Default, $iArrayEndRow = Default, $iArrayEndCol = Default)
    Local $iExcelEndRow = 1
    Local $iExcelEndCol = 1

    If Not IsObj($oExcel) Then Return SetError(1, 0, 0)
    If UBound($aArray, 0) <> 2 Then Return SetError(2, 0, 0)
    If $iExcelStartRow = Default Then $iExcelStartRow = 1
    If $iExcelStartCol = Default Then $iExcelStartCol = 1
    If $iArrayStartRow = Default Then $iArrayStartRow = 0
    If $iArrayStartCol = Default Then $iArrayStartCol = 0
    If $iArrayEndRow = Default Then $iArrayEndRow = UBound($aArray) - 1
    If $iArrayEndCol = Default Then $iArrayEndCol = UBound($aArray, 2) - 1
    If $iExcelStartRow < 1 Then $iExcelStartRow = 1
    If $iExcelStartCol < 1 Then $iExcelStartCol = 1
    If $iArrayStartRow < 0 Then $iArrayStartRow = 0
    If $iArrayStartCol < 0 Then $iArrayStartCol = 0
    If $iArrayEndRow < $iArrayStartRow Then Return SetError(3, 1, 0)
    If $iArrayEndCol < $iArrayStartCol Then Return SetError(4, 1, 0)
    If $iArrayEndRow > UBound($aArray) - 1 Then $iArrayEndRow = UBound($aArray) - 1 ; <<<<<<<<<<<<<<< added this line, because there's no check to see if the $iArrayEndRow is larger than the end of the array
    $iExcelEndRow = $iExcelStartRow + $iArrayEndRow - $iArrayStartRow
    $iExcelEndCol = $iExcelStartCol + $iArrayEndCol - $iArrayStartCol

    ; Check if only part of the array is to written to the speadsheet
    If $iArrayStartRow <> 0 Or $iArrayStartCol <> 0 Or $iArrayEndRow <> UBound($aArray) - 1 Or $iArrayEndCol = UBound($aArray, 2) - 1 Then
        ;Copy specified array range to a temporary array
        Local $aTemp[$iArrayEndRow - $iArrayStartRow + 1][$iArrayEndCol - $iArrayStartCol + 1]
        Local $iRow = 0
        Local $iCol = 0
        For $i = $iArrayStartRow To $iArrayEndRow
            $iCol = 0
            For $j = $iArrayStartCol To $iArrayEndCol
                $aTemp[$iRow][$iCol] = $aArray[$i][$j]
                $iCol += 1
            Next
            $iRow += 1
        Next
        With $oExcel.ActiveSheet
            .Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).Select
            .Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).value = $oExcel.Application.WorksheetFunction.Transpose($aTemp)
        EndWith
    Else
        With $oExcel.ActiveSheet
            .Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).Select
            .Range(.Cells($iExcelStartRow, $iExcelStartCol), .Cells($iExcelEndRow, $iExcelEndCol)).value = $oExcel.Application.WorksheetFunction.Transpose($aArray)
        EndWith
    EndIf

EndFunc   ;==>_ExcelSheetWriteFromArray

Func Test_Main()
    Local $oExcel = 0
    Local $Tim = 0
    Local $aData[131072][1]

    For $i = 0 To 131071
        $aData[$i][0] = "DATA_TEST_ROW_0" & StringFormat("%03i", $i + 1)
    Next
    ; Speed Test of new function
    $oExcel = _ExcelBookNew()
    $Tim = TimerInit()
    ; This only writes 65535 elements of the array at a time, if your array is larger than that, then it will split it up
    ; for you.
    For $loop = 0 To UBound($aData) - 1 Step 65536 
        _ExcelSheetWriteFromArray($oExcel, $aData, $loop + 1, 0, $loop, 0, $Loop + 65535)
    Next
    ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : TimerDiff($Tim) = ' & TimerDiff($Tim) & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

EndFunc   ;==>Test_Main

This rewrites the test script that tempman posted above and also corrects a bug in the _ExcelSheetWriteFromArray function if you send it an end row of the array that is beyond the UBound dimension of the array.

This will write 65535 rows to the spreadsheet until you've hit the end of the array, and the array can be of any size either larger or smaller than 65535 rows.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
GMK
updated.

Share this post


Link to post
Share on other sites
water

Thanks GMK!

I hope to release a new alpha version with your changes quite soon.

Edited by water

My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2018-12-03 - Version 1.4.11.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (2018-10-31 - Version 1.3.4.1) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
PowerPoint (2017-06-06 - Version 0.0.5.0) - Download - General Help & Support
Excel - Example Scripts - Wiki
Word - Wiki
 
Tutorials:

ADO - Wiki

 

Share this post


Link to post
Share on other sites
water

I posted a revised version of _Excel_RangeWrite for you to test.

Would like to hear from Excel 2003 testers as I only can test with Excel 2010.


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2018-12-03 - Version 1.4.11.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (2018-10-31 - Version 1.3.4.1) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
PowerPoint (2017-06-06 - Version 0.0.5.0) - Download - General Help & Support
Excel - Example Scripts - Wiki
Word - Wiki
 
Tutorials:

ADO - Wiki

 

Share this post


Link to post
Share on other sites
DeDee

Hi

Is there a script for Fast Array Write from Excel? I see that it takes a lot of time for this as well - when using big Excel files.

Thank you!

Share this post


Link to post
Share on other sites
water

Welcome to AutoIt and the forum,

the rewrite of the Excel UDF I'm currently working on has a _Excel_RangeRead function too.

Please check the for the latest download. But be aware that it is still an early Alpha version!

If you search the forum you will find some other fast _ExcelReadSheetToArray functions too.


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2018-12-03 - Version 1.4.11.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (2018-10-31 - Version 1.3.4.1) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
PowerPoint (2017-06-06 - Version 0.0.5.0) - Download - General Help & Support
Excel - Example Scripts - Wiki
Word - Wiki
 
Tutorials:

ADO - Wiki

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Similar Content

    • robertocm
      By robertocm
      I'm using this for replacing text strings in the VBProject of all excel files in a folder and subfolders.
      I have the same text string in several lines and those lines could have some differences between files: then not feasible for .ReplaceLine method
      I'm not interested in placing all the code in a xla AddIn, because the code is similar but not exactly the same in all the files.
       
      Opt("WinTitleMatchMode", 2) ;1=start, 2=subStr, 3=exact, 4=advanced, -1 to -4=Nocase #include <File.au3> #include <WinAPIFiles.au3> #include <Excel.au3> $oMyError = ObjEvent("AutoIt.Error", "ErrFunc") ;Install a custom error handler Global $iEventError ; to be checked to know if com error occurs. Must be reset after handling. Global Const $sMessage = "Select Folder" Global $sFileSelectFolder = FileSelectFolder($sMessage, "") If @error Then MsgBox(0, "", "No folder was selected.") Exit EndIf Global $bFileOpen ;Look for excel files in selected directory and all subdirectories Global $aFileList = _FileListToArrayRec($sFileSelectFolder, "*.xlsm", $FLTAR_FILES, $FLTAR_RECUR, $FLTAR_NOSORT, $FLTAR_FULLPATH) If Not @error Then Local $oAppl = _Excel_Open(Default, Default, False, Default, True) ;If @error Then Exit MsgBox(0, "Error", "Error _Excel_Open" & @CRLF & "@error = " & @error & ", @extended = " & @extended) For $i = 1 To $aFileList[0] $bFileOpen = _WinAPI_FileInUse($aFileList[$i]) If $bFileOpen = 0 Then ;ShellExecute($aFileList[$i]) Local $oWorkbook = _Excel_BookOpen($oAppl, $aFileList[$i]) ;If @error Then Exit MsgBox(0, "Error", "Error _Excel_BookOpen: " & $sFilePath & @CRLF & "@error = " & @error & ", @extended = " & @extended) Local $oProject = $oWorkbook.VBProject ;From: Adapt VBA in a workbook using VBA / http://www.snb-vba.eu/index_en.html ;2.7.2 Macromodule delete With $oProject .VBComponents.Remove(.VBComponents("SplashText")) If $iEventError Then Consolewrite("SplashText Form not found: " & $aFileList[$i] & @CRLF) $iEventError = 0 ; Reset after displaying a COM Error occurred EndIf .VBComponents.Import("C:\Documents and Settings\XP\Escritorio\PLANTILLAS_EXPORT\SplashText.frm") EndWith ;3.2.1.8 Macro: delete With $oProject.VBComponents("Actual").CodeModule ;3.2.1.2 Macro: find If .Find("Sub Check_NumPed(", 1, 1, -1, -1) Then ;Note: using '+ 1' at the end of the line because i'm used to add an empty line between procedures (see vba help for ProcCountLines) .DeleteLines( .ProcStartLine("Check_NumPed", 0), .ProcCountLines("Check_NumPed", 0) + 1) EndIf EndWith ;Check if range name exists. If not create named ranges If Not IsObj($oWorkbook.Sheets("DATOS").Evaluate("Booking_DestPort")) Then ;If Not IsObj($oWorkbook.Sheets("DATOS").Range("Booking_DestPort")) Then If $oWorkbook.Sheets("DATOS").Range("AC7").value = "DestPort" Then $oWorkbook.Names.Add("Booking_DestPort", "=DATOS!$AC$8") Else ConsoleWrite("-> Not: 'DestPort' in AC7" & @TAB & $aFileList[$i] & @CRLF) EndIf If $oWorkbook.Sheets("DATOS").Range("AD7").value = "FinalDest" Then $oWorkbook.Names.Add("Booking_FinalDest", "=DATOS!$AD$8") Else ConsoleWrite("-> Not: 'FinalDest' in AD7" & @TAB & $aFileList[$i] & @CRLF) EndIf EndIf ;Open VBE Editor (like Alt+F11) $oAppl.VBE.MainWindow.Visible = True ;$oAppl.VBE.Windows("Inmediato").Visible = True ;https://www.autoitscript.com/forum/topic/77545-resolved-vbaofficeexcel-experts/ ;Spiff59, Aug 2008 ;Local $oModules = $oProject.VBComponents ;Local $oModules = $oWorkbook.VBProject.VBComponents ;$oModules.Item(1).CodeModule.CodePane.Show ;$oModules.Item(1).Activate ; With $oModules.Item($y).CodeModule ; .ReplaceLine (1 , "Sub SpellCheck()") ; .DeleteLines (10, 1) ; .InsertLines (7 , "TEST") ; EndWith ;Wait 30 seconds for the window to appear. Local $hWnd = WinWait("Microsoft Visual Basic - ", "Proyecto - VBAProjec", 30) WinActivate($hWnd) WinWaitActive($hWnd, "", 30) If WinActive($hWnd, "") Then ;Sleep(100) ;Send("{F7}") $oProject.VBComponents("Actual").Activate ;Wait 30 seconds for the window to appear. Local $hWnd2 = WinWait(" - [Actual (Código)]", "Proyecto - VBAProject", 30) WinActivate($hWnd2) WinWaitActive($hWnd2, "", 30) ;First Replace If WinActive($hWnd2, "") Then Send("{CTRLDOWN}h{CTRLUP}") ;Wait 30 seconds for the window to appear. Local $hWnd3 = WinWait("Reemplazar", "&Procedimiento actua", 30) WinActivate($hWnd3) WinWaitActive($hWnd3, "", 30) Sleep(100) Send('Sheets("DATOS").Range("AC8")') Sleep(200) Send("{TAB}") Sleep(100) ;Send("{DEL}") Send('Range("Booking_DestPort")') Sleep(200) ControlClick("Reemplazar", "", "[ID:4892]") Sleep(100) Send("{ALTDOWN}z{ALTUP}") Local $hWnd4 = WinWait("Microsoft Visual Basic", "Se ha buscado en la ", 2) ;WinActivate($hWnd4) ;WinWaitActive($hWnd4, "", 3) If WinActive($hWnd4, "") Then Sleep(100) Send("{SPACE}") Else Consolewrite("Not found 1: " & $aFileList[$i] & @CRLF) Local $hWnd5 = WinWait("Microsoft Visual Basic", "No se encontró", 0) ;WinActivate($hWnd5) ;WinWaitActive($hWnd5, "", 2) Sleep(100) Send("{SPACE}") EndIf Sleep(100) If WinActive($hWnd3, "") Then ;Alt+F4 Send("!{F4}") Sleep(100) EndIf EndIf ;Second Replace If WinActive($hWnd2, "") Then Send("{CTRLDOWN}h{CTRLUP}") ;Wait 30 seconds for the window to appear. Local $hWnd3 = WinWait("Reemplazar", "&Procedimiento actua", 30) WinActivate($hWnd3) WinWaitActive($hWnd3, "", 30) Sleep(100) Send('Sheets("DATOS").Range("AD8")') Sleep(200) Send("{TAB}") Sleep(100) Send('Range("Booking_FinalDest")') Sleep(200) ControlClick("Reemplazar", "", "[ID:4892]") Sleep(100) Send("{ALTDOWN}z{ALTUP}") Local $hWnd4 = WinWait("Microsoft Visual Basic", "Se ha buscado en la ", 2) ;WinActivate($hWnd4) ;WinWaitActive($hWnd4, "", 3) If WinActive($hWnd4, "") Then Sleep(100) Send("{SPACE}") Else Consolewrite("Not found 2: " & $aFileList[$i] & @CRLF) Local $hWnd5 = WinWait("Microsoft Visual Basic", "No se encontró", 0) ;WinActivate($hWnd5) ;WinWaitActive($hWnd5, "", 2) Sleep(100) Send("{SPACE}") EndIf Sleep(100) If WinActive($hWnd3, "") Then ;Alt+F4 Send("!{F4}") Sleep(100) EndIf EndIf ;Close VBE Editor If WinActive($hWnd2, "") Then $oAppl.VBE.ActiveWindow.Close ;Send("^{F4}") ;Sleep(100) ;Send("^s") $oAppl.VBE.MainWindow.Visible = False ;Sleep(100) ;Send("!{F4}") EndIf EndIf _Excel_BookClose($oWorkbook, True) ;If @error Then Exit MsgBox(0, "Error", "Error _Excel_BookClose: " & $sFilePath & @CRLF & "@error = " & @error & ", @extended = " & @extended) EndIf Next Else MsgBox(16, "Error", "No files were found in the folder specified.") EndIf _Excel_Close($oAppl) ;If @error Then Exit MsgBox(0, "Error", "Error _Excel_Close" & @CRLF & "@error = " & @error & ", @extended = " & @extended) ;This is a custom error handler Func ErrFunc() $HexNumber = Hex($oMyError.number, 8) ;~ MsgBox(0, "", "We intercepted a COM Error !" & @CRLF & _ ;~ "Number is: " & $HexNumber & @CRLF & _ ;~ "WinDescription is: " & $oMyError.windescription) ConsoleWrite("-> We intercepted a COM Error !" & @CRLF & _ "-> err.number is: " & @TAB & $HexNumber & @CRLF & _ "-> err.source: " & @TAB & $oMyError.source & @CRLF & _ "-> err.windescription: " & @TAB & $oMyError.windescription & _ "-> err.scriptline is: " & @TAB & $oMyError.scriptline & @CRLF) $iEventError = 1 ; Use to check when a COM Error occurs EndFunc ;==>ErrFunc  
    • lattey
      By lattey
      hi,
      i have checkboxes and each checkbox that checked, i put in array. 
      now, im stuck on how to loop the checked array and store in in one variable. what i can do now, is only write the result into a text file. 
      below is the code:
      #include <GUIConstantsEx.au3> ;~ #include <MsgBoxConstants.au3> #include <ButtonConstants.au3> #include <Array.au3> Global $Count = 3 Global $CheckBoxP[$Count] Global $step[$Count] global $array1[1] Global $ExitResult $hGUI = GUICreate("Summary Steps", 500, 400) GUISetFont(12, 400, "Tahoma") GUICtrlCreateLabel( "Please Select the Summary Steps for Script Check", 70, 20) GUISetFont(10, 400, "Tahoma") Global $array_Pstep[3] = ["fix2","fix1","fix3"] global $step[3] = ["2","3","4"] $Spacing = 50 For $i = 0 To UBound($array_Pstep) - 1 $CheckBoxP[$i] = GUICtrlCreateCheckbox($array_Pstep[$i], 80, $Spacing + (20 * $i), 65, 17) Next $submit = GUICtrlCreateButton("Submit",180, 280, 80, 30) $exit = GUICtrlCreateButton("Exit",180, 320, 80, 30) GUISetState() While 1 $Msg = GUIGetMsg() Select case $Msg=$submit For $i = 0 To $Count - 1 If GUICtrlRead($CheckBoxP[$i]) = $GUI_CHECKED Then _ArrayAdd($array1, $step[$i]) EndIf Next Global $logfilerray = @WorkingDir & "\checkedlist.txt" FileDelete ($logfilerray) Global $readlogfile = FileOpen($logfilerray,1) for $a = 1 to UBound($array1) - 1 ;~ $var=$array1[$a] FileWriteLine($readlogfile,$array1[$a]) Next FileClose($readlogfile) Exit case $Msg=$exit $ExitResult = MsgBox(1,"Summary Step", "Continue to Exit ?") if $ExitResult = 1 Then ;ok Exit EndIf Exit EndSelect WEnd  
    • gahhon
      By gahhon
      Hi Guys,
      I was trying to read some data from the excel file and without opening the file. But I tried a lot of methods, it still open the file.
      And also, I am able to capture the ColumnA value but not Column B.
      Thanks for advance information.
      Global $oDataA, $oDataB Call ("ExcelRead", "B2", "C2") Func ExcelRead($oColumnA, $oColumnB) Local $oPath = @ScriptDir & "\MyFile.xlsx" Local $oExcel = _Excel_Open() Local $oWorkbook = _Excel_BookOpen($oExcel, $oPath, 1, 0) $oDataA = _Excel_RangeRead($oWorkbook, "Sheet 1", $oColumnA) $oDataB = _Excel_RangeRead($oWorkbook, "Sheet 1", $oColumnB) MsgBox(0, "Test Value", $oDataA & ", " & $oDataB) EndFunc  
    • omicron
      By omicron
      How do you perform a nested loop function with a multidimensional array from 2 lists.
      for i in list1
      (open file) extract variable
          while open for i in list 2
          (open file2) extract variable
       
      var1 + var2 = (search term)

      The list sizes will more than likely consist of different lengths.
       
      What is the best approach to accomplishing this method?
             
    • omicron
      By omicron
      Hello!

      I am working on a function that I am just getting lost on. The goal is a multiple nested loop.

      Here are the steps:
      Contents of file1.txt::
      [topic] var1=Name var2=OtherName var3=SomeotheName Contents of file2.txt::
      [subTopic] top=sub1 top2=sub2 top3=sub3 The Shell I am working from::
      #include <file.au3> $file = "c:\yourfile.txt" FileOpen($file, 0) For $i = 1 to _FileCountLines($file) $line = FileReadLine($file, $i) msgbox(0,'','the line ' & $i & ' is ' & $line) Next FileClose($file) Understanding however that the "msgbox" needs to then become a variable. in example the following::
      $file = "c:\yourfile.txt" FileOpen($file, 0) While true( prog.exe is running && "WinName" is open) do For $i = 1 to _FileCountLines($file) $line = FileReadLine($file, $i) ;Open File to log "current location of file 1" FileWriteLine ("filename", $i & ' is ' & $line) var = $line Next $file2 = "c:\yourfile.txt" FileOpen($file, 0) For $i = 1 to _FileCountLines($file) $line = FileReadLine($file, $i) ; OpenFile to log "Current location of file 2" FileWriteLine ("filename", $i & ' is ' & $line) Next FileClose($file2) FileClose($file) The goal in written form is the following ::

      While in "OpenWindow"
          read from file 1 starting at line 1 until end of file.
         file 1 is a list of names to be searched.
         With $line selected, add this element to the element in file 2.
       
      The search of a variables in list 1 and list 2 differ on the amount of posts that day. (This is not a web based platform, it is a game) I need to search 2 names and take a screenshot of the out put. The sizes of the names list depend on the activity of names at the time of search.
      This loop continues until all the names from both lists have been searched. Mostly in the format of::
      File1= item
      File2= Vendor
       
      Item + Vendor  ( Capture screen, scroll) -- Not sure how to detect if I need to scroll)
       
      Thank you for your help and support!
×