Jump to content
UEZ

[Solved] Create an array from filtered Excel sheet

Recommended Posts

SpecialCells: 64.8110058748907
RangeRead: 23518.2372043567
ArrayConcat: 3263.37492148338

 


Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites

Using the clipboard is the fastest version but clipboard will be overwritten:

Local $fTimer = TimerInit()

$oRange = $oWorkbook.ActiveSheet.Usedrange.SpecialCells($xlCellTypeVisible)
_Excel_RangeCopyPaste($oWorkbook.Activesheet, $oWorkbook.ActiveSheet.AutoFilter.Range)
$sClipboard_Excel = ClipGet() ;tab delimited
$aResult = _CSVSplit($sClipboard_Excel, @TAB)

ConsoleWrite(TimerDiff($fTimer) & @CRLF)

_ArrayDisplay($aResult)


; #FUNCTION# ====================================================================================================================
; Name...........: _CSVSplit
; Description ...: Converts a string in CSV format to a two dimensional array (see comments)
; Syntax.........: CSVSplit ( $aArray [, $sDelim ] )
; Parameters ....: $aArray  - The array to convert
;                  $sDelimiter  - Optional - Delimiter set to comma by default (see 2nd comment)
; Return values .: Success  - Returns a two dimensional array or a one dimensional array (see 1st comment)
;                  Failure  - Sets @error to:
;                 |@error = 1 - First parameter is not a valid string
;                 |@error = 2 - Second parameter is not a valid string
;                 |@error = 3 - Could not find suitable delimiter replacements
; Author ........: czardas
; Comments ......; Returns a one dimensional array if the input string does not contain the delimiter string
;                ; Some CSV formats use semicolon as a delimiter instead of a comma
;                ; Set the second parameter to @TAB To convert to TSV
; ===============================================================================================================================
Func _CSVSplit($string, $sDelimiter = ",") ; Parses csv string input and returns a one or two dimensional array
    If Not IsString($string) Or $string = "" Then Return SetError(1, 0, 0) ; Invalid string
    If Not IsString($sDelimiter) Or $sDelimiter = "" Then Return SetError(2, 0, 0) ; Invalid string

    $string = StringRegExpReplace($string, "[\r\n]+\z", "") ; [Line Added] Remove training breaks
    Local $iOverride = 63743, $asDelim[3] ; $asDelim => replacements for comma, new line and double quote
    For $i = 0 To 2
        $asDelim[$i] = __GetSubstitute($string, $iOverride) ; Choose a suitable substitution character
        If @error Then Return SetError(3, 0, 0) ; String contains too many unsuitable characters
    Next
    $iOverride = 0

    Local $aArray = StringRegExp($string, '\A[^"]+|("+[^"]+)|"+\z', 3) ; Split string using double quotes delim - largest match
    $string = ""

    Local $iBound = UBound($aArray)
    For $i = 0 To $iBound - 1
        $iOverride += StringInStr($aArray[$i], '"', 0, -1) ; Increment by the number of adjacent double quotes per element
        If Mod($iOverride + 2, 2) = 0 Then ; Acts as an on/off switch
            $aArray[$i] = StringReplace($aArray[$i], $sDelimiter, $asDelim[0]) ; Replace comma delimeters
            $aArray[$i] = StringRegExpReplace($aArray[$i], "(\r\n)|[\r\n]", $asDelim[1]) ; Replace new line delimeters
        EndIf
        $aArray[$i] = StringReplace($aArray[$i], '""', $asDelim[2]) ; Replace double quote pairs
        $aArray[$i] = StringReplace($aArray[$i], '"', '') ; Delete enclosing double quotes - not paired
        $aArray[$i] = StringReplace($aArray[$i], $asDelim[2], '"') ; Reintroduce double quote pairs as single characters
        $string &= $aArray[$i] ; Rebuild the string, which includes two different delimiters
    Next
    $iOverride = 0

    $aArray = StringSplit($string, $asDelim[1], 2) ; Split to get rows
    $iBound = UBound($aArray)
    Local $aCSV[$iBound][2], $aTemp
    For $i = 0 To $iBound - 1
        $aTemp = StringSplit($aArray[$i], $asDelim[0]) ; Split to get row items
        If Not @error Then
            If $aTemp[0] > $iOverride Then
                $iOverride = $aTemp[0]
                ReDim $aCSV[$iBound][$iOverride] ; Add columns to accomodate more items
            EndIf
        EndIf
        For $j = 1 To $aTemp[0]
            If StringLen($aTemp[$j]) Then
                If Not StringRegExp($aTemp[$j], '[^"]') Then ; Field only contains double quotes
                    $aTemp[$j] = StringTrimLeft($aTemp[$j], 1) ; Delete enclosing double quote single char
                EndIf
                $aCSV[$i][$j - 1] = $aTemp[$j] ; Populate each row
            EndIf
        Next
    Next

    If $iOverride > 1 Then
        Return $aCSV ; Multiple Columns
    Else
        For $i = 0 To $iBound - 1
            If StringLen($aArray[$i]) And (Not StringRegExp($aArray[$i], '[^"]')) Then ; Only contains double quotes
                $aArray[$i] = StringTrimLeft($aArray[$i], 1) ; Delete enclosing double quote single char
            EndIf
        Next
        Return $aArray ; Single column
    EndIf

EndFunc   ;==>_CSVSplit

; #INTERNAL_USE_ONLY# ===========================================================================================================
; Name...........: __GetSubstitute
; Description ...: Searches for a character to be used for substitution, ie one not contained within the input string
; Syntax.........: __GetSubstitute($string, ByRef $iCountdown)
; Parameters ....: $string   - The string of characters to avoid
;                  $iCountdown - The first code point to begin checking
; Return values .: Success   - Returns a suitable substitution character not found within the first parameter
;                  Failure   - Sets @error to 1 => No substitution character available
; Author ........: czardas
; Comments ......; This function is connected to the function _CSVSplit and was not intended for general use
;                  $iCountdown is returned ByRef to avoid selecting the same character on subsequent calls to this function
;                  Initially $iCountown should be passed with a value = 63743
; ===============================================================================================================================
Func __GetSubstitute($string, ByRef $iCountdown)
    If $iCountdown < 57344 Then Return SetError(1, 0, "") ; Out of options
    Local $sTestChar
    For $i = $iCountdown To 57344 Step -1
        $sTestChar = ChrW($i)
        $iCountdown -= 1
        If Not StringInStr($string, $sTestChar) Then
            Return $sTestChar
        EndIf
    Next
    Return SetError(1, 0, "") ; Out of options
EndFunc   ;==>__GetSubstitute

 

It takes around 800 ms to generate the array.

 

Edited by UEZ

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites

Using the internal transpose function makes the script much faster:

Local $hSpecialCells, $iSpecialCells = 0, $hRangeRead, $iRangeRead = 0, $hArrayConcat, $iArrayConcat = 0
$hSpecialCells = TimerInit()
$oRange = $oWorkbook.ActiveSheet.Usedrange.SpecialCells($xlCellTypeVisible)
$iSpecialCells = TimerDiff($hSpecialCells)
Local $aResult[1][$oRange.columns.Count], $aContent
; Read the data of all Ranges in the Area and concatenate the returned arrays.
For $oArea In $oRange.Areas
    $hRangeRead = TimerInit()
    $aContent = _Excel_RangeRead($oWorkbook, Default, $oArea, Default, True)
    $iRangeRead = $iRangeRead + TimerDiff($hRangeRead)
    $hArrayConcat = TimerInit()
    _ArrayConcatenate($aResult, $aContent)
    $iArrayConcat = $iArrayConcat + TimerDiff($hArrayConcat)
Next
MsgBox(0, "", "SpecialCells: " & $iSpecialCells & @CRLF & _
              "RangeRead: " & $iRangeRead & @CRLF & _
              "ArrayConcat: " & $iArrayConcat)

What do you get now?

Edited by water
Fixed: _Excel_RangeReadEX

My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2019-08-19 - Version 1.4.13.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2019-07-24 - Version 1.3.6.0) - Download - General Help & Support - Example Scripts - Wiki
Outlook Tools (NEW 2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
PowerPoint (2017-06-06 - Version 0.0.5.0) - Download - General Help & Support
Excel - Example Scripts - Wiki
Word - Wiki
Taks Scheduler (NEW 2019-09-14 - Version 0.5.0.0) - Download - General Help & Support - Wiki

Tutorials:
ADO - Wiki

 

Share this post


Link to post
Share on other sites

Drop the EX. Was for testonly. 


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2019-08-19 - Version 1.4.13.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2019-07-24 - Version 1.3.6.0) - Download - General Help & Support - Example Scripts - Wiki
Outlook Tools (NEW 2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
PowerPoint (2017-06-06 - Version 0.0.5.0) - Download - General Help & Support
Excel - Example Scripts - Wiki
Word - Wiki
Taks Scheduler (NEW 2019-09-14 - Version 0.5.0.0) - Download - General Help & Support - Wiki

Tutorials:
ADO - Wiki

 

Share this post


Link to post
Share on other sites

Much better now:

SpecialCells: 37.1230689016705
RangeRead: 879.62468028853
ArrayConcat: 3876.63181144516

I think _ArrayConcatenate can be replaced by a faster array function.

Edited by UEZ

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites

So it seems the internal Excel function "Transpose" has a problem :huh:


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2019-08-19 - Version 1.4.13.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2019-07-24 - Version 1.3.6.0) - Download - General Help & Support - Example Scripts - Wiki
Outlook Tools (NEW 2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
PowerPoint (2017-06-06 - Version 0.0.5.0) - Download - General Help & Support
Excel - Example Scripts - Wiki
Word - Wiki
Taks Scheduler (NEW 2019-09-14 - Version 0.5.0.0) - Download - General Help & Support - Wiki

Tutorials:
ADO - Wiki

 

Share this post


Link to post
Share on other sites

I think I will provide a solution to get the filtered rows in a single array and add that to the Excel wiki.

Done :)


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2019-08-19 - Version 1.4.13.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2019-07-24 - Version 1.3.6.0) - Download - General Help & Support - Example Scripts - Wiki
Outlook Tools (NEW 2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
PowerPoint (2017-06-06 - Version 0.0.5.0) - Download - General Help & Support
Excel - Example Scripts - Wiki
Word - Wiki
Taks Scheduler (NEW 2019-09-14 - Version 0.5.0.0) - Download - General Help & Support - Wiki

Tutorials:
ADO - Wiki

 

Share this post


Link to post
Share on other sites

I'm gettinbg this result for an individual _ArrayConcatenate function:

Local $hSpecialCells, $iSpecialCells = 0, $hRangeRead, $iRangeRead = 0, $hArrayConcat, $iArrayConcat = 0
$hSpecialCells = TimerInit()
$oRange = $oWorkbook.ActiveSheet.Usedrange.SpecialCells($xlCellTypeVisible)
$iSpecialCells = TimerDiff($hSpecialCells)

Local $iUBound1 = 10000
Local $iUBound2 = $oRange.columns.Count
Local $aResult[$iUBound1][$iUBound2], $aContent, $iX, $iY, $iPosY = 0
; Read the data of all Ranges in the Area and concatenate the returned arrays.
For $oArea In $oRange.Areas
    $hRangeRead = TimerInit()
    $aContent = _Excel_RangeRead($oWorkbook, Default, $oArea, Default, True)
    $iRangeRead += TimerDiff($hRangeRead)
    $hArrayConcat = TimerInit()
;~     _ArrayConcatenate($aResult, $aContent)
    For $iY = 0 To UBound($aContent) - 1
        For $iX = 0 To UBound($aContent, 2) - 1
            $aResult[$iPosY][$iX] = $aContent[$iY][$iX]
        Next
        $iPosY += 1
        If $iPosY = $iUBound1 Then
            ReDim $aResult[$iPosY + $iUBound1][$iUBound2]
            $iUBound1 += $iPosY
        EndIf
    Next
    $iArrayConcat += TimerDiff($hArrayConcat)
Next
ReDim $aResult[$iPosY][$iUBound2]

MsgBox(0, "", "SpecialCells: " & $iSpecialCells & @CRLF & _
              "RangeRead: " & $iRangeRead & @CRLF & _
              "ArrayConcat: " & $iArrayConcat)

 

SpecialCells: 400.998033492214
RangeRead: 885.23066438404
ArrayConcat: 94.1944913149327

 


Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • By VinMe
      I am unable to execute the below script, my requirement is to copy the content from active excel sheet and to display the same.
      Please let me know where i am missing!
      #include <Excel.au3>
      #include <MsgBoxConstants.au3>
      #include <Array.au3>
      #include <StringConstants.au3>
      Local $oExcel = _Excel_Open()
      $LastRow2 = $oExcel.UsedRange.Rows.Count
      $Tissue = _Excel_RangeRead($oExcel, Default, "E1:E" & $LastRow2)
      $TshNr = _Excel_RangeRead($oExcel, Default, "F1:F" & $LastRow2)
      _ArrayDisplay($Tissue)
      _ArrayDisplay($TshNr)
    • By jmp
      Script running good but error in line 7.
      When i run this script :
      #include <IE.au3> #include <Array.au3> $oIE = _IEAttach ("Shop") $oTable = _IETableGetCollection ($oIE, 1) $aTableData = _IETableWriteToArray ($oTable) For $inumber = 1 To UBound($aTableData) -1 $table = $aTableData[4][$inumber] MsgBox(0, "", $table) Next I got Error: array variable has incorrect number of subscripts or subscript dimension range exceeded
    • By nacerbaaziz
      Good morning guys, i hope that you're all well.
      guys, i have a problem and i hope that you can help me
      i've created an 3d array
      the array Contain a Categories info
      as folow
      $array[n][0][0] = Categorie name
      $array[n][0][1] = Categorie file path
      $array[n][0][2] = Categorie contents number
      $array[n][m][0] = link name
      $array[n][m][1] = link url
      $array[n][m][2] = link section name
      in my tool i want to add an option to delete a Category
      as you know the Categorie mean that must delete a region from the array
      when i tried to use _arrayDelete
      with the 2d array it work well
      but here i couldn't find any way to do that, can any one help me please?
      thanks in advance.
    • By nacerbaaziz
      hello sirs, please help me
      i tried to create a function that read a folder files to 3d array
      e.g
      $array[n][0][0] = ctName
      $array[n][0][1] = ctFilePath
      $array[n][0][2] = crtsections number
      $array[n][m][0] = KeyName
      $array[n][m][1] = KeyVal
      $array[n][m][2] = keySectionName
       
      that the array
      when i put one file into the folder all things work fine
      but when i put more than one file
      the last file worked fine but the others only the first key is showing
      please can you help me to correct this problem
      here is the example with the folder
      please accept my greetings
      and thanks in advance
       
      array3d.zip
×
×
  • Create New...