Idea(s) for _ArrayUnique that returns counts - (Moved)

[qwert]

I have need for an inventory of the multiple occurrences of entries in an array.

_ArrayUnique works well to produce an array of entries. But in looking at the code in the UDF, I see that it uses a Scripting.Dictionary object that I not at all familiar with.

Does anyone have a suggestion of how to adapt the _ArrayUnique code? Or should I simply post-process in my own script to count occurrences? I ask because this seems like some standard solution could be useful in a variety of situations.

Thanks in advance for any help.

[JLogan3o13]

Moved to the appropriate forum, as the Developer General Discussion forum very clearly states:

Quote

General development and scripting discussions. If it's super geeky and you don't know where to put it - it's probably here.

---

Do not create AutoIt-related topics here, use the AutoIt General Help and Support or AutoIt Technical Discussion forums.

Moderation Team

[Nine]

@qwert Scripting Dictionary is a very powerful class.  Look at this document here,  There is only few methods and properties, so it is very straightforward to use.  It would be useful for you especially if your array is 1D.  I tested the class over a standard array, and for the use you intend to have, I believe it would be a lot faster.

 

[qwert]

Thanks for the link. I will investigate. object.Count looks promising. And, yes, my array is 1D.

[Subz]

You could use _ArrayUnique and then loop through the results and use _ArrayFindAll to get the count (assumes you haven't destroyed the original array).  Alternatively create a custom _ArrayUnique function:

nb: Tried to explain the process but, it was just easier for me to create an example, just compare to the original function to see the changes.

expandcollapse popup

#include <Array.au3>
Global $aArray[] = ["one", "two", "three", "two", "four", "one", "three", 1, 1, "two", "three", 1, 2, 3]
Global $aUnique = __ArrayUnique($aArray, 0, 0, 0, 0, $ARRAYUNIQUE_AUTO, True)

_ArrayDisplay($aArray)
_ArrayDisplay($aUnique)

; #FUNCTION# ====================================================================================================================
; Author ........: SmOke_N
; Modified.......: litlmike, Erik Pilsits, BrewManNH, Melba23
; ===============================================================================================================================
Func __ArrayUnique(Const ByRef $aArray, $iColumn = 0, $iBase = 0, $iCase = 0, $iCount = $ARRAYUNIQUE_COUNT, $iIntType = $ARRAYUNIQUE_AUTO, $bItemCount = False)

    If $iColumn = Default Then $iColumn = 0
    If $iBase = Default Then $iBase = 0
    If $iCase = Default Then $iCase = 0
    If $iCount = Default Then $iCount = $ARRAYUNIQUE_COUNT
    ; Check array
    If UBound($aArray, $UBOUND_ROWS) = 0 Then Return SetError(1, 0, 0)
    Local $iDims = UBound($aArray, $UBOUND_DIMENSIONS), $iNumColumns = UBound($aArray, $UBOUND_COLUMNS)
    If $iDims > 2 Then Return SetError(2, 0, 0)
    ; Check parameters
    If $iBase < 0 Or $iBase > 1 Or (Not IsInt($iBase)) Then Return SetError(3, 0, 0)
    If $iCase < 0 Or $iCase > 1 Or (Not IsInt($iCase)) Then Return SetError(3, 0, 0)
    If $iCount < 0 Or $iCount > 1 Or (Not IsInt($iCount)) Then Return SetError(4, 0, 0)
    If $iIntType < 0 Or $iIntType > 4 Or (Not IsInt($iIntType)) Then Return SetError(5, 0, 0)
    If $iColumn < 0 Or ($iNumColumns = 0 And $iColumn > 0) Or ($iNumColumns > 0 And $iColumn >= $iNumColumns) Then Return SetError(6, 0, 0)

    ; Autocheck of first element
    If $iIntType = $ARRAYUNIQUE_AUTO Then
        Local $bInt, $sVarType
        If $iDims = 1 Then
            $bInt = IsInt($aArray[$iBase])
            $sVarType = VarGetType($aArray[$iBase])
        Else
            $bInt = IsInt($aArray[$iBase][$iColumn])
            $sVarType = VarGetType($aArray[$iBase][$iColumn])
        EndIf
        If $bInt And $sVarType = "Int64" Then
            $iIntType = $ARRAYUNIQUE_FORCE64
        Else
            $iIntType = $ARRAYUNIQUE_FORCE32
        EndIf
    EndIf
    ; Create error handler
    ObjEvent("AutoIt.Error", __ArrayUnique_AutoErrFunc)
    ; Create dictionary
    Local $oDictionary = ObjCreate("Scripting.Dictionary")
    ; Set case sensitivity
    $oDictionary.CompareMode = Number(Not $iCase)
    ; Add elements to dictionary
    Local $vElem, $sType, $vKey, $bCOMError = False
    For $i = $iBase To UBound($aArray) - 1
        If $iDims = 1 Then
            ; 1D array
            $vElem = $aArray[$i]
        Else
            ; 2D array
            $vElem = $aArray[$i][$iColumn]
        EndIf
        ; Determine method to use
        Switch $iIntType
            Case $ARRAYUNIQUE_FORCE32
                ; Use element as key
                $oDictionary.Item($vElem) ; Check if key exists - automatically created if not
                If @error Then
                    $bCOMError = True ; Failed with an Int64, Ptr or Binary datatype
                    ExitLoop
                EndIf
            Case $ARRAYUNIQUE_FORCE64
                $sType = VarGetType($vElem)
                If $sType = "Int32" Then
                    $bCOMError = True ; Failed with an Int32 datatype
                    ExitLoop
                EndIf ; Create key
                $vKey = "#" & $sType & "#" & String($vElem)
                If Not $oDictionary.Item($vKey) Then ; Check if key exists
                    $oDictionary($vKey) = $vElem ; Store actual value in dictionary
                EndIf
            Case $ARRAYUNIQUE_MATCH
                $sType = VarGetType($vElem)
                If StringLeft($sType, 3) = "Int" Then
                    $vKey = "#Int#" & String($vElem)
                Else
                    $vKey = "#" & $sType & "#" & String($vElem)
                EndIf
                If Not $oDictionary.Item($vKey) Then ; Check if key exists
                    $oDictionary($vKey) = $vElem ; Store actual value in dictionary
                EndIf
            Case $ARRAYUNIQUE_DISTINCT
                $vKey = "#" & VarGetType($vElem) & "#" & String($vElem)
                If Not $oDictionary.Item($vKey) Then ; Check if key exists
                    $oDictionary($vKey) = $vElem ; Store actual value in dictionary
                EndIf
        EndSwitch
        $oDictionary.Item($vElem) += 1
    Next
    ; Create return array
    Local $aValues, $aItemCount, $j = 0
    If $bCOMError Then ; Mismatch Int32/64
        Return SetError(7, 0, 0)
    ElseIf $iIntType <> $ARRAYUNIQUE_FORCE32 Then
        ; Extract values associated with the unique keys
        Local $aValues[$oDictionary.Count]
        For $vKey In $oDictionary.Keys()
            $aValues[$j] = $oDictionary($vKey)
            ; Check for Ptr datatype
            If StringLeft($vKey, 5) = "#Ptr#" Then
                $aValues[$j] = Ptr($aValues[$j])
            EndIf
            $j += 1
        Next
    Else
        ; Only need to list the unique keys
        $aValues = $oDictionary.Keys()
    EndIf
    ;~ Add Item count if required
    If $bItemCount Then
        _ArrayTranspose($aValues)
        $aItemCount = $oDictionary.Items
        _ArrayTranspose($aItemCount)
        _ArrayConcatenate($aItemCount, $aValues)
        $aValues = $aItemCount
    EndIf
    ; Add count if required
    If $iCount Then
        _ArrayInsert($aValues, 0, $oDictionary.Count)
    EndIf
    ; Return array
    Return $aValues

EndFunc   ;==>_ArrayUnique

 

[Nine]

1 hour ago, qwert said:

Thanks for the link. I will investigate. object.Count looks promising. And, yes, my array is 1D.

To help you get started :

Local $oDict = ObjCreate("Scripting.Dictionary")

  Local $key
  For $i = 0 to Ubound($YourArray)-1
    $key = $YourArray[$i]
    If $oDict.Exists($key) Then
      $oDict.Item($key) = $oDict.Item($key) + 1
    Else
      $oDict.Add($key, 1)
    EndIf
  Next

  Local $Count = $oDict.Count ; this tells you how many keys you got in $oDict
  Local $Array[$Count][2]
  
  ; one way to extract data from $oDict
  
  Local $p = 0
  For $vKeys in $oDict
    $Array[$p][0] = $vKeys
    $Array[$p][1] = $oDict.item ($vKeys)
    $p += 1
  Next

As you can see it is very easy to use...

Edited April 26, 2019 by Nine

[TheXman]

Or you could let SQLite do the heavy lifting for you.

Since I read that your array is 1D, the example below is written based on a 1D array as input.

 

expandcollapse popup

#include <SQLite.au3>

example()

Func example()
        Const $SQLITE_DLL = "c:\program files\sqlite\sqlite3.dll" ;<-- Change to the location of your sqlite dll

        Local $aRow, _
              $aTestArray[10] = [1,2,1,3,6,3,5,4,4,1]
        Local $sSql
        Local $hQuery

        ;Init sqlite and create a memory db
        _SQLite_Startup($SQLITE_DLL, False, 1)
        If @error Then Exit MsgBox($MB_ICONERROR, "SQLite Error", "Unable to start SQLite. Check existence of DLL")
        _SQLite_Open()

        ;Create a temp table to hold the 1D array data
        _SQLite_Exec(-1, "create temp table array_data (value);")

        ;Load 1D array into the temp table
        _SQLite_Exec(-1, "begin transaction;")
        For $i = 0 To UBound($aTestArray) - 1
            _SQLite_Exec(-1, StringFormat("INSERT INTO array_data values(%s)", _SQLite_Escape($aTestArray[$i])))
        Next
        _SQLite_Exec(-1, "commit;")

        ;Display the count of unique values in the 1D array
        $sSql =  "SELECT value, count(*) FROM array_data GROUP BY value "
        If _SQLite_Query(-1, $sSql, $hQuery) = $SQLITE_OK Then
            ConsoleWrite(StringFormat('Item\tCount') & @CRLF)
            While _SQLite_FetchData($hQuery, $aRow) = $SQLITE_OK
                ConsoleWrite(StringFormat('%s\t%s', $aRow[0], $aRow[1]) & @CRLF)
            WEnd
        EndIf

        ;Close db and shutdown sqlite
        _SQLite_Close()
        _SQLite_Shutdown()
EndFunc

 

[iamtheky]

#include <Array.au3>

Global $aArray[] = ["one", "two", "three", "two", "four", "one", "three", 1, 1, "two", "three", 1, 2, 3]
$aUnique = _ArrayUnique($aArray , 0 , 0 , 0 , 0)

Global $aCount[ubound($aUnique)]
_ArrayColInsert($aUnique , 1)
$str =  _ArrayToString($aArray , ",")

For $i = 0 to ubound($aUnique) - 1
    stringreplace($str , string($aUnique[$i][0]) , string($aUnique[$i][0]))
    $aUnique[$i][1] = @extended
Next

_ArrayDisplay($aUnique)

tried again and ended up essentially the same place with arrayadd instead of arraycolinsert, but fun array playing nonetheless

#include <Array.au3>

Global $aArray[] = ["one", "two", "three", "two", "four", "one", "three", 1, 1, "two", "three", 1, 2, 3]
$aUnique = _ArrayUnique($aArray , 0 , 0 , 0 , 0)
$str =  _ArrayToString($aArray , ",")
local $aOut[0][2]

For $i = 0 to ubound($aUnique) - 1
    stringreplace($str , string($aUnique[$i]) , string($aUnique[$i]))
    $aUnique[$i] = $aUnique[$i] & "|" & @extended
Next

_ArrayAdd($aOut , _ArrayToString($aUnique , "," , -1 , -1 , ";") , 0 , "|" , ",")
_ArrayDisplay($aOut)

 

Edited April 27, 2019 by iamtheky

[mikell]

6 hours ago, iamtheky said:

fun array playing nonetheless

Well exceptionally let me suggest a SRER instead of Stringreplace to fit this kind of perverse cases

Global $aArray[] = ["one", "two", "three", "two", "four", "done", "prednisolone", "three", 1, 1, "two", "three", 1, 2, 3]

[jchd]

10 hours ago, mikell said:

Global $aArray[] = ...

Isn't this a Map variable?

EDIT: I meant "conflict with"

Edited April 27, 2019 by jchd

[mikell]

I thought so. But the help file says

Assigning element values when declaring makes the variable an Array - these three lines are functionally equivalent:

Local $vVar[3] = [1, 2, 3] ; An Array
Local $vVar[] = [1, 2, 3] ; An Array
Local $vVar = [1, 2, 3] ; An Array

 

[iamtheky]

i always forget the 3rd option, force of habit is the culprit behind most of my bloat.

[jchd]

@mikell Granted but that precludes using the beta for those who used it in older code.

Both work in release and beta, but the second blocks under the release.

Local $a[] = [1]
ConsoleWrite(VarGetType($a) & @CRLF)

Local $m[]
ConsoleWrite(VarGetType($m) & @CRLF)

I'd recommend staying away from using [] in array declaration. Also explicitely specifying bounds in dimensions should be restricted in cases where you know in advance that the init data is unkown at declare time or incomplete and you'll need larger bounds, without having to ReDim later on.

[qwert]

Thanks for the suggestions and information. For my current need, I've fashioned a method from what imtheky posed. I wasn't aware of this Au3 feature:

Quote

Returns the new string with the number of replacements performed stored in the @extended macro.

Regarding an addition to the Array UDF, I'm still looking at that possibility. My attempt got tripped up by the _ArrayTranspose calls. It's new to me and I wasn't able to readily understand what I was seeing in the returned array. But in the longer term, I think a UDF call is worth having.

I appreciate the help.

 

[iamtheky]

 

fixed array example, the findall penalty may sting if it has to loop too many times.

 

#include <Array.au3>

Global $aArray[] = ["one", "two", "three", "two", "four", "one", "three", 1, 1, "two", "three", 1, 2, 3]

For $i = ubound($aArray) - 1 to 0 step -1

        $aMatch = _ArrayFindAll($aArray , $aArray[$i])
        $aArray[$i] = $aArray[$i] & " | " & ubound($aMatch)
            _ArrayPop($aMatch)
            _ArrayInsert($aMatch , 0 , ubound($aMatch))
            _ArrayDelete($aArray , $aMatch)

    If ubound($aMatch) > 1 Then $i -= $aMatch[0]

Next

_ArrayDisplay($aArray , $i)

Edited April 27, 2019 by iamtheky

[mikell]

@iamtheky
This array way is a little cumbersome IMHO 
Personally I keep on thinking that as Nine said the SD way is better/lighter/faster

#include <Array.au3>

Global $aArray[] = ["one", "two", "three", "two", "four", "one", "three", 1, 1, "two", "three", 1, 2, 3]

Local $oDict = ObjCreate("Scripting.Dictionary")
For $key In $aArray
  $oDict.Item($key) = $oDict.Exists($key) ? $oDict.Item($key)+1 : 1
Next

Local $count = $oDict.Count, $res[$count][2], $keys = $oDict.Keys
For $i = 0 to $count-1
   $res[$i][0] = $keys[$i]
   $res[$i][1] = $oDict.Item($keys[$i])
Next

_ArrayDisplay($res)

 

[Nine]

@mikell I remember having tested the two methods of transfert from SD to array.  And the use of index is far slower than using the keys.

This is slower :

For $i = 0 to $count-1
   $res[$i][0] = $keys[$i]
   $res[$i][1] = $oDict.Item($keys[$i])
Next

This is faster :

Local $p = 0
  For $vKeys in $oDict
    $Array[$p][0] = $vKeys
    $Array[$p][1] = $oDict.item ($vKeys)
    $p += 1
  Next

I had tested it to count pixels of a full screen.  I am not sure exactly why but I believe the reason is because by using the index SD needs to scan part of the dictionary each time...

[iamtheky]

5 hours ago, mikell said:

This array way is a little cumbersome

Only a little?  You should have seen the Rube Goldberg machine I refined to that.

Edited April 28, 2019 by iamtheky

[iamtheky]

TIL after 10 years of this language:

For debug or even shitty user presentation; if you put a function in the title, you can use the extended return in the body.

#include <Array.au3>

Global $aArray = ["one", "two", "three", "two", "four", "one", "three", 1, 1, "two", "three", 1, 2, 3]

$str = _ArrayToString($aArray , ",")

For $i = 0 to ubound($aArray) - 1
    msgbox(0,  stringreplace($str , $aArray[$i] & ",", $aArray[$i] & ",") , $aArray[$i] & "|" & @extended)
Next

 

 

[jchd]

That can be any expression. It gets converted to a string if necessary.