Jump to content
qwert

Idea(s) for _ArrayUnique that returns counts - (Moved)

Recommended Posts

I have need for an inventory of the multiple occurrences of entries in an array.

_ArrayUnique works well to produce an array of entries. But in looking at the code in the UDF, I see that it uses a Scripting.Dictionary object that I not at all familiar with.

Does anyone have a suggestion of how to adapt the _ArrayUnique code? Or should I simply post-process in my own script to count occurrences? I ask because this seems like some standard solution could be useful in a variety of situations.

Thanks in advance for any help.

Share this post


Link to post
Share on other sites

Moved to the appropriate forum, as the Developer General Discussion forum very clearly states:

Quote

General development and scripting discussions. If it's super geeky and you don't know where to put it - it's probably here.


Do not create AutoIt-related topics here, use the AutoIt General Help and Support or AutoIt Technical Discussion forums.

Moderation Team

Share this post


Link to post
Share on other sites

@qwert Scripting Dictionary is a very powerful class.  Look at this document here,  There is only few methods and properties, so it is very straightforward to use.  It would be useful for you especially if your array is 1D.  I tested the class over a standard array, and for the use you intend to have, I believe it would be a lot faster.

 

Share this post


Link to post
Share on other sites

Thanks for the link. I will investigate. object.Count looks promising. And, yes, my array is 1D.

Share this post


Link to post
Share on other sites

You could use _ArrayUnique and then loop through the results and use _ArrayFindAll to get the count (assumes you haven't destroyed the original array).  Alternatively create a custom _ArrayUnique function:

nb: Tried to explain the process but, it was just easier for me to create an example, just compare to the original function to see the changes.

#include <Array.au3>
Global $aArray[] = ["one", "two", "three", "two", "four", "one", "three", 1, 1, "two", "three", 1, 2, 3]
Global $aUnique = __ArrayUnique($aArray, 0, 0, 0, 0, $ARRAYUNIQUE_AUTO, True)

_ArrayDisplay($aArray)
_ArrayDisplay($aUnique)

; #FUNCTION# ====================================================================================================================
; Author ........: SmOke_N
; Modified.......: litlmike, Erik Pilsits, BrewManNH, Melba23
; ===============================================================================================================================
Func __ArrayUnique(Const ByRef $aArray, $iColumn = 0, $iBase = 0, $iCase = 0, $iCount = $ARRAYUNIQUE_COUNT, $iIntType = $ARRAYUNIQUE_AUTO, $bItemCount = False)

    If $iColumn = Default Then $iColumn = 0
    If $iBase = Default Then $iBase = 0
    If $iCase = Default Then $iCase = 0
    If $iCount = Default Then $iCount = $ARRAYUNIQUE_COUNT
    ; Check array
    If UBound($aArray, $UBOUND_ROWS) = 0 Then Return SetError(1, 0, 0)
    Local $iDims = UBound($aArray, $UBOUND_DIMENSIONS), $iNumColumns = UBound($aArray, $UBOUND_COLUMNS)
    If $iDims > 2 Then Return SetError(2, 0, 0)
    ; Check parameters
    If $iBase < 0 Or $iBase > 1 Or (Not IsInt($iBase)) Then Return SetError(3, 0, 0)
    If $iCase < 0 Or $iCase > 1 Or (Not IsInt($iCase)) Then Return SetError(3, 0, 0)
    If $iCount < 0 Or $iCount > 1 Or (Not IsInt($iCount)) Then Return SetError(4, 0, 0)
    If $iIntType < 0 Or $iIntType > 4 Or (Not IsInt($iIntType)) Then Return SetError(5, 0, 0)
    If $iColumn < 0 Or ($iNumColumns = 0 And $iColumn > 0) Or ($iNumColumns > 0 And $iColumn >= $iNumColumns) Then Return SetError(6, 0, 0)

    ; Autocheck of first element
    If $iIntType = $ARRAYUNIQUE_AUTO Then
        Local $bInt, $sVarType
        If $iDims = 1 Then
            $bInt = IsInt($aArray[$iBase])
            $sVarType = VarGetType($aArray[$iBase])
        Else
            $bInt = IsInt($aArray[$iBase][$iColumn])
            $sVarType = VarGetType($aArray[$iBase][$iColumn])
        EndIf
        If $bInt And $sVarType = "Int64" Then
            $iIntType = $ARRAYUNIQUE_FORCE64
        Else
            $iIntType = $ARRAYUNIQUE_FORCE32
        EndIf
    EndIf
    ; Create error handler
    ObjEvent("AutoIt.Error", __ArrayUnique_AutoErrFunc)
    ; Create dictionary
    Local $oDictionary = ObjCreate("Scripting.Dictionary")
    ; Set case sensitivity
    $oDictionary.CompareMode = Number(Not $iCase)
    ; Add elements to dictionary
    Local $vElem, $sType, $vKey, $bCOMError = False
    For $i = $iBase To UBound($aArray) - 1
        If $iDims = 1 Then
            ; 1D array
            $vElem = $aArray[$i]
        Else
            ; 2D array
            $vElem = $aArray[$i][$iColumn]
        EndIf
        ; Determine method to use
        Switch $iIntType
            Case $ARRAYUNIQUE_FORCE32
                ; Use element as key
                $oDictionary.Item($vElem) ; Check if key exists - automatically created if not
                If @error Then
                    $bCOMError = True ; Failed with an Int64, Ptr or Binary datatype
                    ExitLoop
                EndIf
            Case $ARRAYUNIQUE_FORCE64
                $sType = VarGetType($vElem)
                If $sType = "Int32" Then
                    $bCOMError = True ; Failed with an Int32 datatype
                    ExitLoop
                EndIf ; Create key
                $vKey = "#" & $sType & "#" & String($vElem)
                If Not $oDictionary.Item($vKey) Then ; Check if key exists
                    $oDictionary($vKey) = $vElem ; Store actual value in dictionary
                EndIf
            Case $ARRAYUNIQUE_MATCH
                $sType = VarGetType($vElem)
                If StringLeft($sType, 3) = "Int" Then
                    $vKey = "#Int#" & String($vElem)
                Else
                    $vKey = "#" & $sType & "#" & String($vElem)
                EndIf
                If Not $oDictionary.Item($vKey) Then ; Check if key exists
                    $oDictionary($vKey) = $vElem ; Store actual value in dictionary
                EndIf
            Case $ARRAYUNIQUE_DISTINCT
                $vKey = "#" & VarGetType($vElem) & "#" & String($vElem)
                If Not $oDictionary.Item($vKey) Then ; Check if key exists
                    $oDictionary($vKey) = $vElem ; Store actual value in dictionary
                EndIf
        EndSwitch
        $oDictionary.Item($vElem) += 1
    Next
    ; Create return array
    Local $aValues, $aItemCount, $j = 0
    If $bCOMError Then ; Mismatch Int32/64
        Return SetError(7, 0, 0)
    ElseIf $iIntType <> $ARRAYUNIQUE_FORCE32 Then
        ; Extract values associated with the unique keys
        Local $aValues[$oDictionary.Count]
        For $vKey In $oDictionary.Keys()
            $aValues[$j] = $oDictionary($vKey)
            ; Check for Ptr datatype
            If StringLeft($vKey, 5) = "#Ptr#" Then
                $aValues[$j] = Ptr($aValues[$j])
            EndIf
            $j += 1
        Next
    Else
        ; Only need to list the unique keys
        $aValues = $oDictionary.Keys()
    EndIf
    ;~ Add Item count if required
    If $bItemCount Then
        _ArrayTranspose($aValues)
        $aItemCount = $oDictionary.Items
        _ArrayTranspose($aItemCount)
        _ArrayConcatenate($aItemCount, $aValues)
        $aValues = $aItemCount
    EndIf
    ; Add count if required
    If $iCount Then
        _ArrayInsert($aValues, 0, $oDictionary.Count)
    EndIf
    ; Return array
    Return $aValues

EndFunc   ;==>_ArrayUnique

 

Share this post


Link to post
Share on other sites
Posted (edited)
1 hour ago, qwert said:

Thanks for the link. I will investigate. object.Count looks promising. And, yes, my array is 1D.

To help you get started :

Local $oDict = ObjCreate("Scripting.Dictionary")

  Local $key
  For $i = 0 to Ubound($YourArray)-1
    $key = $YourArray[$i]
    If $oDict.Exists($key) Then
      $oDict.Item($key) = $oDict.Item($key) + 1
    Else
      $oDict.Add($key, 1)
    EndIf
  Next

  Local $Count = $oDict.Count ; this tells you how many keys you got in $oDict
  Local $Array[$Count][2]
  
  ; one way to extract data from $oDict
  
  Local $p = 0
  For $vKeys in $oDict
    $Array[$p][0] = $vKeys
    $Array[$p][1] = $oDict.item ($vKeys)
    $p += 1
  Next

As you can see it is very easy to use...

Edited by Nine

Share this post


Link to post
Share on other sites

Or you could let SQLite do the heavy lifting for you.

Since I read that your array is 1D, the example below is written based on a 1D array as input.

 

#include <SQLite.au3>

example()

Func example()
        Const $SQLITE_DLL = "c:\program files\sqlite\sqlite3.dll" ;<-- Change to the location of your sqlite dll

        Local $aRow, _
              $aTestArray[10] = [1,2,1,3,6,3,5,4,4,1]
        Local $sSql
        Local $hQuery

        ;Init sqlite and create a memory db
        _SQLite_Startup($SQLITE_DLL, False, 1)
        If @error Then Exit MsgBox($MB_ICONERROR, "SQLite Error", "Unable to start SQLite. Check existence of DLL")
        _SQLite_Open()

        ;Create a temp table to hold the 1D array data
        _SQLite_Exec(-1, "create temp table array_data (value);")

        ;Load 1D array into the temp table
        _SQLite_Exec(-1, "begin transaction;")
        For $i = 0 To UBound($aTestArray) - 1
            _SQLite_Exec(-1, StringFormat("INSERT INTO array_data values(%s)", _SQLite_Escape($aTestArray[$i])))
        Next
        _SQLite_Exec(-1, "commit;")

        ;Display the count of unique values in the 1D array
        $sSql =  "SELECT value, count(*) FROM array_data GROUP BY value "
        If _SQLite_Query(-1, $sSql, $hQuery) = $SQLITE_OK Then
            ConsoleWrite(StringFormat('Item\tCount') & @CRLF)
            While _SQLite_FetchData($hQuery, $aRow) = $SQLITE_OK
                ConsoleWrite(StringFormat('%s\t%s', $aRow[0], $aRow[1]) & @CRLF)
            WEnd
        EndIf

        ;Close db and shutdown sqlite
        _SQLite_Close()
        _SQLite_Shutdown()
EndFunc

 

Share this post


Link to post
Share on other sites
Posted (edited)
#include <Array.au3>

Global $aArray[] = ["one", "two", "three", "two", "four", "one", "three", 1, 1, "two", "three", 1, 2, 3]
$aUnique = _ArrayUnique($aArray , 0 , 0 , 0 , 0)

Global $aCount[ubound($aUnique)]
_ArrayColInsert($aUnique , 1)
$str =  _ArrayToString($aArray , ",")

For $i = 0 to ubound($aUnique) - 1
    stringreplace($str , string($aUnique[$i][0]) , string($aUnique[$i][0]))
    $aUnique[$i][1] = @extended
Next

_ArrayDisplay($aUnique)

tried again and ended up essentially the same place with arrayadd instead of arraycolinsert, but fun array playing nonetheless

#include <Array.au3>

Global $aArray[] = ["one", "two", "three", "two", "four", "one", "three", 1, 1, "two", "three", 1, 2, 3]
$aUnique = _ArrayUnique($aArray , 0 , 0 , 0 , 0)
$str =  _ArrayToString($aArray , ",")
local $aOut[0][2]

For $i = 0 to ubound($aUnique) - 1
    stringreplace($str , string($aUnique[$i]) , string($aUnique[$i]))
    $aUnique[$i] = $aUnique[$i] & "|" & @extended
Next

_ArrayAdd($aOut , _ArrayToString($aUnique , "," , -1 , -1 , ";") , 0 , "|" , ",")
_ArrayDisplay($aOut)

 

Edited by iamtheky

,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites
6 hours ago, iamtheky said:

fun array playing nonetheless

Well exceptionally let me suggest a SRER instead of Stringreplace to fit this kind of perverse cases

Global $aArray[] = ["one", "two", "three", "two", "four", "done", "prednisolone", "three", 1, 1, "two", "three", 1, 2, 3]

:idiot:

Share this post


Link to post
Share on other sites
Posted (edited)
10 hours ago, mikell said:

Global $aArray[] = ...

Isn't this a Map variable?

EDIT: I meant "conflict with"

Edited by jchd

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

I thought so. But the help file says

Assigning element values when declaring makes the variable an Array - these three lines are functionally equivalent:

Local $vVar[3] = [1, 2, 3] ; An Array
Local $vVar[] = [1, 2, 3] ; An Array
Local $vVar = [1, 2, 3] ; An Array

 

Share this post


Link to post
Share on other sites

i always forget the 3rd option, force of habit is the culprit behind most of my bloat.


,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites

@mikell Granted but that precludes using the beta for those who used it in older code.

Both work in release and beta, but the second blocks under the release.

Local $a[] = [1]
ConsoleWrite(VarGetType($a) & @CRLF)

Local $m[]
ConsoleWrite(VarGetType($m) & @CRLF)

I'd recommend staying away from using [] in array declaration. Also explicitely specifying bounds in dimensions should be restricted in cases where you know in advance that the init data is unkown at declare time or incomplete and you'll need larger bounds, without having to ReDim later on.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Thanks for the suggestions and information. For my current need, I've fashioned a method from what imtheky posed. I wasn't aware of this Au3 feature:

Quote

Returns the new string with the number of replacements performed stored in the @extended macro.

Regarding an addition to the Array UDF, I'm still looking at that possibility. My attempt got tripped up by the _ArrayTranspose calls. It's new to me and I wasn't able to readily understand what I was seeing in the returned array. But in the longer term, I think a UDF call is worth having.

I appreciate the help.

 

Share this post


Link to post
Share on other sites
Posted (edited)

 

fixed array example, the findall penalty may sting if it has to loop too many times.

 

#include <Array.au3>

Global $aArray[] = ["one", "two", "three", "two", "four", "one", "three", 1, 1, "two", "three", 1, 2, 3]

For $i = ubound($aArray) - 1 to 0 step -1

        $aMatch = _ArrayFindAll($aArray , $aArray[$i])
        $aArray[$i] = $aArray[$i] & " | " & ubound($aMatch)
            _ArrayPop($aMatch)
            _ArrayInsert($aMatch , 0 , ubound($aMatch))
            _ArrayDelete($aArray , $aMatch)

    If ubound($aMatch) > 1 Then $i -= $aMatch[0]

Next

_ArrayDisplay($aArray , $i)
Edited by iamtheky

,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites

@iamtheky
This array way is a little cumbersome IMHO  :sweating:
Personally I keep on thinking that as Nine said the SD way is better/lighter/faster

#include <Array.au3>

Global $aArray[] = ["one", "two", "three", "two", "four", "one", "three", 1, 1, "two", "three", 1, 2, 3]

Local $oDict = ObjCreate("Scripting.Dictionary")
For $key In $aArray
  $oDict.Item($key) = $oDict.Exists($key) ? $oDict.Item($key)+1 : 1
Next

Local $count = $oDict.Count, $res[$count][2], $keys = $oDict.Keys
For $i = 0 to $count-1
   $res[$i][0] = $keys[$i]
   $res[$i][1] = $oDict.Item($keys[$i])
Next

_ArrayDisplay($res)

 

Share this post


Link to post
Share on other sites

@mikell I remember having tested the two methods of transfert from SD to array.  And the use of index is far slower than using the keys.

This is slower :

For $i = 0 to $count-1
   $res[$i][0] = $keys[$i]
   $res[$i][1] = $oDict.Item($keys[$i])
Next

This is faster :

Local $p = 0
  For $vKeys in $oDict
    $Array[$p][0] = $vKeys
    $Array[$p][1] = $oDict.item ($vKeys)
    $p += 1
  Next

I had tested it to count pixels of a full screen.  I am not sure exactly why but I believe the reason is because by using the index SD needs to scan part of the dictionary each time...

Share this post


Link to post
Share on other sites
Posted (edited)
5 hours ago, mikell said:

This array way is a little cumbersome

Only a little?  You should have seen the Rube Goldberg machine I refined to that.

Edited by iamtheky

,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites

TIL after 10 years of this language:

For debug or even shitty user presentation; if you put a function in the title, you can use the extended return in the body.

#include <Array.au3>

Global $aArray = ["one", "two", "three", "two", "four", "one", "three", 1, 1, "two", "three", 1, 2, 3]

$str = _ArrayToString($aArray , ",")

For $i = 0 to ubound($aArray) - 1
    msgbox(0,  stringreplace($str , $aArray[$i] & ",", $aArray[$i] & ",") , $aArray[$i] & "|" & @extended)
Next

 

MoreYouKnow.png.1b7dbf206d31fb1deeb6ea213bf0674f.png

 


,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites

That can be any expression. It gets converted to a string if necessary.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...