Jump to content
dejhost

Speeding up array processing

Recommended Posts

Is this what you are looking for?

#include <Array.au3>

$bit = 7

$fTimer = TimerInit()
Global $BinArray[2 ^ $bit][$bit + 1]
Global $aResult[$bit + 1]
For $i = 0 to 2 ^ $bit - 1
    $binary = Integer2Binary($i, $BinArray, $i, $bit)
Next
_ArraySort($BinArray, 0, 0, 0, $bit)
$pos = 0
For $i = 0 to UBound($BinArray) - 2
    If $BinArray[$i][$bit] <> $BinArray[$i + 1][$bit] Then
        $sBits = ""
        For $x = 0 To $bit - 1
            $sBits &= $BinArray[$i][$x] = "" ? 0 : $BinArray[$i][$x]
        Next
        $aResult[$pos] = $sBits
        $pos += 1
    EndIf
Next
$sBits = ""
For $x = 0 To $bit - 1
    $sBits &= $BinArray[$i][$x] = "" ? 0 : $BinArray[$i][$x]
Next
$aResult[$pos] = $sBits
ConsoleWrite("Runtime: " & TimerDiff($fTimer) & @CRLF)
_ArrayDisplay($aResult)

Func Integer2Binary($in, ByRef $BinArray, $pos, $bit)
    Local $bin, $b = $Bit - 1
    While $in > 0
        $bBit = Mod($in, 2)
        If $bBit Then $BinArray[$pos][$bit] += 1
        $BinArray[$pos][$b] = $bBit
        $in = Floor($in / 2)
        $b -= 1
    WEnd
EndFunc

 


Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites

lets see...

Yes, bruteforced, in some seconds...completely in assembly it will take only some milliseconds.

The function ROL has 3 parameters, the number to be ROLled, the count how often is ROLled and the number of bits to ROLl

It should be possible to everyone to replace the numbers in the array by their binary equivalent

#include <Array.au3>
;#include <assembleit2_64.au3>
#AutoIt3Wrapper_UseX64=n



#cs ROL_it
    use32
    mov eax, [esp+4]        ;number
    mov edi, [esp+8]        ;count
    mov ecx, [esp+12]       ;bits to rol

    dec ecx                 ;bitnumber to carry
    xor ebx,ebx             ;=0

    jecxz end_rol           ;exit, if cnt=zero
    more_rol:               ;loop
    btr eax,ecx             ;bit into carryflag, clear bit
    setc bl                 ;Carry into bl
    ;~     _asmdbg_()            ;debugger
    shl eax,1               ;ROL it
    add al,bl               ;add eax and carrybit
    sub edi,1
    jnz more_rol            ;until count is reached
    end_rol:

    ret                     ;eax is returned
#ce



$bits = 14
$ar = 2 ^ ($bits) - 1

$binarycode = "0x8B4424048B7C24088B4C240C4931DBE30F0FB3C80F92C3D1E000D883EF0175F1C3"
$tCodeBuffer = dllstructcreate("byte[" & StringLen($binarycode) / 2 - 1 & "]") ;reserve Memory for opcodes
DllStructSetData($tCodeBuffer, 1, $binarycode)


Dim $a[$ar + 1]

For $i = 0 To $ar           ;all numbers
    $a[$i] = $i             ;into array
Next


For $i = 1 To $ar Step 2    ;all numbers
    If $a[$i] <> 0 Then     ;only if not deleted

        For $x = 1 To $bits - 1 ;ROL loop through all bits
            $t = ROL($i, $x, $bits) ;ROLled numbers...
            $a[$t] = 0      ;...are always deleted
        Next

    EndIf
Next

$r = _ArrayUnique($a)
_ArrayDisplay($r)


Func ROL($number, $anz, $bits)
;~  $return=_AssembleIt2("uint", "ROL_it", "uint", $number, "uint", $anz, "uint", $bits)
;~  return $return
    $ret = DllCallAddress("uint:cdecl", DllStructGetPtr($tCodeBuffer), "uint", $number, "uint", $anz, "uint", $bits)
    Return $ret[0]
EndFunc                     ;==>ROL

 

Share this post


Link to post
Share on other sites

Hi all,

I had a brainwave while waiting for my take-away curry - and after eating it (very tasty indeed) - I went back to the drawing board:

#include <Array.au3>

$nBegin = TimerInit()

Local $iBit = 14

; Create array to hold elements in columns according to the number of 1s in each element
Local $aBinArray[(2 ^ $iBit) + 1][$iBit + 1]

ConsoleWrite("Filling array" & @CRLF)

For $i = 0 to (2 ^ $iBit) - 1
    $sBinary = (Dec2Bin($i))                                            ; Create binary string
    $sBinary = StringRegExpReplace($sBinary, "(?U)^(.*)0*$", "$1")      ; Strip any trailing zeros
    $sBinary = StringFormat("%0" & $iBit & "s", $sBinary)               ; Pad with leading zeros as required
    StringReplace($sBinary, "1", "")                                    ; Get number of 1s within it
    $iCount = @extended
    $aBinArray[0][$iCount] +=1
    $aBinArray[$aBinArray[0][$iCount]][$iCount]= $sBinary               ; Store in required column
Next
;_ArrayDisplay($aBinArray, "Full", Default, 8)

; Create array to hold unique elements
Local $aUnique[UBound($aBinArray) + 1] = [0]

; Add elements from first and second columns which are unique by definition
For $i = 0 To 1
    $aUnique[0] += 1
    $aUnique[$aUnique[0]] = $aBinArray[1][$i]
Next

; Now loop through other columns checking for duplicates when rotated
For $i = 2 To $iBit - 2 ; Column

    ConsoleWrite("Column: " & $i & @CRLF)

    ; Extract column from main array
    $aCol = _ArrayExtract($aBinArray, 1, $aBinArray[0][$i], $i, $i)

    ConsoleWrite("Rotating"  & @CRLF)

    ; Now work through array rotating each element
    For $j = 0 To UBound($aCol) - 2
        ; Check string
        $sRotated = $aCol[$j]
        ; If this element has not already been deleted
        If $sRotated Then
            ; Loop through all possible rotations
            For $k = 1 To $iBit
                ; Compare to elements in array - only need to look below the current element
                For $m = $j + 1 To UBound($aCol) - 1
                    If $aCol[$m] = $sRotated Then
                        ; If there is a match then delete the element
                        $aCol[$m] = ""
                    EndIf
                Next
                ; Rotate the element to the next position
                $sRotated = _Rotate($sRotated)
            Next
        EndIf
    Next

    ConsoleWrite("Adding" & @CRLF)

    ; Add remaining elements to the unique array
    For $j = 0 To UBound($aCol) - 1
        If $aCol[$j] Then
            $aUnique[0] += 1
            $aUnique[$aUnique[0]] = $aCol[$j]
        EndIf
    Next

Next

; Add elements from the penultimate and final columns which are also are unique by definition
For $i = $iBit - 1 To $iBit
    $aUnique[0] += 1
    $aUnique[$aUnique[0]] = $aBinArray[1][$i]
Next

; Remove blank elements
ReDim $aUnique[$aUnique[0] + 1]

ConsoleWrite(TimerDiff($nBegin) & @CRLF)

_ArrayDisplay($aUnique, "Unique patterns", Default, 8)

Func _Rotate($sString)
    Return StringRight($sString, 1) & StringLeft($sString, $iBit - 1)
EndFunc

Func Dec2Bin($iD)
    Return (BitShift($iD, 1) ? Dec2Bin(BitShift($iD, 1)) : "") & BitAnd($iD, 1)
EndFunc

The trick is to only compare elements which have the same number of 0s and 1s - as only they can be duplicates of each other. This runs with 14-bit values in just over 3 minutes on my old machine and the results look pretty good as far as I can tell.

Over to you for some checking, jchd!

M23


Public_Domain.png.2d871819fcb9957cf44f4514551a2935.png Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind

Open spoiler to see my UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

You could speed up your algorithm if you only look at ODD numbers! Even (not odd, english is not an easy language^^) numbers should never fit the specs because they are always a multiple of 2.

Shift/rotate left of an odd number is a multiple of 2 too....

Btw, my algorithm is wrong, but i am on the way to fix it o:)

Edited by AndyG

Share this post


Link to post
Share on other sites

Granted that can be done in assembly pretty fast but the beauty would be to keep it in native AutoIt for "reasonable bitstring size".

I'm just back at the keyboard. Oh btw, Andy, you miss the "all 1" value (even if, just like 0, it doesn't mean something very practical).


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

AndyG,

Great idea - gets it down to under a minute now:

#include <Array.au3>

$nBegin = TimerInit()

Local $iBit = 14

Local $aBinArray[(2 ^ $iBit) + 1][$iBit + 1]

; Add all 0s element to 0 column
$aBinArray[0][0] = 1
$aBinArray[1][0] = ""
For $i = 1 To $iBit
    $aBinArray[1][0] &= "0"
Next

ConsoleWrite("Filling array" & @CRLF)

; Skip all even numbers - they must be duplicates when rotated
For $i = 1 to (2 ^ $iBit) - 1 Step 2
    $sBinary = (Dec2Bin($i))                                            ; Create binary string
    ;ConsoleWrite($sBinary & @TAB)
    $sBinary = StringRegExpReplace($sBinary, "(?U)^(.*)0*$", "$1")      ; Strip any trailing zeros
    ;ConsoleWrite($sBinary & @TAB)
    $sBinary = StringFormat("%0" & $iBit & "s", $sBinary)               ; Pad with leading zeros as required
    ;ConsoleWrite($sBinary & @CRLF)
    StringReplace($sBinary, "1", "")                                    ; Get number of 1s within it
    $iCount = @extended
    $aBinArray[0][$iCount] +=1
    $aBinArray[$aBinArray[0][$iCount]][$iCount]= $sBinary               ; Store in required column
Next
;_ArrayDisplay($aBinArray, "Full", Default, 8)

; Create array to hold unique elements
Local $aUnique[UBound($aBinArray) + 1] = [0]

; Add elements from first and second columns which are unique by definition
For $i = 0 To 1
    $aUnique[0] += 1
    $aUnique[$aUnique[0]] = $aBinArray[1][$i]
Next

; Now loop through other columns checking for duplicates when rotated
For $i = 2 To $iBit - 2 ; Column

    ConsoleWrite("Column: " & $i & @CRLF)

    ; Extract column from main array
    $aCol = _ArrayExtract($aBinArray, 1, $aBinArray[0][$i], $i, $i)

    ConsoleWrite("Rotating"  & @CRLF)

    ; Now work through array rotating each element
    For $j = 0 To UBound($aCol) - 2
        ; Check string
        $sRotated = $aCol[$j]
        ; If this element has not already been deleted
        If $sRotated Then
            ; Loop through all possible rotations
            For $k = 1 To $iBit
                ; Compare to elements in array - only need to look below the current element
                For $m = $j + 1 To UBound($aCol) - 1
                    If $aCol[$m] = $sRotated Then
                        ; If there is a match then delete the element
                        $aCol[$m] = ""
                    EndIf
                Next
                $sRotated = _Rotate($sRotated)
            Next
        EndIf
    Next

    ConsoleWrite("Adding" & @CRLF)

    ; Add remaining elements to the unique array
    For $j = 0 To UBound($aCol) - 1
        If $aCol[$j] Then
            $aUnique[0] += 1
            $aUnique[$aUnique[0]] = $aCol[$j]
        EndIf
    Next

Next

; Add elements from the penultimate and final columnss which are also are unique by definition
For $i = $iBit - 1 To $iBit
    $aUnique[0] += 1
    $aUnique[$aUnique[0]] = $aBinArray[1][$i]
Next

ReDim $aUnique[$aUnique[0] + 1]

ConsoleWrite(TimerDiff($nBegin) & @CRLF)

_ArrayDisplay($aUnique, "Unique patterns", Default, 8)

Func _Rotate($sString)
    Return StringRight($sString, 1) & StringLeft($sString, $iBit - 1)
EndFunc

Func Dec2Bin($iD)
    Return (BitShift($iD, 1) ? Dec2Bin(BitShift($iD, 1)) : "") & BitAnd($iD, 1)
EndFunc

it produces the same number of "unique" strings as before - 1183. Any other brilliant suggestions out there?

M23


Public_Domain.png.2d871819fcb9957cf44f4514551a2935.png Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind

Open spoiler to see my UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites
7 hours ago, dejhost said:

Sorry for the confusion. The purpose of the script:

I am generating a lookup table for photogrammetric markers. This is a binary code (bits can be set to white or to black), arranged in a radial way. The markers have no orientation; which means, that there is no "first bit", and no "last bit":  "10000" = "01000" = "00100" = "00010" = "00001".

Am I wrong with my outcome?

What I understood from the op is that he wants only unique bit sum counts or?

"10000" = "01000" = "00100" = "00010" = "00001" -> only one from these values is listed!?!?


Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites

UEZ,

Unfortunately I believe your algorithm is too restrictive. It returns only a single example of each mix of digits - for example for "two 1s in an 8-bit string" you provide only: 00010001, whereas 00000011, 00000101, 00001001 are also unique patterns for that mix.

M23


Public_Domain.png.2d871819fcb9957cf44f4514551a2935.png Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind

Open spoiler to see my UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

Here my 2 cents,

It elaborates 14 bits numbers in about 15 seconds in pure AutoIt.

#include <array.au3>
Local $iTimer = TimerInit()
Local $iBits = 14 ; nr. of bit
Local $iTotalCombinations = 2 ^ $iBits ; total number of combinations
Local $aCombinations[$iTotalCombinations][2] ; make space to develop all combinations

For $i = $iTotalCombinations - 1 To 0 Step -1
    $aCombinations[$i][1] = _ToBase($i, 2, $iBits) ; number in binary digits (base number)
    $aCombinations[$i][0] = $i ; index of this "base" number
    $sBits = $aCombinations[$i][1]
    For $iRotation = 1 To $iBits - 1 ; generate all rotations of this number.
        $sBits = StringRight($sBits, $iBits - 1) & StringLeft($sBits, 1) ; rotate the bits
        $aCombinations[_Bin_To_Dec($sBits)][0] = $i ; same index as the base for any rotated variation
    Next
Next

$aIndexes = _ArrayUnique($aCombinations, 0, 0, 0, $ARRAYUNIQUE_NOCOUNT) ; keep only uniques indexes
Local $aResult[UBound($aIndexes)] ; make room for result

For $i = 0 To UBound($aIndexes) - 1
    $aResult[$i] = $aCombinations[$aIndexes[$i]][1] ; take only one of the possible rotations.
Next
ConsoleWrite("Elaborated " & $iBits & " in " & Int(TimerDiff($iTimer) / 1000) & " seconds." & @CRLF)
_ArrayDisplay($aResult, "uniques")

; http://www.autoitscript.com/forum/topic/93742-converting-decimal-numbers-to-another-base/#entry673358
Func _ToBase($iNumber, $iBase, $iPad = 1)
    Local $sRet = "", $iDigit
    Do
        $iDigit = Mod($iNumber, $iBase)
        If $iDigit < 10 Then
            $sRet = String($iDigit) & $sRet
        Else
            $sRet = Chr(55 + $iDigit) & $sRet
        EndIf
        $iNumber = Int($iNumber / $iBase)
    Until ($iNumber = 0) And (StringLen($sRet) >= $iPad)
    Return $sRet
EndFunc   ;==>_ToBase

Func _Bin_To_Dec($BinNum) ; from binary to decimal
    Local $dec = 0
    For $i = 0 To StringLen($BinNum) - 1
        $dec += 2 ^ $i * StringMid($BinNum, StringLen($BinNum) - $i, 1)
    Next
    Return $dec
EndFunc   ;==>_Bin_To_Dec

 


small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Share this post


Link to post
Share on other sites

Chimp,

We get the same number of "unique" patterns - and your script takes the same time as mine on my machine (just under a minute). Perhaps time for me to get a newer and faster one!

M23


Public_Domain.png.2d871819fcb9957cf44f4514551a2935.png Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind

Open spoiler to see my UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

...... a little adjustment in the logic, now with 14 bits it runs in about 2 seconds.

#include <array.au3>
Local $iTimer = TimerInit()
Local $iBits = 14 ; nr. of bit
Local $iTotalCombinations = 2 ^ $iBits ; total number of combinations
Local $aCombinations[$iTotalCombinations][2] ; make space to develop all combinations

For $i = 0 To $iTotalCombinations - 1 ; To 0 Step -1
    $aCombinations[$i][1] = _ToBase($i, 2, $iBits) ; number in binary digits (base number)
    If $aCombinations[$i][0] = "" Then ; is this combination to be calculated ??
        $aCombinations[$i][0] = $i ; index of this "base" number
        $sBits = $aCombinations[$i][1]
        For $iRotation = 1 To $iBits - 1 ; generate all rotations of this number.
            $sBits = StringRight($sBits, $iBits - 1) & StringLeft($sBits, 1) ; rotate the bits
            $aCombinations[_Bin_To_Dec($sBits)][0] = $i ; same index as the base for any rotated variation
        Next
    EndIf
Next

$aIndexes = _ArrayUnique($aCombinations, 0, 0, 0, $ARRAYUNIQUE_NOCOUNT) ; keep only uniques indexes
Local $aResult[UBound($aIndexes)] ; make room for result

For $i = 0 To UBound($aIndexes) - 1
    $aResult[$i] = $aCombinations[$aIndexes[$i]][1] ; take only one of the possible rotations.
Next
ConsoleWrite("Elaborated " & $iBits & " in " & Int(TimerDiff($iTimer) / 1000) & " seconds." & @CRLF)
_ArrayDisplay($aResult, "uniques")

; http://www.autoitscript.com/forum/topic/93742-converting-decimal-numbers-to-another-base/#entry673358
Func _ToBase($iNumber, $iBase, $iPad = 1)
    Local $sRet = "", $iDigit
    Do
        $iDigit = Mod($iNumber, $iBase)
        If $iDigit < 10 Then
            $sRet = String($iDigit) & $sRet
        Else
            $sRet = Chr(55 + $iDigit) & $sRet
        EndIf
        $iNumber = Int($iNumber / $iBase)
    Until ($iNumber = 0) And (StringLen($sRet) >= $iPad)
    Return $sRet
EndFunc   ;==>_ToBase

Func _Bin_To_Dec($BinNum) ; from binary to decimal
    Local $dec = 0
    For $i = 0 To StringLen($BinNum) - 1
        $dec += 2 ^ $i * StringMid($BinNum, StringLen($BinNum) - $i, 1)
    Next
    Return $dec
EndFunc   ;==>_Bin_To_Dec

 


small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Share this post


Link to post
Share on other sites

hmmm, seems we have the same algorithm:D

Chimp, so nice:thumbsup:

#include <Array.au3>
;#include <assembleit2_64.au3>
#AutoIt3Wrapper_UseX64=n



#cs ROL_it
    use32
    mov eax, [esp+4]       ;number
    mov edi, [esp+8]       ;count
    mov ecx, [esp+12]      ;bits to rol

    dec ecx                ;bitnumber to carry
    xor ebx,ebx            ;=0

    jecxz end_rol          ;exit, if cnt=zero
    more_rol:              ;loop
    btr eax,ecx            ;bit into carryflag, clear bit
    setc bl                ;Carry into bl
    ;~     _asmdbg_()            ;debugger
    shl eax,1              ;ROL it
    add al,bl              ;add eax and carrybit
    sub edi,1
    jnz more_rol           ;until count is reached
    end_rol:

    ret                    ;eax is returned
#ce



$bits = 14
$ar = 2 ^ ($bits) - 1
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $ar = ' & Dec2Bin($ar) & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

$binarycode = "0x8B4424048B7C24088B4C240C4931DBE30F0FB3C80F92C3D1E000D883EF0175F1C3"
$tCodeBuffer = DllStructCreate("byte[" & StringLen($binarycode) / 2 - 1 & "]") ;reserve Memory for opcodes
DllStructSetData($tCodeBuffer, 1, $binarycode)


Dim $a[$ar + 1]

For $i = 0 To $ar          ;all numbers
    $a[$i] = $i            ;into array
Next


For $i = 1 To $ar Step 2   ;all numbers in array
    If $a[$i] <> 0 Then    ;only if not deleted
        For $x = 1 To $bits - 1 ;ROL loop through all bits
            $t = ROL($i, $x, $bits) ;ROLled numbers...
            If $t > $i Then $a[$t] = 0 ;...are always deleted
        Next
    EndIf
Next

$r = _ArrayUnique($a)
;~ _ArrayDisplay($r)
Dim $c[UBound($r) + 1][2]

For $i = 1 To UBound($r) - 1
    $c[$i][0] = $r[$i]
    $c[$i][1] = Dec2Bin($r[$i])
Next

$z = _ArrayDisplay($c)

Func Dec2Bin($D)
    Return (BitShift($D, 1) ? Dec2Bin(BitShift($D, 1)) : "") & BitAND($D, 1)
EndFunc                    ;==>Dec2Bin


Func ROL($number, $anz, $bits)
;~     $return=_AssembleIt2("uint", "ROL_it", "uint", $number, "uint", $anz, "uint", $bits)
;~     return $return
    $ret = DllCallAddress("uint:cdecl", DllStructGetPtr($tCodeBuffer), "uint", $number, "uint", $anz, "uint", $bits)
    Return $ret[0]
EndFunc                    ;==>ROL

 

Share this post


Link to post
Share on other sites

Now I understood what phase 2 is about. I totally got it wrong. :>


Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites

and the one in native AutoIt, fast :sweating:

#include <Array.au3>


$bits = 14
$ar = 2 ^ ($bits) - 1


Dim $a[$ar + 1]

For $i = 0 To $ar            ;all numbers
    $a[$i] = $i              ;into array
Next


For $i = 1 To $ar Step 2     ;all numbers
    If $a[$i] <> 0 Then      ;only if not deleted
        For $x = 1 To $bits - 1 ;ROL loop through all bits
            $t = ROL2($i, $x, $bits) ;ROLled numbers...
            If $t > $i Then $a[$t] = 0 ;...are always deleted
        Next
    EndIf
Next

$r = _ArrayUnique($a)
;~ _ArrayDisplay($r)
Dim $c[UBound($r) + 1][2]

For $i = 1 To UBound($r) - 1
    $c[$i][0] = $r[$i]
    $c[$i][1] = Dec2Bin($r[$i])
Next

$z = _ArrayDisplay($c)

Func Dec2Bin($D)
    Return (BitShift($D, 1) ? Dec2Bin(BitShift($D, 1)) : "") & BitAND($D, 1)
EndFunc                      ;==>Dec2Bin


Func ROL2($number, $anz, $bits)
    $var3 = $number
    $n = 2 ^ $bits
    For $i = 1 To $anz
        $var = BitShift($var3, -1) ;shift left 1
        $var1 = (BitAND($var, $n) >= 1) ;msb true or false / 1 or 0
        $var2 = BitAND($var, $n - 1) ;eliminate msb
        $var3 = $var2 + $var1 ;add shifted number + MSB
    Next
    Return $var3
EndFunc                      ;==>ROL2

 

Share this post


Link to post
Share on other sites

This is a very good concept which I have used myself. I don't know what you call it, but it's a kind of modular binary compression. In my implementation, all variants are modulated to produce the smallest odd value. This value then becomes a key: representing all the other modular rotations. The key is then added to a look up table. This is the first time I have seen anyone else using a similar concept. It can be a very effective way to organize certain types of data. Very geeky! :)

Share this post


Link to post
Share on other sites

Looking though the archives - my implementation is for exactly 3 nibbles and the code is suboptimal for 14 bits. I believe this can be done quite efficiently though. Here's part of that old code which might give you some new ideas. You only need one inversion to represent them all. I wish I had more time to spend on questions like this. Brute force is certainly not the way to go.

Local $sString = '10001100110101' ; _ModeInversion() is intended for strings containing 1's and zeros
Local $sInversion = $sString

Do
    $sInversion = _ModeInversion($sInversion)
    ConsoleWrite($sInversion & @LF)
Until $sInversion = $sString ; set to ignore duplicate patterns [also occurring due to symmetry]

; Return the 1st Inversion of a binary string variant [odd numbers only]
Func _ModeInversion($sBin)
    Return StringRegExpReplace($sBin, "(0*1)(.*)", "$2$1")
EndFunc

Edit: The string must end with a 1. Odd numbers only - duh!.

Edited by czardas

Share this post


Link to post
Share on other sites

never thought that this issue actually is sooo, soo, so "not trivial!

  • I need [14, 16, 18, 20] digits. While testing, I reduce to 6.
  • A simple solution would be the following: put the pc to work every night for the next 7 days in order to generate the requested arrays. Save them as a textfile. Include textfiles using FileInstall() - and access them instead of generating them. But this is kind of cheating, isn't it?
  • I will - of course - test your suggestions now. So maybe the following has been considered alreaddy: One could split up the array containing all alterations (so before removing the "duplicates") into 10 different arrays that contain only binary-numbers of equal checksums:. For
    • Array_1 contains all binary numbers with a single "1" - and 9 "0".
    • Array_2 contains all binary numbers with a two"1" - and 8 "0".
    • Array_3 contains all binary numbers with a three"1" - and 7 "0".

         Then, _ArrayUnique is applied within the arrays (rotating bit by bit).

Share this post


Link to post
Share on other sites
3 minutes ago, czardas said:

Inefficient!

Just tested the code from AndyG... no more questions on my behalf! The skript runs so fast, the user doesn't even notice any computation time ;-)

Thanks for all your input!

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • By Stew
      (Edited from original.  Please note that I AM NOT AN AUTOIT EXPERT.  I write code using Autoit frequently but I am no expert, especially when it comes to I/O.  So any remarks that start with "Why did you..." can be answered by referring to the first sentence.  This project was done in Autoit because of an interface I built to display the data.)
      Attached is a program and ascii input file I wrote to read stock price data, convert it to binary and then read it back into the program in binary.  The goal was to show increased performance for reading the files in binary and provide a demo on how to read/write binary for int32, int64, double and strings for anyone who might find it helpful.  The results on my PC show the following:
      Time to read ascii file only: 456.981951167202
      Ascii read & process time: 6061.83075631701
      Binary write file time: 14787.9184635239
      Time just to read binary file: 42.418867292311
      Binary read and process time: 4515.16129830537
      A couple things to note:
      1) The 32 MB ascii file took 10x longer to read than the 15 MB binary file.  Not entirely sure why.  Both were read into a buffer.
      2) The Binary write takes a long time but I made no effort to optimize this because the plan was to write this file one time only so I don't mind if it takes longer to write this file.  I care much more about how long it takes to read the file because I will be reading it many times.
      3) There was a modest gain in converting the ascii file to binary in terms of file size and reading speed.
      So big picture... not sure it's worth the effort to convert the files to binary even though most of the data is numerical data in the binary file.  That was actually surprising as I expected there would be more of a difference.  Any ideas on how to get the binary data to read at a faster rate would be great.
       
      binary.au3
      2019_02_08.zip
    • By FMS
      Hello,
      I'm trying to read a binary file to an array but couln't get it to work.
      Also I coul not find any help in the forum around this subject whish was helpfull.
      Is there any way it could be done?
      I tried a lot of ways but maybe somebody know's the right way?
      #AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7 #include <File.au3> #include <Array.au3> #include <AutoItConstants.au3> Local $in=FileOpen("TEST_labels.idx1-ubyte",16) ; 16+0=Read binary Local $data = FileRead($in) Local $FileArray = BinaryToString($data,4) ;~ $FileArray = StringSplit($BinarydData, @CRLF, 1+2) ;~ Local $FileArray = StringRegExp($BinarydData, "[^\r\n]+", 3) FileClose($in) _ArrayDisplay($FileArray,"$FileArray","",32) MsgBox($MB_SYSTEMMODAL, "", "$FileArray = " & $FileArray )  
      TEST_labels.idx1-ubyte
    • By Dragonfighter
      I'm searching a way to do xor and shift and if possible also other operations. Thanks in advance for the replies.
    • By rudi
      Hello.
      I'm too stupid to see my mistake:
      To investigate the internal "dictionary" of TIFF files I'd like to read in the files in binary mode and to check, if there are more than one pages "in" this TIFF.
      Notepad++, "View as Hex" is presenting the first bytes as "49 49 2a 20 08 20 20 20 12" for the TIF attached to this posting
      The "TIFF Header Format" is easy:
      Offset 00h, 2 Byte = Byte Order, "II"=intel, "MM"=motorola. (I = 0x49)
      --> II
      Offset 02h, 2 Byte = Version Nr.
      Offset 04h, 4 Byte = pointer to first IFD entry
      Description of TIFF header: https://www.awaresystems.be/imaging/tiff/faq.html#q3
       

      Howto read and analyse the binary content correctly? This is my messy, not operational code:
       
      $sampleTiff="H:\daten\tif\11\11\111111.TIF" $h=FileOpen($sampleTiff,16) $content=FileRead($h) ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $content = ' & $content & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console FileClose($h) $type=VarGetType($content) ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $type = ' & $type & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console $ToString=BinaryToString($content) ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $ToString = ' & $ToString & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console ConsoleWrite(@CRLF & @CRLF) $content=StringTrimLeft($content,2) ; cut off the leading "0x" ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $content = ' & $content & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console for $i = 1 to 8 step 8 $next=StringMid($content,$i,2) ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $next = ' & $next & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console $Chr=BinaryToString($next) ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $Chr = ' & $Chr & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console ConsoleWrite(@CRLF & "---" & @CRLF) Next Regards, Rudi.
      111111.TIF
    • By ur
      When I am trying to compile the autoit files with aut2exe.
      I am getting below error.
      There is no issue in code as the same code is getting compiled on different machine.
      I tried reinstalling the AUtoIT, but the issue replicates.

      Any suggestions?
×
×
  • Create New...