Jump to content
Sign in to follow this  
nullschritt

String Permutation (RAM Efficient)

Recommended Posts

nullschritt

Hello, got bored and wrote a script to see how efficiently I could enumerate all possible combinations of a set of data(I was watching Elysium and saw the 4-digit lock cracking program spider user near the end, and got inspired).
 
[Note: Though the idea was inspired by use of such a program to crack a lock, I please ask you don't use my code to try to crack any locks/passwords, fun/education only please!]
 
Below is an example that fetches all the possible combinations of the digits 0-9. It takes about 60 seconds, which is pretty good consider it has to do 157,464 calculations.
 
I believe this is the fastest method of permuting data. It's also extremely efficient on ram, as it only remembers the viable results, not all results of all calculations.
 
I wish there was some way to support more or less length, but as far as I can tell there is no way to do this(I don't think you can have a dynamic amount of nested loops), if you have a way, I welcome you to show me. [if you want it to use more or less length, just add/remove for loops and values as necessary]

#include <Array.au3>

_ArrayDisplay(_permute_4('0123456789'))

func _permute_4($string)
Local $permuted = '',  $sourcearray, $aArray, $aNewArray, $opcount = 0

$sourcearray = StringSplit($string, "")
$timer = TimerInit()
for $i = 1 to $sourcearray[0]
    for $i2 = 1 to $sourcearray[0]
        for $i3=1 to $sourcearray[0]
            for $i4=1 to $sourcearray[0]
                if $i=$i2 and $i2 = $i3 and $i3 = $i4 Then
                    $permuted&="|"&$sourcearray[$i]&$sourcearray[$i2]&$sourcearray[$i3]&$sourcearray[$i4]
                    $opcount +=1
                Else
                if StringRegExp($permuted, "(.*?)("&$sourcearray[$i]&$sourcearray[$i2]&$sourcearray[$i3]&$sourcearray[$i4]&")(.*?)") Then
                ConsoleWrite("Data already permuted"&@CRLF)
                Else
                Local $aArray[4] = [$sourcearray[$i], $sourcearray[$i2], $sourcearray[$i3], $sourcearray[$i4]]
                Local $aNewArray = _ArrayPermute($aArray) ;Using Default Parameters
                                $opcount +=30
                for $ix=1 to 24
                    if Not StringRegExp($permuted, "(.*?)("&$aNewArray[$ix]&")(.*?)") and $aNewArray[$ix] <> "" Then
                    $permuted &= '|'&$aNewArray[$ix]
                    ConsoleWrite(StringLen($permuted)/5&@CRLF)
                    EndIf
                $opcount +=1
                Next
                EndIf
                EndIf
            Next
        Next
    Next
Next
$time = TimerDiff($timer)
ConsoleWrite("Took "&$time&" MS for "&$opcount&' operations.'&@CRLF)
$permuted = StringSplit(StringTrimLeft($permuted,1), "|")
return $permuted
EndFunc
Edited by nullschritt

Share this post


Link to post
Share on other sites
jchd

A 4-digit lock having 157464 combinations? :think:


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites
spudw2k

Wouldn't it be 0000 - 9999 (10000 possibilities)?
It would be even less if there could only be one of each digit in the code, but that's too much math for me this early.

Fun concept though.

edit:

It would be even less if there could only be one of each digit in the code...

5040 ;)

#include <Array.au3>

Local $hTimer = TimerInit()
Local $a = _Permute_4('012345')
ConsoleWrite("Time Taken: " & Round(TimerDiff($hTimer) / 1000, 3) & "secs" & @CRLF)

_ArrayDisplay($a, "Array Permuted")


; Returns all possible combinations of 4 characters out of $sCharsToBeUsed, the list of all possible characters from which to select.
Func _Permute_4($sCharsToBeUsed)
    Local $aArray, $aNewArray
    $aArray = StringSplit($sCharsToBeUsed, "", 2)
    $iUB = UBound($aArray)
    ;Local $aNewArray[$iUB * $iUB * $iUB * $iUB] ; 4 digits, with each digit can be 0 to 9 (each digit has 10 possibilities)
    Local $aNewArray[1]
    Local $indx=0
    For $i = 0 To $iUB - 1
        For $j = 0 To $iUB - 1
            If Not StringInstr($aArray[$i],$aArray[$j]) Then
                For $k = 0 To $iUB - 1
                    If Not StringInstr($aArray[$j]&$aArray[$i],$aArray[$k]) Then
                        For $m = 0 To $iUB - 1
                            If Not StringInstr($aArray[$k]&$aArray[$j]&$aArray[$i],$aArray[$m]) Then
                                ;$aNewArray[($i * ($iUB ^ 3)) + ($j * ($iUB ^ 2)) + ($k * $iUB) + $m] = $aArray[$i] & $aArray[$j] & $aArray[$k] & $aArray[$m]
                                ReDim $aNewArray[$indx+1]
                                $indx += 1
                                $aNewArray[$indx-1] = $aArray[$i] & $aArray[$j] & $aArray[$k] & $aArray[$m]
                            EndIf
                        Next
                    EndIf
                Next
            EndIf
        Next
    Next
    Return $aNewArray
EndFunc   ;==>_Permute_4

Edited by spudw2k

Share this post


Link to post
Share on other sites
Malkey

Here is another method without any regular expressions in the middle For...Next loop.

#include <Array.au3>

Local $hTimer = TimerInit()
Local $a = _Permute_4('0123456789')
ConsoleWrite("Time Taken: " & Round(TimerDiff($hTimer) / 1000, 3) & "secs" & @CRLF)

_ArrayDisplay($a, "Array Permuted")


; Returns all possible combinations of 4 characters out of $sCharsToBeUsed, the list of all possible characters from which to select.
Func _Permute_4($sCharsToBeUsed)
    Local $aArray, $aNewArray
    $aArray = StringSplit($sCharsToBeUsed, "", 2)
    $iUB = UBound($aArray)
    Local $aNewArray[$iUB * $iUB * $iUB * $iUB] ; 4 digits, with each digit can be 0 to 9 (each digit has 10 possibilities)
    For $i = 0 To $iUB - 1
        For $j = 0 To $iUB - 1
            For $k = 0 To $iUB - 1
                For $m = 0 To $iUB - 1
                    $aNewArray[($i * ($iUB ^ 3)) + ($j * ($iUB ^ 2)) + ($k * $iUB) + $m] = $aArray[$i] & $aArray[$j] & $aArray[$k] & $aArray[$m]
                Next
            Next
        Next
    Next
    Return $aNewArray
EndFunc   ;==>_Permute_4

Share this post


Link to post
Share on other sites
nullschritt

A 4-digit lock having 157464 combinations? :think:

It doesn't have 157,464 combinations, that's simply how many times 4 nested loops runs, estimating all combinations of 10 values, at 4 digits in length.

If you run the code you'll see it only returns 10,000 results. Most of the time is spent making sure that no values overlap from a previously permuted string.

Share this post


Link to post
Share on other sites
nullschritt

 

Here is another method without any regular expressions in the middle For...Next loop.

#include <Array.au3>

Local $hTimer = TimerInit()
Local $a = _Permute_4('0123456789')
ConsoleWrite("Time Taken: " & Round(TimerDiff($hTimer) / 1000, 3) & "secs" & @CRLF)

_ArrayDisplay($a, "Array Permuted")


; Returns all possible combinations of 4 characters out of $sCharsToBeUsed, the list of all possible characters from which to select.
Func _Permute_4($sCharsToBeUsed)
    Local $aArray, $aNewArray
    $aArray = StringSplit($sCharsToBeUsed, "", 2)
    $iUB = UBound($aArray)
    Local $aNewArray[$iUB * $iUB * $iUB * $iUB] ; 4 digits, with each digit can be 0 to 9 (each digit has 10 possibilities)
    For $i = 0 To $iUB - 1
        For $j = 0 To $iUB - 1
            For $k = 0 To $iUB - 1
                For $m = 0 To $iUB - 1
                    $aNewArray[($i * ($iUB ^ 3)) + ($j * ($iUB ^ 2)) + ($k * $iUB) + $m] = $aArray[$i] & $aArray[$j] & $aArray[$k] & $aArray[$m]
                Next
            Next
        Next
    Next
    Return $aNewArray
EndFunc   ;==>_Permute_4

I was trying to do something similar to this but couldn't figure it out. Wow. That's impressively fast!

Edit: Could you explain the math going on the calculates the position of the new string? I would like to test it with longer amounts of data.

Edited by nullschritt

Share this post


Link to post
Share on other sites
jchd

I don't get it:

#include <Array.au3>

Local $hTimer = TimerInit()
Local $a
_Permute_4('0123456789', $a)
ConsoleWrite("Time Taken: " & Round(TimerDiff($hTimer) / 1000, 3) & "secs" & @CRLF)

_ArrayDisplay($a, "Array Permuted")

$hTimer = TimerInit()
Local $b
_inc4($b)
ConsoleWrite("Time Taken: " & Round(TimerDiff($hTimer) / 1000, 3) & "secs" & @CRLF)

_ArrayDisplay($b, "Array Permuted")

Func _inc4(ByRef $b)
    Dim $b[10^4]
    For $i = 0 To UBound($b) - 1
        $b[$i] = StringRight("000" & $i, 4)
    Next
EndFunc

Exit

; Returns all possible combinations of 4 characters out of $sCharsToBeUsed, the list of all possible characters from which to select.
Func _Permute_4($sCharsToBeUsed, ByRef $aNewArray)
    Local $aArray
    $aArray = StringSplit($sCharsToBeUsed, "", 2)
    $iUB = UBound($aArray)
    Dim $aNewArray[$iUB ^ 4] ; 4 digits, with each digit can be 0 to 9 (each digit has 10 possibilities)
    For $i = 0 To $iUB - 1
        For $j = 0 To $iUB - 1
            For $k = 0 To $iUB - 1
                For $m = 0 To $iUB - 1
                    $aNewArray[($i * ($iUB ^ 3)) + ($j * ($iUB ^ 2)) + ($k * $iUB) + $m] = $aArray[$i] & $aArray[$j] & $aArray[$k] & $aArray[$m]
                Next
            Next
        Next
    Next
EndFunc   ;==>_Permute_4

Edit: I was displaying $a twice, but anyway.

Edited by jchd

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites
nullschritt

 

I don't get it:

#include <Array.au3>

Local $hTimer = TimerInit()
Local $a
_Permute_4('0123456789', $a)
ConsoleWrite("Time Taken: " & Round(TimerDiff($hTimer) / 1000, 3) & "secs" & @CRLF)

_ArrayDisplay($a, "Array Permuted")

$hTimer = TimerInit()
Local $b
_inc4($b)
ConsoleWrite("Time Taken: " & Round(TimerDiff($hTimer) / 1000, 3) & "secs" & @CRLF)

_ArrayDisplay($a, "Array Permuted")

Func _inc4(ByRef $b)
    Dim $b[10^4]
    For $i = 0 To UBound($b) - 1
        $b[$i] = StringRight("000" & $i, 4)
    Next
EndFunc

Exit

; Returns all possible combinations of 4 characters out of $sCharsToBeUsed, the list of all possible characters from which to select.
Func _Permute_4($sCharsToBeUsed, ByRef $aNewArray)
    Local $aArray
    $aArray = StringSplit($sCharsToBeUsed, "", 2)
    $iUB = UBound($aArray)
    Dim $aNewArray[$iUB ^ 4] ; 4 digits, with each digit can be 0 to 9 (each digit has 10 possibilities)
    For $i = 0 To $iUB - 1
        For $j = 0 To $iUB - 1
            For $k = 0 To $iUB - 1
                For $m = 0 To $iUB - 1
                    $aNewArray[($i * ($iUB ^ 3)) + ($j * ($iUB ^ 2)) + ($k * $iUB) + $m] = $aArray[$i] & $aArray[$j] & $aArray[$k] & $aArray[$m]
                Next
            Next
        Next
    Next
EndFunc   ;==>_Permute_4

It calculates more than just 10 digits. You can put any set of data into it to be permuted. you could for example use 'abcdefghijklmnopqrstuvwxyz' to get all 4 character combinations of the alphabet/ 

Share this post


Link to post
Share on other sites
jchd

OK:

#include <Array.au3>

Local $set = 'abcdefghijklmnopqrstuvwxyz'
Local $hTimer = TimerInit()
Local $a
_Permute_4($set, $a)
ConsoleWrite("Time Taken: " & Round(TimerDiff($hTimer) / 1000, 3) & "secs" & @CRLF)

_ArrayDisplay($a, "Array Permuted")

$hTimer = TimerInit()
Local $b
_inc4($set, $b)
ConsoleWrite("Time Taken: " & Round(TimerDiff($hTimer) / 1000, 3) & "secs" & @CRLF)

_ArrayDisplay($b, "Array Permuted")

Func _inc4($charset, ByRef $b)
    Local $n = StringLen($charset)
    Dim $b[$n ^ 4]
    Local $chars = StringSplit($charset, '', 2)
    For $i = 0 To UBound($b) - 1
        $b[$i] = $chars[Mod($i/$n^3, $n)] & $chars[Mod($i/$n^2, $n)] & $chars[Mod($i/$n, $n)] & $chars[Mod($i, $n)]
    Next
EndFunc

Exit

; Returns all possible combinations of 4 characters out of $sCharsToBeUsed, the list of all possible characters from which to select.
Func _Permute_4($sCharsToBeUsed, ByRef $aNewArray)
    Local $aArray
    $aArray = StringSplit($sCharsToBeUsed, "", 2)
    Local $iUB = UBound($aArray)
    Dim $aNewArray[$iUB ^ 4] ; 4 digits, with each digit can be 0 to 9 (each digit has 10 possibilities)
    For $i = 0 To $iUB - 1
        For $j = 0 To $iUB - 1
            For $k = 0 To $iUB - 1
                For $m = 0 To $iUB - 1
                    $aNewArray[($i * ($iUB ^ 3)) + ($j * ($iUB ^ 2)) + ($k * $iUB) + $m] = $aArray[$i] & $aArray[$j] & $aArray[$k] & $aArray[$m]
                Next
            Next
        Next
    Next
EndFunc   ;==>_Permute_4

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites
nullschritt

 

OK:

#include <Array.au3>

Local $set = 'abcdefghijklmnopqrstuvwxyz'
Local $hTimer = TimerInit()
Local $a
_Permute_4($set, $a)
ConsoleWrite("Time Taken: " & Round(TimerDiff($hTimer) / 1000, 3) & "secs" & @CRLF)

_ArrayDisplay($a, "Array Permuted")

$hTimer = TimerInit()
Local $b
_inc4($set, $b)
ConsoleWrite("Time Taken: " & Round(TimerDiff($hTimer) / 1000, 3) & "secs" & @CRLF)

_ArrayDisplay($b, "Array Permuted")

Func _inc4($charset, ByRef $b)
    Local $n = StringLen($charset)
    Dim $b[$n ^ 4]
    Local $chars = StringSplit($charset, '', 2)
    For $i = 0 To UBound($b) - 1
        $b[$i] = $chars[Mod($i/$n^3, $n)] & $chars[Mod($i/$n^2, $n)] & $chars[Mod($i/$n, $n)] & $chars[Mod($i, $n)]
    Next
EndFunc

Exit

; Returns all possible combinations of 4 characters out of $sCharsToBeUsed, the list of all possible characters from which to select.
Func _Permute_4($sCharsToBeUsed, ByRef $aNewArray)
    Local $aArray
    $aArray = StringSplit($sCharsToBeUsed, "", 2)
    Local $iUB = UBound($aArray)
    Dim $aNewArray[$iUB ^ 4] ; 4 digits, with each digit can be 0 to 9 (each digit has 10 possibilities)
    For $i = 0 To $iUB - 1
        For $j = 0 To $iUB - 1
            For $k = 0 To $iUB - 1
                For $m = 0 To $iUB - 1
                    $aNewArray[($i * ($iUB ^ 3)) + ($j * ($iUB ^ 2)) + ($k * $iUB) + $m] = $aArray[$i] & $aArray[$j] & $aArray[$k] & $aArray[$m]
                Next
            Next
        Next
    Next
EndFunc   ;==>_Permute_4

Smart. Significant preformance impact. I tried to beef it up to 6 digits but I got an error saying the maximum array size was exceeded :(

Share this post


Link to post
Share on other sites
jchd

AutoIt arrays have a limit of 16 millions cells, which (as I just tested) is in fact 16 * 1024 * 1024 = 16 777 216.

266 = 308 915 776


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites
UEZ

Here a recursion variant of permutation (at least what I understand of the word permutation):

#include <Array.au3>

Global $string = "ABCD", $perms = 0
Permutator(1, StringLen($string), $string)
ConsoleWrite("Permutation: " & $perms & @LF)

Func Permutator($a, $b, $s) ;coded by UEZ 2014
    If $a = $b Then
        ConsoleWrite($s & @CRLF)
        $perms += 1
    Else
        Local $aChars = StringSplit($s, ""), $t, $i
        For $i = $a to $b
            $t = $aChars[$a]
            $aChars[$a] = $aChars[$i]
            $aChars[$i] = $t
            Permutator($a + 1, $b, _ArrayToString($aChars, "", 1))
        Next
    EndIf
EndFunc

Recusion is unfortunatelly limited to the recursion stack size!
 
Br,
UEZ

Edited by UEZ

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites
nullschritt

Here a recursion variant of permutation (at least what I understand of the word permutation):

#include <Array.au3>

Global $string = "ABCD", $perms = 0
Permutator(1, StringLen($string), $string)
ConsoleWrite("Permutation: " & $perms & @LF)

Func Permutator($a, $b, $s) ;coded by UEZ 2014
    If $a = $b Then
        ConsoleWrite($s & @CRLF)
        $perms += 1
    Else
        Local $aChars = StringSplit($s, ""), $t, $i
        For $i = $a to $b
            $t = $aChars[$a]
            $aChars[$a] = $aChars[$i]
            $aChars[$i] = $t
            Permutator($a + 1, $b, _ArrayToString($aChars, "", 1))
        Next
    EndIf
EndFunc

Recusion is unfortunatelly limited to the recursion stack size!

 

Br,

UEZ

Seems efficient but the length of the data should be independent of the length of possible data. (eg we should be able to set it to 4 digits even if the data set is 10 long)

Share this post


Link to post
Share on other sites
jchd

Unfortunately Execute takes exponential time over the complexity of fed instruction (i.e. be patient!):

#include <Array.au3>

Local $set = 'abcdefghijkl'
Local $hTimer = TimerInit()
Local $a
_Permute_4($set, $a)
ConsoleWrite("Time Taken: " & Round(TimerDiff($hTimer) / 1000, 3) & "secs" & @CRLF)

_ArrayDisplay($a, "Array Permuted")

$hTimer = TimerInit()
Local $b
_incN($set, 5, $b)
ConsoleWrite("Time Taken: " & Round(TimerDiff($hTimer) / 1000, 3) & "secs" & @CRLF)

_ArrayDisplay($b, "Array Permuted")

Func _incN($charset, $size, ByRef $b)
    Local $chars
    #forceref $chars
    $size = Int($size)
    If $size < 1 Then Return(SetError(1))
    Local $n = StringLen($charset)
    If $n ^ $size > 16 * 1024 * 1024 Then Return(SetError(2))
    Dim $b[$n ^ $size]
    $chars = StringSplit($charset, '', 2)
    Local $inst
    For $i = $size To 2 Step -1
        $inst &= "$chars[Mod($i/$n^" & $i & ", $n)] & "
    Next
    $inst &= "$chars[Mod($i, $n)]"
    For $i = 0 To UBound($b) - 1
        $b[$i] = Execute($inst)
    Next
EndFunc



Exit

; Returns all possible combinations of 4 characters out of $sCharsToBeUsed, the list of all possible characters from which to select.
Func _Permute_4($sCharsToBeUsed, ByRef $aNewArray)
    Local $aArray
    $aArray = StringSplit($sCharsToBeUsed, "", 2)
    Local $iUB = UBound($aArray)
    Dim $aNewArray[$iUB ^ 4] ; 4 digits, with each digit can be 0 to 9 (each digit has 10 possibilities)
    For $i = 0 To $iUB - 1
        For $j = 0 To $iUB - 1
            For $k = 0 To $iUB - 1
                For $m = 0 To $iUB - 1
                    $aNewArray[($i * ($iUB ^ 3)) + ($j * ($iUB ^ 2)) + ($k * $iUB) + $m] = $aArray[$i] & $aArray[$j] & $aArray[$k] & $aArray[$m]
                Next
            Next
        Next
    Next
EndFunc   ;==>_Permute_4
Edited by jchd

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites
wraithdu

I wrote this a while back. Just added a callback option for result sets that would be too big for an array.

#include-once
; #FUNCTION# ;===================================================================================================================
; Name...........: _DuplicatePermute
; Description ...: Returns an array of permutations (incl duplicates) of a given length from items in an array
; Syntax.........: _DuplicatePermute(Const Byref $aArray, $iLen[, $sDelim = ""[, $cFunc = Default]])
; Parameters ....: $aArray         - The array to get permutations
;                  $iLen           - Length of each permutation
;                  $sDelim         - [optional] String result separator, default is "" for none
;                  $cFunc          - [optional] Callback function to be called with each permutation. The function will be
;                                    called with 3 parameters:
;                                        1 - count of current permutation
;                                        2 - count of total permutations
;                                        3 - current permutation as a string
;                                    The function must return non-zero to continue.
; Return values .: Success - Returns an Array of permutations (incl duplicates)
;                  Returns an array, the first element ($array[0]) contains the number of strings returned. If $cFunc is Default,
;                  the remaining elements ($array[1], $array[2], etc.) contain the permutations. Otherwise the return array has
;                  2 elements and $array[1] contains the number of permutations processed.
;                  Failure - Returns 0 and Sets @error:
;                  1 - The input must be an array
;                  2 - The function stopped before processing all permutations
; Author ........: Erik Pilsits
; Modified.......: 02/07/2014
; Remarks .......: The input array must be 0-based, ie no counter in $array[0]
;                  Based on an algorithm by Sandro Magi
; Related .......:
; Link ..........: http://higherlogics.blogspot.com/2008/04/permutations-with-duplicates-in-c.html
;
; Example .......:
; ===============================================================================================================================
Func _DuplicatePermute(Const ByRef $aArray, $iLen, $sDelim = "", $cFunc = Default)
    If Not IsArray($aArray) Then
        Return SetError(1, 0, 0)
    EndIf

    Local Const $n = UBound($aArray)
    Local $avSlots[$iLen], $i, $avResult, $count, $total

    $total = $n ^ $iLen
    If $cFunc = Default Then
        Local $avResult[$total + 1]
        $avResult[0] = $total
    Else
        Local $avResult[2] = [$total]
    EndIf
    $count = 1
    For $i = 0 To $iLen - 1
        $avSlots[$i] = 0
    Next
    While _DuplicateInternal($aArray, $avSlots, $iLen, $n, $sDelim, $avResult, $count, $total, $cFunc)
    WEnd
    Local $err = @error
    If $cFunc <> Default Then $avResult[1] = $count
    Return SetError($err, 0, $avResult)
EndFunc   ;==>_DuplicatePermute

Func _DuplicateInternal(Const ByRef $aArray, ByRef $avSlots, $iLen, $n, $sDelim, ByRef $avResult, ByRef $count, $total, $cFunc)
    Local $i, $b, $carry = 1, $str = ""

    For $i = 0 To $iLen - 1
        $str &= $aArray[$avSlots[$i]] & $sDelim
    Next
    If $sDelim <> "" Then $str = StringTrimRight($str, 1)
    If $cFunc = Default Then
        $avResult[$count] = $str
    Else
        If Not $cFunc($count, $total, $str) Then Return SetError(2, 0, 0)
    EndIf
    $count += 1

    For $i = 0 To $iLen - 1
        $b = $avSlots[$i] + $carry
        $carry = Int($b / $n)
        $avSlots[$i] = Mod($b, $n)
    Next
    Return Not $carry
EndFunc   ;==>_DuplicateInternal

Example

#include <Array.au3>
#include <_DuplicatePermute.au3>

Global $a = StringSplit("0123456789", "", 2)
$b = _DuplicatePermute($a, 3, "")
_ArrayDisplay($b)
$b = _DuplicatePermute($a, 3, "", myfunc)
_ArrayDisplay($b)

Func myfunc($c, $t, $s)
    ConsoleWrite($c & " : " & $t & " : " & $s & @CRLF)
    If $c > 9 Then
        Return 0
    Else
        Return 1
    EndIf
EndFunc

Not tested for speed, but I think it's pretty good. It's iterative too, so no worries about hitting the recursion limit.

Edited by wraithdu

Share this post


Link to post
Share on other sites
Chimp

#include <array.au3>
Global $string = "ABCDE"
$hTimer = TimerInit()
Local $chars = StringSplit($string, '', 2)
Local $Permutations = _ArrayPermute($chars)
ConsoleWrite("Time Taken: " & Round(TimerDiff($hTimer) / 1000, 3) & "secs" & @CRLF)
_ArrayDisplay($Permutations)

Edited by PincoPanco

small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Share this post


Link to post
Share on other sites
wraithdu

That's actually my function also. Difference is though, that version doesn't produce duplicates, so you won't get 0000, 1111, 0011, etc. It also only uses the full data set, so you can't do permutations of 4 elements out of a set of 10.

Share this post


Link to post
Share on other sites
nullschritt
#include <array.au3>
Global $string = "ABCDE"
$hTimer = TimerInit()
Local $chars = StringSplit($string, '', 2)
Local $Permutations = _ArrayPermute($chars)
ConsoleWrite("Time Taken: " & Round(TimerDiff($hTimer) / 1000, 3) & "secs" & @CRLF)
_ArrayDisplay($Permutations)

As stated this results in replicates and only produces data the length of the input data. You'll see my very first example uses the _arraypermute() function inside of several loops(with regexs) to simulate support for say x chars in a set of 4 digits, without repeating digits, though the other posts are considerably quicker!

Share this post


Link to post
Share on other sites
wraithdu

If you want no repeating digits, then use _ArrayPermute along with _ArrayCombinations to permute each unique combination. The UDF functions aren't written with array size in mind, so be careful about the size of your result set. Example:

#include <Array.au3>

Global $a = StringSplit("0123456789", "", 2)
Global $b = _ArrayCombinations($a, 4)
Global $c, $d, $e[1] = [0]
For $i = 1 To $b[0]
    ; get combination and create array
    $c = StringSplit($b[$i], "", 2)
    ; permute
    $d = _ArrayPermute($c)
    ; build result
    $e[0] += $d[0]
    _ArrayConcatenate($e, $d, 1)
Next
_ArrayDisplay($e)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Similar Content

    • liagason
      By liagason
      Hello everyone,
      How can I display in ascending  sequence some numbers stored in a string variable?
      $str = "18,03,48,23" MsgBox(0,"test",$str) I would like it to display "03,18,23,48"
    • Rskm
      By Rskm
      Hi, I have the following line in a text file 'input.txt'. I know the line number - say '6'. I wish to replace the text 'WWW' in the below line with a random number (I can generate that with random()).
      WERIS  WWWJP   3.83  8.330  1.000                1097.RAXX 
      The WWW is a 3 digit integer (could be any number between 0 to 999), I can use stringtrimleft and get the numerical value of WWW in this file
      so, basically, I know the string to replace (ie; WWW stored in a variable), I know the line number to work on and the file location/name and the replacement variable (through random()). My requirement is to fill that 3 spaces with my random number (which Is a integer between 1 and 999)
      please put ur suggestions
       
    • nacerbaaziz
      By nacerbaaziz
      Hello Members of this best Forum
      i have a question please
      for example if i have a long string
      and i want to extract a text between two tag
      what i can do to make that?
      note :
      i know that there is a
      StringRegExp function
      it's do that work
      but it result is be as an array
      i want the result to be a string
      is there any function on autoit can do that?
      Thanks in advance.
    • mistersquirrle
      By mistersquirrle
      Hello!
       
      I wrote myself a script to follow Google Maps Polyline encoding steps: https://developers.google.com/maps/documentation/utilities/polylinealgorithm, and that works (although I think that it's a bit janky), but now I'm having issues getting the output.
       
      When I run the script, all the points come out correctly in the console, and even when they're the only things that I log, it displays them fine. However, I'm adding each point into a variable to return all of them at once at the end, fully formatted, and it's only taking the very first point. I can't figure out what I'm doing wrong, as it seems fine.
       
      When run with the default value, it should output this at the end: Custom Polygon: _p~iF~ps|U_ulLnnqC_mqNvxq`@
      But instead I'm just getting this: Custom Polygon: _p~iF
       
      I know that it's reaching the string combination lines because it's logging the data before it (and even if a put log AFTER the $sPolygon &= $aPoints[0], it's logged fine).
       
      Here's my full code (problem is lines ~209 - 234, search "$sPolygon &= $aPoints[1]"):
      #include <Array.au3> #include <ButtonConstants.au3> #include <EditConstants.au3> #include <GUIConstantsEx.au3> #include <StaticConstants.au3> #include <WindowsConstants.au3> _PolyGUI() Func _PolyGUI() #Region ### START Koda GUI section ### Form= $hInputGUI = GUICreate("Lat Long encoder", 403, 301, 192, 124) GUISetFont(8, 400, 0, "Consolas") GUICtrlCreateLabel("Input polygon points here, format as:", 8, 8, 263, 19) GUICtrlSetFont(-1, 10, 800, 0, "Consolas") GUICtrlCreateLabel("Lat Long - Single point", 8, 24, 142, 17) GUICtrlCreateLabel("Lat Long, Lat Long, Lat Long - Multiple points", 8, 40, 280, 17) Local $sPoints = GUICtrlCreateEdit("", 8, 64, 385, 201, BitOR($ES_WANTRETURN, $WS_VSCROLL)) GUICtrlSetData(-1, "38.5 -120.2, 40.7 -120.95, 43.252 -126.453") GUICtrlSetFont(-1, 10, 400, 0, "Consolas") $bOK = GUICtrlCreateButton("bOK", 16, 272, 123, 25) GUICtrlSetFont(-1, 12, 800, 0, "Consolas") $bCancel = GUICtrlCreateButton("bCancel", 304, 272, 75, 25) GUICtrlSetFont(-1, 12, 800, 0, "Consolas") GUISetState(@SW_SHOW, $hInputGUI) #EndRegion ### END Koda GUI section ### While 1 $nMsg = GUIGetMsg() Switch $nMsg Case $GUI_EVENT_CLOSE Exit Case $bCancel Exit Case $bOK $sPoints = GUICtrlRead($sPoints) GUISetState(@SW_HIDE, $hInputGUI) _GetPoly($sPoints, True) ExitLoop EndSwitch Sleep(10) WEnd EndFunc ;==>_PolyGUI ;https://developers.google.com/maps/documentation/utilities/polylinealgorithm ;https://app.dsmobileidx.com/api/DescribeSearchForLinkId?linkId=469787 ; Note that this will only really work inside the US (this side of the World), as it's assuming any negative is the Longitude ;https://gist.github.com/ismaels/6636986 - decoder ;Using: 41.83162 -87.64696 ; Expected: sfi~F np}uO ; Actual: sfi~f np}uo ; If we remove 32 from the last ASCII code, since the last bit chunk is 0, we get the correct case/ characters ; We need to run this logic back through all the indexes though and do this to all that that <= 63 ;LinkId=469787 ; Expected: q{`aHpa_iVi[kp@}`Aa{@e[eCoqBbAyc@iRy{@g_@mz@|gA{eAh~@Vf~Etv@gB~p@gQ`^yg@~p@ekAldA{KfFxIrJ^pO~Mtl@dPrJnUz[nSpo@wf@fc@yw@n@ob@ ; Actual: s{`aHpa_iVg[kp@}`Aa{@g[gCmqBbA{c@iRy{@e_@kz@|gA{eAh~@Td~Evv@gB|p@gQb^wg@|p@ekAndA{KfFvIpJ`@rO~Mrl@dPrJnU|[lSpo@wf@dc@yw@n@mb@ ; I assume that this is because of bad data, the points have repeating 9's and 0's, which looks fishy. The polygon is (very) close, but not quite the same. Func _GetPoly($sPoints, $bLog = False) Local $timer = TimerInit(), $sConsole[11] Local $sPolygon = "" ; Step 1, take the initial signed value: Local $aCoords = StringRegExp($sPoints, "(-*?\d*\.\d*) (-*?\d*\.\d*)", 3), $aPoints[2] ;~ _ArrayDisplay($aCoords) If $bLog Then _Log(_ArrayToString($aCoords)) For $c = 0 To (UBound($aCoords) - 1) Step 2 ;~ If $bLog Then _Log($c) If $c = 0 Then $aPoints[0] = $aCoords[$c] $aPoints[1] = $aCoords[$c + 1] Else $aPoints[0] = $aCoords[$c] - $aCoords[$c - 2] $aPoints[1] = $aCoords[$c + 1] - $aCoords[$c - 1] EndIf If $bLog Then _Log("- Step 1, take the initial signed value:") _Log(" " & $aPoints[0]) _Log(" " & $aPoints[1]) EndIf ; Step 2, multiply each by 1e5, and round $aPoints[0] = Round($aPoints[0] * 1e5, 0) $aPoints[1] = Round($aPoints[1] * 1e5, 0) If $bLog Then _Log("- Step 2, multiply each by 1e5, and round") _Log(" " & $aPoints[0]) _Log(" " & $aPoints[1]) EndIf ; Step 3, convert Decimal to Binary, using two's complement for negatives. Padded to 32 bits $aPoints[0] = _NumberToBinary($aPoints[0]) $aPoints[1] = _NumberToBinary($aPoints[1]) If $bLog Then _Log("- Step 3, convert Decimal to Binary, using two's complement for negatives. Padded to 32 bits") _Log(" " & $aPoints[0]) _Log(" " & $aPoints[1]) EndIf ; Step 4, left-shifted 1 bit $aPoints[0] = StringTrimLeft($aPoints[0], 1) & "0" $aPoints[1] = StringTrimLeft($aPoints[1], 1) & "0" If $bLog Then _Log("- Step 4, left-shifted 1 bit") _Log(" " & $aPoints[0]) _Log(" " & $aPoints[1]) EndIf ; Step 5, if negative, invert binary If $c = 0 Then If $aCoords[$c] < 0 Then $aPoints[0] = _InvertBinary($aPoints[0]) If $aCoords[$c + 1] < 0 Then $aPoints[1] = _InvertBinary($aPoints[1]) Else If $aCoords[$c] - $aCoords[$c - 2] < 0 Then $aPoints[0] = _InvertBinary($aPoints[0]) If $aCoords[$c + 1] - $aCoords[$c - 1] < 0 Then $aPoints[1] = _InvertBinary($aPoints[1]) EndIf If $bLog Then _Log("- Step 5, if negative, invert binary") _Log(" " & $aPoints[0]) _Log(" " & $aPoints[1]) EndIf Local $aChunks[2][6], $0x20 For $i = 0 To 1 $0x20 = "1" ; This is out BitOR flag, 0x20 BitOR'd onto our 5-bit chunks is always 1*****, except the last chunk $sConsole[5] = "" ; Clearing console variables $sConsole[6] = "" $sConsole[7] = "" $sConsole[8] = "" $sConsole[9] = "" For $j = 0 To 5 ;There will always be 6 chunks ; Step 6 & 7, break into 5-bit chunks, and reverse order $aChunks[$i][$j] = StringTrimLeft($aPoints[$i], StringLen($aPoints[$i]) - 5) ; This splits into 5-bit chunks in reverse order, doing 6 & 7 in one operation ;~ If $bLog Then _Log(" " & $aPoints[$i]) ;~ If $bLog Then _Log(" " & StringLen($aPoints[$i])) ;~ If $bLog Then _Log(" " & StringTrimLeft($aPoints[$i], StringLen($aPoints[$i]) - 5)) ;~ If $bLog Then _Log(" " & $aChunks[$i][$j]) ; Here we consume the original binary string, so the next loop gets the correct next 5-bit chunk $aPoints[$i] = StringTrimRight($aPoints[$i], 5) $sConsole[5] &= $aChunks[$i][$j] & " " ; Once consumed, if the remaining length isn't enough for another bit chunk, switch 0x20 to 0 (no following chunks) If StringLen($aPoints[$i]) <= 5 Then $0x20 = "0" ; Step 8, BitOR 100000 (0x20) to our 5-bit chunks (effectively) $aChunks[$i][$j] = $0x20 & $aChunks[$i][$j] $sConsole[7] &= $aChunks[$i][$j] & " " ; Step 9, converting the chunk from Binary back to Decimal $aChunks[$i][$j] = _BinaryToDec($aChunks[$i][$j]) $sConsole[8] &= $aChunks[$i][$j] & " " ; Step 10, adding 63 to decimal values $aChunks[$i][$j] += 63 $sConsole[9] &= $aChunks[$i][$j] & " " If StringLen($aPoints[$i]) < 5 Then ExitLoop Next If $bLog Then _Log("- Step 6 & 7 (part " & $i & "), break into 5-bit chunks, and reverse order") _Log(" " & $sConsole[5]) _Log("- Step 8 (part " & $i & "), BitOR 100000 (0x20) to our 5-bit chunks (effectively)") _Log(" " & $sConsole[7]) _Log("- Step 9 (part " & $i & "), converting the chunk from Binary back to Decimal") _Log(" " & $sConsole[8]) _Log("- Step 10 (part " & $i & "), adding 63 to decimal values") _Log(" " & $sConsole[9]) EndIf Next Local $aASCII[0] For $i = 0 To 1 Dim $aASCII[0] ; Reset ASCII array For $j = 0 To (UBound($aChunks, 2) - 1) ; For both chunk sets ReDim $aASCII[UBound($aASCII) + 1] ; Add an index for the ASCII array If $aChunks[$i][$j] = "" Or $aChunks[$i][$j] <= 63 Then ; If the chunk is not useful $l = $j For $k = $l To 1 Step -1 If $aChunks[$i][$k] = "" Or $aChunks[$i][$k] <= 63 Or $aASCII[$k] <= 63 Then $aASCII[$k - 1] -= 32 If $aASCII[$k - 1] <= 63 Then _ArrayDelete($aASCII, $k - 1) Else ExitLoop EndIf Next ExitLoop EndIf $aASCII[$j] = Int($aChunks[$i][$j]) Next ;Step 11, convert each value to ASCII equivalent For $k = UBound($aASCII) - 1 To 0 If $aASCII[$k] <= 63 Or $aASCII[$k] = "" Then ReDim $aASCII[UBound($aASCII) - 1] Else ExitLoop EndIf Next $aPoints[$i] = StringFromASCIIArray($aASCII, 0, -1, 0) Next If $bLog Then _Log("- Step 11, convert each value to ASCII equivalent, finished") If $aCoords[$c] <= 0 Then ;@CRLF & " " & If $bLog Then _Log($aPoints[1]) _Log($aPoints[0]) _Log("Next set") EndIf $sPolygon &= $aPoints[1] $sPolygon &= $aPoints[0] Else If $bLog Then _Log($aPoints[0]) _Log($aPoints[1]) _Log("Next set") EndIf $sPolygon &= $aPoints[0] $sPolygon &= $aPoints[1] EndIf Next If $bLog Then _Log("Custom Polygon: " & $sPolygon) _Log(TimerDiff($timer) & @CRLF) EndIf Return $sPolygon EndFunc ;==>_GetPoly Func _NumberToBinary($iNumber) Local $sBinString = "" ; Maximum 32-bit # range is -2147483648 to 2147483647 If $iNumber < -2147483648 Or $iNumber > 2147483647 Then Return SetError(1, 0, "") ; Convert to a 32-bit unsigned integer. We can't work on signed #'s $iUnsignedNumber = BitAND($iNumber, 0x7FFFFFFF) ; Cycle through each bit, shifting to the right until 0 Do $sBinString = BitAND($iUnsignedNumber, 1) & $sBinString $iUnsignedNumber = BitShift($iUnsignedNumber, 1) Until Not $iUnsignedNumber ; Was it a negative #? Put the sign bit on top, and pad the bits that aren't set If $iNumber < 0 Then Return '1' & StringRight("000000000000000000000000000000" & $sBinString, 31) ; Always return 32 bit binaries If StringLen($sBinString) < 32 Then Return StringRight("0000000000000000000000000000000" & $sBinString, 32) Return $sBinString EndFunc ;==>_NumberToBinary Func _BinaryToDec($sBinary) Local Const $aPower[8] = [128, 64, 32, 16, 8, 4, 2, 1] Local $iDec If StringRegExp($sBinary, "[0-1]") Then If StringLen($sBinary) < 8 Then Do $sBinary = "0" & $sBinary Until StringLen($sBinary) = 8 EndIf $aBinary = StringSplit($sBinary, "", 2) For $i = 0 To UBound($aBinary) - 1 ;~ $aBinary[$i] = $aBinary[$i] * $aPower[$i] $iDec += $aBinary[$i] * $aPower[$i] Next Return $iDec Else Return SetError(0, 0, "Not a binary string") EndIf EndFunc ;==>_BinaryToDec Func _InvertBinary($iNumber) ;~ ConsoleWrite(@CRLF & $iNumber) Local $sNumber $aNumber = StringSplit($iNumber, "") For $i = 1 To $aNumber[0] If $aNumber[$i] = 0 Then $aNumber[$i] = 1 ElseIf $aNumber[$i] = 1 Then $aNumber[$i] = 0 Else Return SetError(0, 0, "Not a binary number") EndIf $sNumber &= String($aNumber[$i]) Next Return $sNumber EndFunc ;==>_InvertBinary Func _Log($data) ;~ Local Static $LogEnable = True ConsoleWrite(@CRLF & @HOUR & ":" & @MIN & "." & @SEC & " " & $data) LogData(@HOUR & ":" & @MIN & "." & @SEC & " " & $data, "logs/LOGFILE.txt") EndFunc ;==>_Log Func LogData($text, $File = "logs/LOGFILE.txt") Global $LogFile = "" If $LogFile = "" Then $LogFile = FileOpen($File, 9) OnAutoItExitRegister(CloseLog) EndIf FileWriteLine($LogFile, $text) EndFunc ;==>LogData Func CloseLog() If $LogFile <> "" Then _Log("Closing LoD script" & @CRLF) FileClose($LogFile) EndIf EndFunc ;==>CloseLog  
      I've tried:
      $sPolygon &= $aPoints[0] & $aPoints[1] ;---- $sPolygon = $sPolygon & $aPoints[0] & $aPoints[1] ;---- $sPolygon = $sPolygon & String($aPoints[0] & $aPoints[1]) ;---- $sPolygon = String($sPolygon) & String($aPoints[0]) & String($aPoints[1]) ;---- $sPolygon &= $aPoints[1] $sPolygon &= $aPoints[0] ;----  
      I'm sure it's something basic that I'm overlooking, but I don't understand why it's not combining the strings. 
      Also, unrelated, why doesn't $LogFile = FileOpen($File, 9) create the directory/ file if they don't exist? 9 should be $FO_CREATEPATH (8) + $FO_APPEND (1)?
      Thanks!
    • careca
      By careca
      This is another take on string triggers, triggers on specific strings.
      Able to simple text pasting,
      opening links (as long as there's a www. http:\\ or https:\\ at the beggining)
      and is able to open applications.
      The user selects the modifier key, and then uses a combination of that key with a couple others to perform tasks like
      screenshot the active window, (modkey + prtscr), turn off the screen (modkey + pause / break),
      open clipboard string in registry (modkey + R), change system volume (modkey + arrouw up/dn).
      The following keys pressed at the same time prompt for shutdown: S+D+T
      The following keys pressed at the same time prompt for restart: S+R+T
      Middle mouse button click on titlebar minimizes to tray, or a left mouse button click in the icon in the tray also minimizes.
      Trigger is set off by space or enter, and timeouts after 3 seconds.
      Shows your external, lan, and gateway ip's, can refresh with right mouse click, and opens the default browser if the correspondent button is pressed.
      Able to change system volume by a set percentage, reading from the inputbox the number the user sets, if 0 or empty uses system default.
      I made this because the existing string trigger applications didn't do it for me for a number of reasons.
      I did this for me, but if someone finds it useful all the better.
×