Jump to content
CFLam

Generate unique 8 numbers combination from 8 arrays issue (slow)

Recommended Posts

CFLam
Posted (edited)

Hello all,

    I have a array combinations problem

I need to generate a list of unique 8-numbers array (A,B,C,D,E,F,G) from 8 lines (arrays) of number in 1 file as below:
 

Line 1 17,1,2,3,4,5,6,7,10,11,12,13,15,16,20,22,24,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
Line 2 16,1,2,3,4,5,6,7,8,9,10,11,12,16,17,21,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
Line 3 23,4,5,6,7,8,9,10,12,13,14,15,16,17,18,19,20,23,25,26,28,29,35,36,,,,,,,,,,,,,,,,,,,,,,,,,,
Line 4 27,8,9,11,12,13,14,15,17,19,20,21,22,23,24,26,27,28,29,30,31,32,34,37,39,40,41,42,,,,,,,,,,,,,,,,,,,,,,
Line 5 27,14,16,17,18,19,20,22,23,24,25,26,27,28,29,30,31,32,33,35,36,37,38,39,40,42,43,44,,,,,,,,,,,,,,,,,,,,,,
Line 6 24,20,24,26,27,28,29,30,31,32,33,34,35,36,37,39,40,41,42,43,44,45,46,47,48,,,,,,,,,,,,,,,,,,,,,,,,,
Line 7 16,33,34,35,36,37,38,39,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
Line 8 14,35,37,38,39,40,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

i.e. A from Line 1 and B from Line 2 and so on, the 1st number of each line is
the number count in each line i.e. 17 at Line 1 mean that there is 17 number in Line 1. Number in each line is unique and in ascending order, range from 1~49.
I read in the line file into an array and using for loop to do the job
 

func MyArrayGeneration()
 local $LineArr,$uArr
 dim $MyRes[8]

 _FileReadToArray("mylinefile.txt",$LineArr,$FRTA_ENTIRESPLIT , ",")

 $OutName="MyOutPut.txt"
 $OutFile=fileopen($OutName,10)

 $ResuCnt=0
 ;form the gen draw
 for $i=1 to $LineArr[0][0]
  for $j=1 to $LineArr[1][0]
   for $k=1 to $LineArr[2][0]
    for $l=1 to $LineArr[3][0]
     for $m=1 to $LineArr[4][0]
      for $n=1 to $LineArr[5][0]
       for $o=1 to $LineArr[6][0]
        for $p=1 to $LineArr[7][0]
         $MyRes[0]=$LineArr[0][$i]
         $MyRes[1]=$LineArr[1][$j]
         $MyRes[2]=$LineArr[2][$k]
         $MyRes[3]=$LineArr[3][$l]
         $MyRes[4]=$LineArr[4][$m]
         $MyRes[5]=$LineArr[5][$n]
         $MyRes[6]=$LineArr[6][$o]
         $MyRes[7]=$LineArr[7][$p]
         $uArr=_ArrayUnique($MyRes)
         if $uArr[0]=8 then
          _arraysort($MyRes)
          _arraytranspose($MyRes)
          _FileWriteFromArray($OutFile,$MyRes,0,Default,",")
          _arraytranspose($MyRes)
         endif ;$uArr[0]=8
         $uArr=""
        next
       next
      next
     next
    next
   next
  next
 next
 fileclose($OutFile)
endfunc

and the result line cannot contain duplicated number i.e. 1,1,4,8,14,20,33,35 is not allowed
therefor I use arrayunique to test if the formed array has repeated number(s), if not, output to a file.

However, I find that this method is very s-l--o---w! Take over 2 hrs
and still not completed 1 file (I have 800 files need to handled!!). Also, this approach cannot generate unique combination i.e.

 

1,2,4,8,14,20,33,35

may generate more than 1 time!!!! Is there any better method/algorithm/library to handle this ?
Thanks a lot.

Regds
LAM Chi-fung

Edited by CFLam

Share this post


Link to post
Share on other sites
Subz

Are you reading this data from a file?  It kind of looks like a csv file?  If so can you post an example file?

Share this post


Link to post
Share on other sites
Subz

Will assume you're reading from a file in which case you would probably use FileReadToArray you could write the rest as follows:

#include <Array.au3>

Local $aLine[8]
    $aLine[0] = "17,1,2,3,4,5,6,7,10,11,12,13,15,16,20,22,24,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[1] = "16,1,2,3,4,5,6,7,8,9,10,11,12,16,17,21,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[2] = "23,4,5,6,7,8,9,10,12,13,14,15,16,17,18,19,20,23,25,26,28,29,35,36,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[3] = "27,8,9,11,12,13,14,15,17,19,20,21,22,23,24,26,27,28,29,30,31,32,34,37,39,40,41,42,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[4] = "27,14,16,17,18,19,20,22,23,24,25,26,27,28,29,30,31,32,33,35,36,37,38,39,40,42,43,44,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[5] = "24,20,24,26,27,28,29,30,31,32,33,34,35,36,37,39,40,41,42,43,44,45,46,47,48,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[6] = "16,33,34,35,36,37,38,39,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[7] = "14,35,37,38,39,40,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"

For $i = 0 To UBound($aLine) - 1
    _StringParser($aLine[$i])
Next

Func _StringParser($_sString)
    Local $aString = StringSplit($_sString, ",", 2)
    ;~ Delete the first number
    _ArrayDelete($aString, 0)
    Local $aUnique = _ArrayUnique($aString)
    ;~ Delete the last line since it will be blank
    If $aUnique[$aUnique[0]] = "" Then _ArrayDelete($aUnique, $aUnique[0])
    $aUnique[0] = UBound($aUnique) - 1
    _ArrayDisplay($aUnique)
    Local $sUnique = _ArrayToString($aUnique, ",")
    MsgBox(32, "String to Write to File", $sUnique)
EndFunc

 

Share this post


Link to post
Share on other sites
kylomas
Posted (edited)

Similar to _ArrayUnique...

#include <Array.au3>

Local $aLine[8]
$aLine[0] = "17,1,2,3,4,5,6,7,10,11,12,13,15,16,20,22,24,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
$aLine[1] = "16,1,2,3,4,5,6,7,8,9,10,11,12,16,17,21,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
$aLine[2] = "23,4,5,6,7,8,9,10,12,13,14,15,16,17,18,19,20,23,25,26,28,29,35,36,,,,,,,,,,,,,,,,,,,,,,,,,,"
$aLine[3] = "27,8,9,11,12,13,14,15,17,19,20,21,22,23,24,26,27,28,29,30,31,32,34,37,39,40,41,42,,,,,,,,,,,,,,,,,,,,,,"
$aLine[4] = "27,14,16,17,18,19,20,22,23,24,25,26,27,28,29,30,31,32,33,35,36,37,38,39,40,42,43,44,,,,,,,,,,,,,,,,,,,,,,"
$aLine[5] = "24,20,24,26,27,28,29,30,31,32,33,34,35,36,37,39,40,41,42,43,44,45,46,47,48,,,,,,,,,,,,,,,,,,,,,,,,,"
$aLine[6] = "16,33,34,35,36,37,38,39,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
$aLine[7] = "14,35,37,38,39,40,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"

Local $oDict = ObjCreate("Scripting.Dictionary")
Local $atmp, $output

For $1 = 0 To UBound($aLine) - 1
    $atmp = StringSplit($aLine[$1], ',', 3)
    For $2 = 1 To UBound($atmp) - 1
        If $atmp[$2] = '' Then ContinueLoop
        $oDict.Item($atmp[$2])
    Next
    $atmp = $oDict.keys()
    $output &= _ArrayToString($atmp) & @CRLF
    $oDict.removeall()
Next

ConsoleWrite($output & @CRLF)

kylomas

Edited by kylomas

Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites
CFLam
Posted (edited)
9 hours ago, Subz said:

Are you reading this data from a file?  It kind of looks like a csv file?  If so can you post an example file?

Yes, from a file in this format

17,1,2,3,4,5,6,7,10,11,12,13,15,16,20,22,24,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
16,1,2,3,4,5,6,7,8,9,10,11,12,16,17,21,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
23,4,5,6,7,8,9,10,12,13,14,15,16,17,18,19,20,23,25,26,28,29,35,36,,,,,,,,,,,,,,,,,,,,,,,,,,
27,8,9,11,12,13,14,15,17,19,20,21,22,23,24,26,27,28,29,30,31,32,34,37,39,40,41,42,,,,,,,,,,,,,,,,,,,,,,
27,14,16,17,18,19,20,22,23,24,25,26,27,28,29,30,31,32,33,35,36,37,38,39,40,42,43,44,,,,,,,,,,,,,,,,,,,,,,
24,20,24,26,27,28,29,30,31,32,33,34,35,36,37,39,40,41,42,43,44,45,46,47,48,,,,,,,,,,,,,,,,,,,,,,,,,
16,33,34,35,36,37,38,39,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
14,35,37,38,39,40,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

17,1,2,3,4,5,6,7,10,11,12,13,15,16,20,22,24,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
16,1,2,3,4,5,6,7,8,9,10,11,12,16,17,21,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
23,4,5,6,7,8,9,10,12,13,14,15,16,17,18,19,20,23,25,26,28,29,35,36,,,,,,,,,,,,,,,,,,,,,,,,,,
27,8,9,11,12,13,14,15,17,19,20,21,22,23,24,26,27,28,29,30,31,32,34,37,39,40,41,42,,,,,,,,,,,,,,,,,,,,,,
27,14,16,17,18,19,20,22,23,24,25,26,27,28,29,30,31,32,33,35,36,37,38,39,40,42,43,44,,,,,,,,,,,,,,,,,,,,,,
24,20,24,26,27,28,29,30,31,32,33,34,35,36,37,39,40,41,42,43,44,45,46,47,48,,,,,,,,,,,,,,,,,,,,,,,,,
16,33,34,35,36,37,38,39,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
14,35,37,38,39,40,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

and yes, I use filereadtoarray to read in the file

Edited by CFLam

Share this post


Link to post
Share on other sites
iamtheky

Then you could use that array like little arrays.  The func might look funny, but you only have to write it once

#include <Array.au3>

Local $aLine[8]
$aLine[0] = "17,1,2,3,4,5,6,7,10,11,12,13,15,16,20,22,24,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
$aLine[1] = "16,1,2,3,4,5,6,7,8,9,10,11,12,16,17,21,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
$aLine[2] = "23,4,5,6,7,8,9,10,12,13,14,15,16,17,18,19,20,23,25,26,28,29,35,36,,,,,,,,,,,,,,,,,,,,,,,,,,"
$aLine[3] = "27,8,8,11,12,13,14,15,17,19,8,21,22,23,24,26,27,28,29,30,31,32,34,37,39,40,41,42,,,,,,,,,,,,,,,,,,,,,,"
$aLine[4] = "27,14,16,17,18,19,20,22,23,24,25,26,27,28,29,30,31,32,33,35,36,37,38,39,40,42,43,44,,,,,,,,,,,,,,,,,,,,,,"
$aLine[5] = "24,20,24,26,27,28,29,30,31,32,33,34,35,36,37,39,40,41,42,43,44,45,46,47,48,,,,,,,,,,,,,,,,,,,,,,,,,"
$aLine[6] = "16,33,34,35,36,37,38,39,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
$aLine[7] = "14,35,37,38,39,40,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"

_ArrayDisplay(_aLinePull($aLine , 7) , 7)
_ArrayDisplay(_aLinePull($aLine , 3) , 3)


Func _aLinePull($aArr , $nLine)

  return  StringSplit(stringtrimright(StringStripWS(_ArrayToString(_ArrayUnique(StringSplit(stringtrimleft($aArr[$nLine] , StringInStr($aArr[$nLine] , ",")) , "," , 2) , 0 , 0 , 0 , 0) , ",") , 2) , 1) , "," , 2)

EndFunc

 


,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites
CFLam
Posted (edited)

This just output the 8 lines again, I need to take 1 number from each line and form a 8 number array A,B,C,D,E,F,G,H where

A is from line 1
B is from line 2
C is from line 3
D is from line 4
E is from line 5
F is from line 6
G is from line 7
H is from line 8

and from my example file, something like this should be generate:

1,1,4,8,14,20,33,35
1,1,4,8,14,20,33,37
...
1,2,4,8,14,20,33,35

and since repeat number is not allowed in the final output, therefore

1,1,4,8,14,20,33,35
1,1,4,8,14,20,33,37

will be filtered out and

1,2,4,8,14,20,33,35

will be  kept

Therefore I form the A,B,C,D,E,F,G,H ($ResLine) like this

for $i=1 to $LineArr[0][0]
  for $j=1 to $LineArr[1][0]
   for $k=1 to $LineArr[2][0]
    for $l=1 to $LineArr[3][0]
     for $m=1 to $LineArr[4][0]
      for $n=1 to $LineArr[5][0]
       for $o=1 to $LineArr[6][0]
        for $p=1 to $LineArr[7][0]
          $ResLine[0][0]=$LineArr[0][$i]
          $ResLine[0][1]=$LineArr[0][$j]
          $ResLine[0][2]=$LineArr[0][$k]
          $ResLine[0][3]=$LineArr[0][$l]
          $ResLine[0][4]=$LineArr[0][$m]
          $ResLine[0][5]=$LineArr[0][$n]
          $ResLine[0][6]=$LineArr[0][$o]
          $ResLine[0][7]=$LineArr[0][$p]

 

 

and I test if $ResLine contain repeated number, if not, output the line

$uArr=_ArrayUnique($ResLine)
       if $uArr[0]=8 then... (I use filewritefromarray)

My problem is the 8 nested for loop is very slow (may be the arrayunique function as well) and it take over 2 hr still not complete the iteration.

Edited by Melba23
Quote removed

Share this post


Link to post
Share on other sites
CFLam
2 hours ago, kylomas said:

Similar to _ArrayUnique...

#include <Array.au3>

Local $aLine[8]
$aLine[0] = "17,1,2,3,4,5,6,7,10,11,12,13,15,16,20,22,24,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
$aLine[1] = "16,1,2,3,4,5,6,7,8,9,10,11,12,16,17,21,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
$aLine[2] = "23,4,5,6,7,8,9,10,12,13,14,15,16,17,18,19,20,23,25,26,28,29,35,36,,,,,,,,,,,,,,,,,,,,,,,,,,"
$aLine[3] = "27,8,9,11,12,13,14,15,17,19,20,21,22,23,24,26,27,28,29,30,31,32,34,37,39,40,41,42,,,,,,,,,,,,,,,,,,,,,,"
$aLine[4] = "27,14,16,17,18,19,20,22,23,24,25,26,27,28,29,30,31,32,33,35,36,37,38,39,40,42,43,44,,,,,,,,,,,,,,,,,,,,,,"
$aLine[5] = "24,20,24,26,27,28,29,30,31,32,33,34,35,36,37,39,40,41,42,43,44,45,46,47,48,,,,,,,,,,,,,,,,,,,,,,,,,"
$aLine[6] = "16,33,34,35,36,37,38,39,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
$aLine[7] = "14,35,37,38,39,40,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"

Local $oDict = ObjCreate("Scripting.Dictionary")
Local $atmp, $output

For $1 = 0 To UBound($aLine) - 1
    $atmp = StringSplit($aLine[$1], ',', 3)
    For $2 = 1 To UBound($atmp) - 1
        If $atmp[$2] = '' Then ContinueLoop
        $oDict.Item($atmp[$2])
    Next
    $atmp = $oDict.keys()
    $output &= _ArrayToString($atmp) & @CRLF
    $oDict.removeall()
Next

ConsoleWrite($output & @CRLF)

kylomas

The oDict object replace the arrayunique function ?

Share this post


Link to post
Share on other sites
iamtheky

just showing an alternative to dictionary to getting everything all organized to play with.

 if you can treat them as independent arrays then you can extract or concatenate whimsically.  Any additional pseudo randomness to the selection or ensuring that you extracted unique selections is just a loop.


,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites
CFLam
Posted (edited)

This is the function I use to generate output and it take over 2 hrs still not complete

func MyArrayGeneration()
 local $LineArr,$uArr
 dim $MyRes[8]

 _FileReadToArray("mylinefile.txt",$LineArr,$FRTA_ENTIRESPLIT , ",")

 $OutName="MyOutPut.txt"
 $OutFile=fileopen($OutName,10)

 $ResuCnt=0
 ;form the gen draw
 for $i=1 to $LineArr[0][0]
  for $j=1 to $LineArr[1][0]
   for $k=1 to $LineArr[2][0]
    for $l=1 to $LineArr[3][0]
     for $m=1 to $LineArr[4][0]
      for $n=1 to $LineArr[5][0]
       for $o=1 to $LineArr[6][0]
        for $p=1 to $LineArr[7][0]
         $MyRes[0]=$LineArr[0][$i]
         $MyRes[1]=$LineArr[1][$j]
         $MyRes[2]=$LineArr[2][$k]
         $MyRes[3]=$LineArr[3][$l]
         $MyRes[4]=$LineArr[4][$m]
         $MyRes[5]=$LineArr[5][$n]
         $MyRes[6]=$LineArr[6][$o]
         $MyRes[7]=$LineArr[7][$p]
         $uArr=_ArrayUnique($MyRes)
         if $uArr[0]=8 then
          _arraysort($MyRes)
          _arraytranspose($MyRes)
          _FileWriteFromArray($OutFile,$MyRes,0,Default,",")
          _arraytranspose($MyRes)
         endif ;$uArr[0]=8
         $uArr=""
        next
       next
      next
     next
    next
   next
  next
 next
 fileclose($OutFile)
endfunc

 

Edited by CFLam

Share this post


Link to post
Share on other sites
iamtheky

please use code tags, its that little icon that looks like <>


,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites
Subz

Sorry misunderstood, what about something like:

#include <Array.au3>

Global $aArray[0][8], $iColumn = 0
Local $aLine[16]
    $aLine[0] = "17,1,2,3,4,5,6,7,10,11,12,13,15,16,20,22,24,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[1] = "16,1,2,3,4,5,6,7,8,9,10,11,12,16,17,21,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[2] = "23,4,5,6,7,8,9,10,12,13,14,15,16,17,18,19,20,23,25,26,28,29,35,36,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[3] = "27,8,9,11,12,13,14,15,17,19,20,21,22,23,24,26,27,28,29,30,31,32,34,37,39,40,41,42,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[4] = "27,14,16,17,18,19,20,22,23,24,25,26,27,28,29,30,31,32,33,35,36,37,38,39,40,42,43,44,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[5] = "24,20,24,26,27,28,29,30,31,32,33,34,35,36,37,39,40,41,42,43,44,45,46,47,48,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[6] = "16,33,34,35,36,37,38,39,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[7] = "14,35,37,38,39,40,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[8] = "17,1,2,3,4,5,6,7,10,11,12,13,15,16,20,22,24,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[9] = "16,1,2,3,4,5,6,7,8,9,10,11,12,16,17,21,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[10] = "23,4,5,6,7,8,9,10,12,13,14,15,16,17,18,19,20,23,25,26,28,29,35,36,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[11] = "27,8,9,11,12,13,14,15,17,19,20,21,22,23,24,26,27,28,29,30,31,32,34,37,39,40,41,42,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[12] = "27,14,16,17,18,19,20,22,23,24,25,26,27,28,29,30,31,32,33,35,36,37,38,39,40,42,43,44,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[13] = "24,20,24,26,27,28,29,30,31,32,33,34,35,36,37,39,40,41,42,43,44,45,46,47,48,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[14] = "16,33,34,35,36,37,38,39,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[15] = "14,35,37,38,39,40,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"

For $i = 0 To UBound($aLine) - 1
    _StringParser($aLine[$i])
Next

_ArrayDisplay($aArray)

Func _StringParser($_sString)
    Local $aString = StringSplit($_sString, ",", 2)
    ;~ Delete the first number
    _ArrayDelete($aString, 0)
    Local $aUnique = _ArrayUnique($aString, 0, 0, 0, 0)
    ;~ Delete the last line since it will be blank
    If $aUnique[UBound($aUnique) - 1] = "" Then _ArrayDelete($aUnique, UBound($aUnique) - 1)
    If $iColumn = 0 Then
        ReDim $aArray[UBound($aArray) + 1][8]
        $aArray[UBound($aArray) - 1][$iColumn] = $aUnique[0]
        $iColumn += 1
        Return
    EndIf
    _ArrayDisplay($aArray)
    For $i = 0 To UBound($aUnique) - 1
        $iUnique = _ArraySearch($aArray, $aUnique[$i])
        If @error Then
            $aArray[UBound($aArray) - 1][$iColumn] = $aUnique[$i]
            $iColumn += 1
            If $iColumn = 8 Then $iColumn = 0
            Return
        EndIf
    Next
EndFunc

 

Share this post


Link to post
Share on other sites
CFLam
Posted (edited)

No... I need to get 1 number from each line, not 8 number from 1 line, below is my function, only that its very slow

func MyArrayGeneration()
 local $LineArr,$uArr
 dim $MyRes[8]

 _FileReadToArray("mylinefile.txt",$LineArr,$FRTA_ENTIRESPLIT , ",")

 $OutName="MyOutPut.txt"
 $OutFile=fileopen($OutName,10)

 $ResuCnt=0
 ;form the gen draw
 for $i=1 to $LineArr[0][0]
  for $j=1 to $LineArr[1][0]
   for $k=1 to $LineArr[2][0]
    for $l=1 to $LineArr[3][0]
     for $m=1 to $LineArr[4][0]
      for $n=1 to $LineArr[5][0]
       for $o=1 to $LineArr[6][0]
        for $p=1 to $LineArr[7][0]
         $MyRes[0]=$LineArr[0][$i]
         $MyRes[1]=$LineArr[1][$j]
         $MyRes[2]=$LineArr[2][$k]
         $MyRes[3]=$LineArr[3][$l]
         $MyRes[4]=$LineArr[4][$m]
         $MyRes[5]=$LineArr[5][$n]
         $MyRes[6]=$LineArr[6][$o]
         $MyRes[7]=$LineArr[7][$p]
         $uArr=_ArrayUnique($MyRes)
         if $uArr[0]=8 then
          _arraysort($MyRes)
          _arraytranspose($MyRes)
          _FileWriteFromArray($OutFile,$MyRes,0,Default,",")
          _arraytranspose($MyRes)
         endif ;$uArr[0]=8
         $uArr=""
        next
       next
      next
     next
    next
   next
  next
 next
 fileclose($OutFile)
endfunc

 

Edited by Melba23
Quote removed

Share this post


Link to post
Share on other sites
jchd
Posted (edited)

It's hardly surprising to have to wait forever trying to perform such task.

In your example, we have the following (run this snippet):

Local $aLine = [ _
    "17,1,2,3,4,5,6,7,10,11,12,13,15,16,20,22,24,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,", _
    "16,1,2,3,4,5,6,7,8,9,10,11,12,16,17,21,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,", _
    "23,4,5,6,7,8,9,10,12,13,14,15,16,17,18,19,20,23,25,26,28,29,35,36,,,,,,,,,,,,,,,,,,,,,,,,,,", _
    "27,8,9,11,12,13,14,15,17,19,20,21,22,23,24,26,27,28,29,30,31,32,34,37,39,40,41,42,,,,,,,,,,,,,,,,,,,,,,", _
    "27,14,16,17,18,19,20,22,23,24,25,26,27,28,29,30,31,32,33,35,36,37,38,39,40,42,43,44,,,,,,,,,,,,,,,,,,,,,,", _
    "24,20,24,26,27,28,29,30,31,32,33,34,35,36,37,39,40,41,42,43,44,45,46,47,48,,,,,,,,,,,,,,,,,,,,,,,,,", _
    "16,33,34,35,36,37,38,39,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,", _
    "14,35,37,38,39,40,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,," _
]

Local $iEntries, $iCount = 1
For $i = 0 To UBound($aLine) - 1
    $iEntries = Int($aLine[$i])
    $iCount *= $iEntries
    ConsoleWrite("Line " & $i & " has " & $iEntries & " entries, " & $iCount & " combinations until now." & @LF)
Next

So your code is going to evaluate 36,705,312,000 (36.7 billion) combinations of 8 numbers, each one having to be scanned for duplicate entries.

First point: it's going to take forever, whatever powerful language and smart coding you use if you stick to the brute force algorithm.
Corollary: 800 times forever is a long wait: store water and food.
Second point: AutoIt arrays are limited to 2^24 (16 777 216) entries, so if you end up having more than this limit valid arrays, they won't fit in one.

Since it looks like (from that sample only) that there are a lot of duplicate values and that values in each line are in sorted increasing order, you could (not surely) benefit from a setpwise approach. Below letters A,B,C,... are placeholders, not your A, B, C; [*] is an unknown, but this value is bounded. For instance, for step 1 (lines 0 & 1) the maximum entries [*] would be 17×16 = 272. For step 2, [*] would be Ubound($A)×23 after $A has been ReDim-ed, and so on.
Form the resulting array $A[*][2] of valid combinations of lines 0 and 1
Form the resulting array $B[*][3] of valid combinations of $A and line 2
Form the resulting array $C[*][4] of valid combinations of $B and line 3
Form the resulting array $D[*][5] of valid combinations of $C and line 4
Form the resulting array $E[*][6] of valid combinations of $D and line 5
Form the resulting array $F[*][7] of valid combinations of $E and line 6
Form the final resulting array $G[*][8] of valid combinations of $F and line 7

Of course you'll have to take care of not overflowing AutoIt arrays max entries.

If I had time I'd sketch up something in SQLite along these lines (yes I'm mad about it).

EDIT: I also question the actual use context.

Edited by jchd
  • Like 1

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites
Subz

It does only take one number from each line, the two lines it returns in the array are from all 16 lines, however originally I had based this on a 1D Array and so can see an issue, this should be correct:

#include <Array.au3>

Global $aArray1D[0], $aArray2D[0][8], $iRow = 0
Local $aLine[16]
    $aLine[0] = "17,1,2,3,4,5,6,7,10,11,12,13,15,16,20,22,24,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[1] = "16,1,2,3,4,5,6,7,8,9,10,11,12,16,17,21,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[2] = "23,4,5,6,7,8,9,10,12,13,14,15,16,17,18,19,20,23,25,26,28,29,35,36,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[3] = "27,8,9,11,12,13,14,15,17,19,20,21,22,23,24,26,27,28,29,30,31,32,34,37,39,40,41,42,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[4] = "27,14,16,17,18,19,20,22,23,24,25,26,27,28,29,30,31,32,33,35,36,37,38,39,40,42,43,44,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[5] = "24,20,24,26,27,28,29,30,31,32,33,34,35,36,37,39,40,41,42,43,44,45,46,47,48,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[6] = "16,33,34,35,36,37,38,39,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[7] = "14,35,37,38,39,40,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[8] = "17,1,2,3,4,5,6,7,10,11,12,13,15,16,20,22,24,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[9] = "16,1,2,3,4,5,6,7,8,9,10,11,12,16,17,21,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[10] = "23,4,5,6,7,8,9,10,12,13,14,15,16,17,18,19,20,23,25,26,28,29,35,36,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[11] = "27,8,9,11,12,13,14,15,17,19,20,21,22,23,24,26,27,28,29,30,31,32,34,37,39,40,41,42,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[12] = "27,14,16,17,18,19,20,22,23,24,25,26,27,28,29,30,31,32,33,35,36,37,38,39,40,42,43,44,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[13] = "24,20,24,26,27,28,29,30,31,32,33,34,35,36,37,39,40,41,42,43,44,45,46,47,48,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[14] = "16,33,34,35,36,37,38,39,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
    $aLine[15] = "14,35,37,38,39,40,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"

For $i = 0 To UBound($aLine) - 1
    _StringParser($aLine[$i])
Next

_ArrayDisplay($aArray2D)

Func _StringParser($_sString)
    Local $aString = StringSplit($_sString, ",", 2)
    ;~ Delete the first number
    _ArrayDelete($aString, 0)
    Local $aUnique = _ArrayUnique($aString, 0, 0, 0, 0)
    ;~ Delete the last line since it will be blank
    If $aUnique[UBound($aUnique) - 1] = "" Then _ArrayDelete($aUnique, UBound($aUnique) - 1)
    If $iRow = 0 Then
        ReDim $aArray1D[1]
        $aArray1D[0] = $aUnique[0]
        $iRow += 1
        Return
    EndIf
    For $i = 0 To UBound($aUnique) - 1
        $iUnique = _ArraySearch($aArray1D, $aUnique[$i])
        If @error Then
            _ArrayAdd($aArray1D, $aUnique[$i])
            $iRow += 1
            If $iRow = 8 Then
                _ArrayAdd($aArray2D, _ArrayToString($aArray1D))
                $iRow = 0
            EndIf
            Return
        EndIf
    Next
EndFunc

 

Share this post


Link to post
Share on other sites
CFLam
Posted (edited)

Just discove that my approach cannot handle duplicate generation i.e.

1,2,4,8,14,20,33,35

1,2,4,8,14,20,33,35

May generate more than 1 time in my code!!!!!

Edited by Melba23
Quote removed

Share this post


Link to post
Share on other sites
Melba23

CFLam,

When you reply, please use the "Reply to this topic" button at the top of the thread or the "Reply to this topic" editor at the bottom rather than the "Quote" button - responders know what they wrote and it just pads the thread unnecessarily.

M23

 


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites
CFLam

To: jchd

Your stepwise approach is right, I try the following:
1) create a 2 number array from first 2 line, I handle duplicate
number in my code so that 1,1 and 3,3 will not appear, I also sort the 2 numbers
so that 1,3 and 3,1 will all become 1,3. Then add this to a tempArray (as string),
and use _ArrayUnique to eliminate duplicate entry.
2) array from 1) combine with line 3, use stringinstr to check if number
in line 3 already include in array from 1), then add to tempArray2
and use _ArrayUnique to remove duplicate entries.
3) apply the above to line 4,5... but I find that program still run forever in this approach!

I then modify a little bit, instead of add to tempArray and use _ArrayUnique,
I use SQLite i.e. add to a single row table in SQLite instead of add to tempArray
the single row is "PRIMARY KEY UNIQUE" and I select the row back to Autoit.
Practically, I use SQLite to replace the _ArrayUnique function, and this eventually
work! It take 1 hr 20 min to handle 1 file (and I have 799 files coming!)

I am now consider using VC++ to implement this part, however, I am still wonder
I should use std::unique in VC++ or SQLite to do the job (don't know which is better)

Regds
LAM Chi-fung

Share this post


Link to post
Share on other sites
kylomas
Posted (edited)

CFLam,

If your intent is to produce a string of numbers by column without duplicates then try this...

#include <Array.au3>

Local $aLine[8]
$aLine[0] = "17,1,2,3,4,5,6,7,10,11,12,13,15,16,20,22,24,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
$aLine[1] = "16,1,2,3,4,5,6,7,8,9,10,11,12,16,17,21,26,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
$aLine[2] = "23,4,5,6,7,8,9,10,12,13,14,15,16,17,18,19,20,23,25,26,28,29,35,36,,,,,,,,,,,,,,,,,,,,,,,,,,"
$aLine[3] = "27,8,9,11,12,13,14,15,17,19,20,21,22,23,24,26,27,28,29,30,31,32,34,37,39,40,41,42,,,,,,,,,,,,,,,,,,,,,,"
$aLine[4] = "27,14,16,17,18,19,20,22,23,24,25,26,27,28,29,30,31,32,33,35,36,37,38,39,40,42,43,44,,,,,,,,,,,,,,,,,,,,,,"
$aLine[5] = "24,20,24,26,27,28,29,30,31,32,33,34,35,36,37,39,40,41,42,43,44,45,46,47,48,,,,,,,,,,,,,,,,,,,,,,,,,"
$aLine[6] = "16,33,34,35,36,37,38,39,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"
$aLine[7] = "14,35,37,38,39,40,41,42,43,44,45,46,47,48,49,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"

; fimd max # of columns

Local $imaxcol = 0
For $1 = 0 To UBound($aLine) - 1
    StringReplace($aLine[$1], ',', ',')
    $imaxcol = ($imaxcol > @extended ? $imaxcol : @extended)
Next

Local $a2D[UBound($aLine)][$imaxcol + 1], $atmp

; populate working array

For $1 = 0 To UBound($aLine) - 1
    $atmp = StringSplit($aLine[$1], ',', 3)
    For $2 = 0 To UBound($atmp) - 1
        ;ConsoleWrite($1 & ' - ' & $2 & @CRLF)
        $a2D[$1][$2] = $atmp[$2]
    Next
Next

; process each column producing a string of numbers without duplicates, comma delimited

Local $oDict = ObjCreate("Scripting.Dictionary")
Local $atmp, $output

_ArrayDisplay($a2D)

For $1 = 1 To UBound($a2D, 2) - 1
    For $2 = 0 To UBound($a2D) - 1
        If $a2D[$2][$1] = '' Then ContinueLoop
        $oDict.Item($a2D[$2][$1])
    Next
    $atmp = $oDict.keys()
    If UBound($atmp) > 0 Then $output &= _ArrayToString($atmp, ',') & @CRLF
    $oDict.removeall()
Next

ConsoleWrite($output & @CRLF)

 if not, I have no idea what you are trying to do...

kylomas

Edited by kylomas
correct final string delimiter

Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×