Compare between arrays ( More than two )

[ahmeddzcom]

Hello
I use this code , but This take a long time.

#include <File.au3>

    Local $P1 = @ScriptDir&"\P1.txt"
    Local $P2 = @ScriptDir&"\P2.txt"
    Local $Cons1 = @ScriptDir&"\Cons1.txt"
    Local $Cons2 = @ScriptDir&"\Cons2.txt"
    Local $Cons3 = @ScriptDir&"\Cons3.txt"
        For $y = 1 to _FileCountLines($P1)
            Local $sString1 = FileReadLine($P1, $y)
            Local $sString2 = FileReadLine($P2, $y)
            For $i = 1 to _FileCountLines($Cons1)
                Local $sStringC1 = FileReadLine($Cons1, $i)
                Local $sStringC2 = FileReadLine($Cons2, $i)
                If $sString1&$sString2 = $sStringC1&$sStringC2 Then
                    FileWrite("Cons_NoTax1.txt",$sString1&@CRLF)
                    FileWrite("Cons_NoTax2.txt",$sString2&@CRLF)
                    FileWrite("Cons_NoTax3.txt",FileReadLine($Cons3, $i)&@CRLF)
                    ExitLoop
                EndIf
            Next
        Next

I tried several times and it didn't work
i need if any other fast solution please 
thanks.

P1.txt P2.txt Cons1.txt Cons2.txt Cons3.txt

[jchd]

37 minutes ago, ahmeddzcom said:

I tried several times and it didn't work

Not very surprising: P1 and P2 don't have the same number of lines, for instance.

Please explain in plain english what you want to achieve exactly. As it is your post + code + files doesn't make any sense.

[junkew]

In general

• sort the arrays
• deduplicate the arrays

before you start comparing I also would do the concatenation

But as said by jchd:

• Describe your goal is it a compare or you want to know if certain value exist into another array
• Give more background on what you are comparing
• Whats the relation between the files
  If I read your script I see 2 files as input with same no of lines (138)
  compared to 2 other files with same no of lines (5940)
  and you want to write a result from the 3rd file which also has 5940 lines
  So probably your intent is that if line 1000 matches that you write line 1000 of the 3rd file, thats definitely not what your code is doing
• Size of the files

I think your algorithm should be something like below (put the biggest array in the ourer loop)

1. Read the input files P1 and P2 and concatenate them to 1 new array size 138          input
2. Read the  input files C1 and C2 and concatenate them to 1 new array size 5940       compare
3. Read the input file c3 into array outputdata size 5940
4.  for i = 1 to ubound compare array (5940)
            for j = 1 to ubound (input) array 138
                   if match consolewrite(outputdata[$i]

The innerloop you can potentially rewrite to stringinstr logic by making your input array 138 a huge string

Make use of https://www.autoitscript.com/autoit3/docs/libfunctions/_FileReadToArray.htm

 

 

Edited March 4, 2022 by junkew

[ahmeddzcom]

10 hours ago, jchd said:

Not very surprising: P1 and P2 don't have the same number of lines, for instance.

@jchd : P1 and P2 have the same number of lines

Edited March 4, 2022 by ahmeddzcom

[jchd]

Maybe in your dreams.

[junkew]

@jchd  p1 and p2 have 138 lines the C files have 5940

@ahmeddzcom please elaborate on your problem

 

[ahmeddzcom]

Hi @junkew

p1 & p2 the products i have , without price
i need get price from 5940 products

I used :
_ArraySearch 
_ArrayDelete
_ArrayUnique
FileOpen & FileWrite
but i can't
thanks
 

[junkew]

So you use filereadtoarray. You can improve all parts. 

I just improved 1 small thing but you can speed up many more

#include <File.au3>

Local $P1 = @ScriptDir&"\P1.txt"
Local $P2 = @ScriptDir&"\P2.txt"

Local $Cons1 = @ScriptDir&"\Cons1.txt"
Local $Cons2 = @ScriptDir&"\Cons2.txt"
Local $Cons3 = @ScriptDir&"\Cons3.txt"

local $aRetArray
_FileReadToArray($Cons3, $aRetArray)


For $y = 1 to _FileCountLines($P1)
            Local $sString1 = FileReadLine($P1, $y)
            Local $sString2 = FileReadLine($P2, $y)
            For $i = 1 to _FileCountLines($Cons1)
                Local $sStringC1 = FileReadLine($Cons1, $i)
                Local $sStringC2 = FileReadLine($Cons2, $i)
                If $sString1&$sString2 = $sStringC1&$sStringC2 Then
                    FileWrite("Cons_NoTax1.txt",$sString1&@CRLF)
                    FileWrite("Cons_NoTax2.txt",$sString2&@CRLF)
                    ;~FileWrite("Cons_NoTax3.txt",FileReadLine($Cons3, $i)&@CRLF)
                    FileWrite("Cons_NoTax3.txt",$aRetArray[$i] & @CRLF)
                    ExitLoop
                EndIf
            Next
        Next

 

 

[ahmeddzcom]

14 minutes ago, junkew said:

I just improved 1 small thing but you can speed up many more

thank you so much @junkew , your code time taken = 226.77 secs , my code time taken = 227.31 secs 😅

[jchd]

37 minutes ago, ahmeddzcom said:

p1 & p2 the products i have , without price
i need get price from 5940 products

p1 is only 138 lines. How can you expect to get prices for more products?

[pixelsearch]

What I understand is the product code is a combination of P1 & P2 with a "database" corresponding to Cons1 & Cons2 & Cons3

For example, if we look at line #2 in P1 and line #2 in P2, we got "30006" & "30E", that's the key, the product code to search in the database.

Now this key is found at line #462 of Cons1 and line #462 of Cons2 : "30006" & "30E"
When this match is found, then retrieve line #462 in Cons3 to get the prize of this product : 11547.00

P1 & P2 got same numbers of lines (138), let's say these are the products actually available in store.

Cons1 & Cons2 & Cons3 have (of course !) the same number of lines (5940) and could correspond to the list of all products available (but they're not all in store actually)

Please note that not using handles for each opened text file makes it a very long process, as stated in the help file (topic FileReadLine)

I feel that this whole process could be improved, sure someone will find a solution for that.

Edited March 4, 2022 by pixelsearch

[ahmeddzcom]

3 minutes ago, pixelsearch said:

What I understand is the product code is a combination of P1 & P2 with a "database" corresponding to Cons1 & Cons2 & Cons3

For example, if we look at line #2 in P1 and line #2 in P2, we got "30006" & "30E", that's the key, the product code to search in the database.

Now this key is found at line #462 of Cons1 and line #462 of Cons2 : "30006" & "30E"
When this match is found, then retrieve line #462 in Cons3 to get the prize of this product : 11547.00

P1 & P2 got same numbers of lines (138), let's say these are the products actually available in store.

Cons1 & Cons2 & Cons3 have (of course !) the same number of lines (5940) and could correspond to the list of all products available (but they're not all in store actually)

Please note that not using handles for each opened text file makes it a very long process, as stated in the help file (topic FileReadLine)

Exactly  😄

[junkew]

So suggestion to use the filehandles

https://www.autoitscript.com/autoit3/docs/functions/FileOpen.htm

#include <File.au3>

Local $P1 = fileopen(@ScriptDir&"\P1.txt")
Local $P2 = fileopen(@ScriptDir&"\P2.txt")

Local $Cons1 = fileopen(@ScriptDir&"\Cons1.txt")
Local $Cons2 = fileopen(@ScriptDir&"\Cons2.txt")
Local $Cons3 = fileopen(@ScriptDir&"\Cons3.txt")

local $aRetArray
_FileReadToArray($Cons3, $aRetArray)

For $y = 1 to _FileCountLines($P1)
            Local $sString1 = FileReadLine($P1, $y)
            Local $sString2 = FileReadLine($P2, $y)
            For $i = 1 to _FileCountLines($Cons1)
                Local $sStringC1 = FileReadLine($Cons1, $i)
                Local $sStringC2 = FileReadLine($Cons2, $i)
                If $sString1&$sString2 = $sStringC1&$sStringC2 Then
                    FileWrite("Cons_NoTax1.txt",$sString1&@CRLF)
                    FileWrite("Cons_NoTax2.txt",$sString2&@CRLF)
                    ;~FileWrite("Cons_NoTax3.txt",FileReadLine($Cons3, $i)&@CRLF)
                    FileWrite("Cons_NoTax3.txt",$aRetArray[$i] & @CRLF)
                    ExitLoop
                EndIf
            Next
        Next

Powershell

clear-host
#Folder with the files
$scriptdir='C:\Users\ewildsch\Downloads'
get-date

#Read the stuff in arrays
$p1=get-content "$ScriptDir\P1.txt"
$p2=get-content "$ScriptDir\P2.txt"

$cons1=get-content "$ScriptDir\Cons1.txt"
$cons2=get-content "$ScriptDir\Cons2.txt"
$cons3=get-content "$ScriptDir\Cons3.txt"

#New array with productkey
$P3 = for ( $i = 0; $i -lt $p1.count; $i++)
{
    [PSCustomObject]@{
        P  = $P1[$i] + $P2[$i]
        P1 = $P1[$i] 
        P2 = $P2[$i]
    }
}

#New array with all the shopitems
$C4 = for ( $i = 0; $i -lt $cons1.count; $i++)
{
    [PSCustomObject]@{
        P  = $cons1[$i] + $cons2[$i]
        C1 = $cons1[$i] 
        C2 = $cons2[$i]
        C3 = $cons3[$i]
    }
}

write-host "lets shop" 
get-date

#Get the result in an array
$result=foreach ($P in $P3) {
    foreach($C in $c4) {
       if ($C.P -eq $p.P) {
            $C
            break
       }    
    }     
}

#Show the result
$result | format-table

#Uncomment to write to file
#$result.c1 | out-file "$ScriptDir\Cons_NoTax1.txt"
#$result.c2 | out-file "$ScriptDir\Cons_NoTax2.txt"
#$result.c3 | out-file "$ScriptDir\Cons_NoTax3.txt"
get-date

 

Edited March 4, 2022 by junkew
bonus in powershell

[pixelsearch]

@ahmeddzcom: the following code should do it fine.
A message will appear that product 90332 - 362 exist in P1 - P2 but not in Cons1 - Cons2
What exists in Cons1 - Cons2 is 90332 - 360
Good luck

expandcollapse popup

#include <Array.au3>
#include <FileConstants.au3>

Opt("MustDeclareVars", 1)

;=========================
Local $hTimer = TimerInit()

Local $aP1 = FileReadToArray(@ScriptDir&"\P1.txt")
Local $aP2 = FileReadToArray(@ScriptDir&"\P2.txt")
Local $iRows_P = UBound($aP1)

Local $aCons1 = FileReadToArray(@ScriptDir&"\Cons1.txt")
Local $aCons2 = FileReadToArray(@ScriptDir&"\Cons2.txt")
Local $aCons3 = FileReadToArray(@ScriptDir&"\Cons3.txt")
Local $iRows_Cons = UBound($aCons1)

Local $aCons[$iRows_Cons][2]

For $i = 0 To $iRows_Cons - 1
    $aCons[$i][0] = $aCons1[$i] & " - " & $aCons2[$i]
    $aCons[$i][1] = $aCons3[$i]
Next

ConsoleWrite("Generating $aCons 2D array = " & Int(TimerDiff($htimer)) & " ms" & @crlf)
_ArrayDisplay($aCons, "$aCons UNsorted")

;=========================
Local $hTimer = TimerInit()
_ArraySort($aCons) ; mandatory sort for _ArrayBinarySearch below

ConsoleWrite("Sorting $aCons = " & Int(TimerDiff($htimer)) & " ms" & @crlf)
_ArrayDisplay($aCons, "$aCons Sorted")

;=========================
Local $hTimer = TimerInit()

Local $hCons_NoTax1 = FileOpen(@ScriptDir&"\Cons_NoTax1.txt", $FO_APPEND)
Local $hCons_NoTax2 = FileOpen(@ScriptDir&"\Cons_NoTax2.txt", $FO_APPEND)
Local $hCons_NoTax3 = FileOpen(@ScriptDir&"\Cons_NoTax3.txt", $FO_APPEND)
Local $iIndex

For $i = 0 to $iRows_P - 1
    $iIndex = _ArrayBinarySearch($aCons, $aP1[$i] & " - " & $aP2[$i])
    If $iIndex >= 0 Then
        FileWriteLine($hCons_NoTax1, $aP1[$i])
        FileWriteLine($hCons_NoTax2, $aP2[$i])
        FileWriteLine($hCons_NoTax3, $aCons[$iIndex][1])
    Else
        ConsoleWrite($aP1[$i] & " - " & $aP2[$i] & " not found in $aCons" & @crlf)
    EndIf
Next

FileClose($hCons_NoTax1)
FileClose($hCons_NoTax2)
FileClose($hCons_NoTax3)

ConsoleWrite("Writing Cons_NoTax1-2-3 = " & Int(TimerDiff($htimer)) & " ms" & @crlf)

 

[ahmeddzcom]

2 hours ago, junkew said:

Powershell

 

it's really nice work

Thanx mr @junkew

[ahmeddzcom]

35 minutes ago, pixelsearch said:

@ahmeddzcom: the following code should do it fine.
A message will appear that product 90332 - 362 exist in P1 - P2 but not in Cons1 - Cons2
What exists in Cons1 - Cons2 is 90332 - 360
Good luck

expandcollapse popup

#include <Array.au3>
#include <FileConstants.au3>

Opt("MustDeclareVars", 1)

;=========================
Local $hTimer = TimerInit()

Local $aP1 = FileReadToArray(@ScriptDir&"\P1.txt")
Local $aP2 = FileReadToArray(@ScriptDir&"\P2.txt")
Local $iRows_P = UBound($aP1)

Local $aCons1 = FileReadToArray(@ScriptDir&"\Cons1.txt")
Local $aCons2 = FileReadToArray(@ScriptDir&"\Cons2.txt")
Local $aCons3 = FileReadToArray(@ScriptDir&"\Cons3.txt")
Local $iRows_Cons = UBound($aCons1)

Local $aCons[$iRows_Cons][2]

For $i = 0 To $iRows_Cons - 1
    $aCons[$i][0] = $aCons1[$i] & " - " & $aCons2[$i]
    $aCons[$i][1] = $aCons3[$i]
Next

ConsoleWrite("Generating $aCons 2D array = " & Int(TimerDiff($htimer)) & " ms" & @crlf)
_ArrayDisplay($aCons, "$aCons UNsorted")

;=========================
Local $hTimer = TimerInit()
_ArraySort($aCons) ; mandatory sort for _ArrayBinarySearch below

ConsoleWrite("Sorting $aCons = " & Int(TimerDiff($htimer)) & " ms" & @crlf)
_ArrayDisplay($aCons, "$aCons Sorted")

;=========================
Local $hTimer = TimerInit()

Local $hCons_NoTax1 = FileOpen(@ScriptDir&"\Cons_NoTax1.txt", $FO_APPEND)
Local $hCons_NoTax2 = FileOpen(@ScriptDir&"\Cons_NoTax2.txt", $FO_APPEND)
Local $hCons_NoTax3 = FileOpen(@ScriptDir&"\Cons_NoTax3.txt", $FO_APPEND)
Local $iIndex

For $i = 0 to $iRows_P - 1
    $iIndex = _ArrayBinarySearch($aCons, $aP1[$i] & " - " & $aP2[$i])
    If $iIndex >= 0 Then
        FileWriteLine($hCons_NoTax1, $aP1[$i])
        FileWriteLine($hCons_NoTax2, $aP2[$i])
        FileWriteLine($hCons_NoTax3, $aCons[$iIndex][1])
    Else
        ConsoleWrite($aP1[$i] & " - " & $aP2[$i] & " not found in $aCons" & @crlf)
    EndIf
Next

FileClose($hCons_NoTax1)
FileClose($hCons_NoTax2)
FileClose($hCons_NoTax3)

ConsoleWrite("Writing Cons_NoTax1-2-3 = " & Int(TimerDiff($htimer)) & " ms" & @crlf)

 

finally 😅

exactly what I was want

thanx mr @pixelsearch