Sign in to follow this  
Followers 0
anhchangtk

Problem find line duplicate from log

12 posts in this topic

I'm want create project find line duplicate, last save with file name defaul, the project get info form 1 text file, last project will find line duplicate from log file save all line duplicate and line had tested

Me had use code

#include <Array.au3>
#include<File.au3>
#include <ButtonConstants.au3>
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <StaticConstants.au3>
#include <WindowsConstants.au3>

$filename="testduplication.txt"
$file = FileOpen($filename)
$log = "Duplication.log"
While 1
$line = FileReadLine($file,1)
FileWirteLine($log,$line)
WEnd
Fileclose($file)
FileClose($log)

Me can make project write line from testduplication.txt to Duplication.log :unsure:. But i'm want write line done, the app will find duplication in Duplication.log :D

Can you help me ? :D

Thanks you very much :P

Share this post


Link to post
Share on other sites



Let's say testduplication.txt is:

Dog

Cat

Dog

You want Duplication.log to be:

Dog

Cat

Or

Dog?

Share this post


Link to post
Share on other sites

testduplication.txt will save

Dog

Cat

Duplication.log will save

Dog

Cat

Dog

:P Please hlep me :unsure: Thanks you :D

Share this post


Link to post
Share on other sites

Like this?

#Include <Array.au3>
#Include <File.au3>
$file1 = FileOpenDialog("Input", @scriptdir, "Text Documents (*.txt)", 1)
$file2 = FileOpenDialog("Output", @scriptdir, "Text Documents (*.txt)", 1)
Dim $Darray
Dim $Darray2[1]
_FileReadToArray($file1,$Darray)
_ArrayDelete($Darray,0)
$Before_uni = ubound($Darray)
$Darray = _ArrayUnique($Darray)
_ArrayDelete($Darray,0)
$After_uni = ubound($Darray)
_FileWriteFromArray($file2,$Darray)
$Results = $Before_uni - $After_uni
$s = 's'
If $Results = 1 then $s = ''
msgbox(0,"Deleted",$Results & ' Duplicate line'&$s&' deleted.')

Share this post


Link to post
Share on other sites

:P me run the script, been report error :unsure:

$Darray = _ArrayUnique($Darray)
$Darray = ^ ERROR

Share this post


Link to post
Share on other sites

Did you put line "#Include <Array.au3>"? If you did tell me what error you get when running:

#Include <Array.au3>
#Include <File.au3>
$file1 = FileOpenDialog("Input", @scriptdir, "Text Documents (*.txt)", 1)
$file2 = FileOpenDialog("Output", @scriptdir, "Text Documents (*.txt)", 1)
Dim $Darray
Dim $Darray2[1]
_FileReadToArray($file1,$Darray)
_ArrayDelete($Darray,0)
$Before_uni = ubound($Darray)
$Darray = _ArrayUnique($Darray)
msgbox(0,"Error",'ArrayUnique @Error = ' & @Error)
_ArrayDelete($Darray,0)
$After_uni = ubound($Darray)
_FileWriteFromArray($file2,$Darray)
$Results = $Before_uni - $After_uni
$s = 's'
If $Results = 1 then $s = ''
msgbox(0,"Deleted",$Results & ' Duplicate line'&$s&' deleted.')

Share this post


Link to post
Share on other sites

:P now me had you array + for but script not run :unsure:

#Include <Array.au3>
#Include <File.au3>

$filename = "test.txt"
$filelog = "log.log"
$filedone = "done.txt"
$gd = "|"
$vt = "1"
$log = fileopen($filelog,0)
$file = FileOpen($filename, 0)
$done = fileopen($filedone, 1)
If $file = -1 Then
    msgbox(16,"Error","Unable to open file.")
Exit
EndIf

For 1 to $line
$linef = FileReadLine($file)
$linel = filereadline($log)
$arrayf = StringSplit($linef,$gd)
$array1 = $arrayf[$vt]
$arrayl = StringSplit($linel,$gd)
$array2 = $arrayl[$vt]
if $array1 == $array2 Then
filewriteline($log,$array1 & ">> Had saved database, check log")
Else
FileWriteLine($done,$array1)
EndIf
Next
FileClose($file)
FileClose($log)
FileClose($done)

This is terms test.txt

| Dog | 1 | 22.000 | 19.03.2009 |
| Cat | 1 | 32.000 | 19.03.2009 |
| Pig | 1 | 50.000 | 19.03.2009 |

This is terms log.log

| Dog | 1 | 22.000 | 19.03.2009 |
| Cat | 1 | 32.000 | 19.3.2009 |
| Dog | 1 | 22.000 | 19.03.2009 |
| Cat | 1 | 32.000 | 19.3.2009 |
| Dog | 1 | 22.000 | 19.03.2009 |
| Cat | 1 | 32.000 | 19.3.2009 |
| Dog | 1 | 22.000 | 19.03.2009 |
| Cat | 1 | 32.000 | 19.3.2009 |
| Dog | 1 | 22.000 | 19.03.2009 |
| Cat | 1 | 32.000 | 19.3.2009 |
| Dog | 1 | 22.000 | 19.03.2009 |
| Cat | 1 | 32.000 | 19.3.2009 |
| Dog | 1 | 22.000 | 19.03.2009 |
| Cat | 1 | 32.000 | 19.3.2009 |

If the script run it will save to file done.txt

| Pig | 1 | 50.000 | 19.03.2009 |

Please help me :D

Share this post


Link to post
Share on other sites

#include <Array.au3>
#include <File.au3>

Dim $sText = '| Dog | 1 | 22.000 | 19.03.2009 |' & @CRLF & _
             '| Cat | 1 | 32.000 | 19.3.2009 |' & @CRLF & _
             '| Dog | 1 | 22.000 | 19.03.2009 |' & @CRLF & _
             '| Cat | 1 | 32.000 | 19.3.2009 |' & @CRLF & _
             '| Dog | 1 | 22.000 | 19.03.2009 |' & @CRLF & _
             '| Cat | 1 | 32.000 | 19.3.2009 |' & @CRLF & _
             '| Dog | 1 | 22.000 | 19.03.2009 |' & @CRLF & _
             '| Cat | 1 | 32.000 | 19.3.2009 |' & @CRLF & _
             '| Dog | 1 | 22.000 | 19.03.2009 |' & @CRLF & _
             '| Cat | 1 | 32.000 | 19.3.2009 |' & @CRLF & _
             '| Dog | 1 | 22.000 | 19.03.2009 |' & @CRLF & _
             '| Cat | 1 | 32.000 | 19.3.2009 |' & @CRLF & _
             '| Dog | 1 | 22.000 | 19.03.2009 |' & @CRLF & _
             '| Cat | 1 | 32.000 | 19.3.2009 |'
             
FileWrite(@ScriptDir & '\testduplications.txt', $sText) ; ... just to demonstrate.

Dim $sFileText = FileRead(@ScriptDir & '\testduplications.txt')

Dim $aLines = StringSplit($sFileText, @CRLF, 3)
_ArrayDisplay($aLines, Default, -1, Default, Chr(0xFF))

Dim $i = 0, $j = 1
Dim $sTmp

While $i < UBound($aLines)-1
    $j = _ArraySearch($aLines, $aLines[$i], $j)
    
    If $j <> -1 Then
        _ArrayDelete($aLines, $j)
        ;$j =+ 1
    Else
        $i += 1
        $j = $i + 1
    EndIf
WEnd

_ArrayDisplay($aLines, Default, -1, Default, Chr(0xFF))
_FileWriteFromArray(@ScriptDir & '\Duplicates.log', $aLines)

What is the purpose of the terms file? You search for duplicates of lines on the current text aren't you?

Share this post


Link to post
Share on other sites

Result display is correct but i'm want export to file *.txt :P

And the script will find line dup in the testduplications.txt and Duplicates.log

Your script only check dup in the testduplications.txt and save Duplicates.log but check on 2 file :unsure:

Share this post


Link to post
Share on other sites

#11 ·  Posted (edited)

@foster74: Thanks you, but your script not run good, sript been error :D me had add #include <Array.au3> but not run :(

@Authenticity: Can you make to me complete script ??? Complete by code you and this code me :P or you can make a code complete by you give me :D:unsure:

#include <Array.au3>
#include<File.au3>
#include <ButtonConstants.au3>
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <StaticConstants.au3>
#include <WindowsConstants.au3>

$filename="testduplication.txt"
$file = FileOpen($filename)
$log = "Duplication.log"
While 1
$line = FileReadLine($file,1)
FileWirteLine($log,$line)
WEnd
Fileclose($file)
FileClose($log)

Me had apply _arraysearch to my script but the script run done not save to file ;)

Please help me ;)

Thanks you :D

Edited by anhchangtk

Share this post


Link to post
Share on other sites

How villain would I be if I steal the learning treasure from you? ;]

Anyway, try the loop as an example, it may not suite your needs but it's the way to go, or in much clever way which I don't know ;[.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0