Sign in to follow this  
Followers 0
Sercankd

Remove duplicated lines in a text file

9 posts in this topic

i am newbie in autoit so i cant create good programs

i need remove lines they are duplicated

i have a list that contains some important accounts and there is lines with same username but different password i want remove only other lines after first search result in text file

example

first search result is real one and i dont want delete it

asdasd31 - 505052 - asd@asd.com

there is line with same username but password different and this one should removed :)

asdasd31 - 346363 - sercan@sercan.com

thanks in advance

Share this post


Link to post
Share on other sites



#3 ·  Posted (edited)

i tried one but it deletes only lines with same email :S

#Region;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_Icon=icon.ico
#AutoIt3Wrapper_Res_Comment=Sercankd ECSRO
#AutoIt3Wrapper_Res_Description=Kayıt Programı
#AutoIt3Wrapper_Res_Fileversion=1
#EndRegion;**** Directives created by AutoIt3Wrapper_GUI ****
#include <IE.au3>
#include <file.au3>
Global $file
Global $toplam
Opt("TrayMenuMode", 1)
TraySetIcon("Shell32.dll", 13)
HotKeySet("{END}", "Kapat")
Func Kapat()
    Exit 0;
EndFunc  ;==>Kapat
If FileExists("kayitlar.txt") Then
While 1 = 1
    $file = FileOpen("kayitlar.txt", 1);kayit dosyası
    $email = "asd@asd.com";email
    $password = Random(100000, 999999, 1);password 100000 ile 999999 arası rastgele
    $username = "Sercan" & Random(10, 9999, 1);default sercan + rastgele sayı 10 ile 9999 arası
    FileWrite($file, $username & " - " & $password & " - " & $email & @CRLF);dosyaya yaz kayıt tut
    $sayi = Random(100, 999);rastgele fonksiyonu
    $oIE = _IECreate("http://221.195.40.141:1026/fembria/",0,0,0,1)
    _IELoadWait($oIE)
    $oForm = _IEFormGetObjByName($oIE, "register")
    $Usr = _IEGetObjByName($oIE, "account")
    _IEFormElementSetValue($Usr, $username)
    $Usr1 = _IEGetObjByName($oIE, "myemail")
    _IEFormElementSetValue($Usr1, $email)
    $Usr2 = _IEGetObjByName($oIE, "username")
    _IEFormElementSetValue($Usr2, $username)
    $Usr3 = _IEGetObjByName($oIE, "pasword")
    _IEFormElementSetValue($Usr3, $password)
    $Usr4 = _IEGetObjByName($oIE, "pssword")
    _IEFormElementSetValue($Usr4, $password)
    $Btn = _IEGetObjByName($oIE, "submit")
    _IEAction($Btn, "click")
    $toplam = _FileCountLines("kayitlar.txt")
    TrayTip("Toplam", "Toplam Üyelik Sayısı: " & $toplam , 2, 1)
    Sleep(8000)
    _IEQuit($oIE)
    FileClose($file)
WEnd
Else
        MsgBox(48,"Hata!", "Kayıt dosyası bulunamadı.Program kapatılıyor.")
EndIf

this is the code i wrote for output registered accounts and

and this is the results

Sercan674 - 822856 - asd@asd.com

Sercan281 - 530128 - asd@asd.com

Sercan494 - 626108 - asd@asd.com

Sercan674 - 505645 - asd@asd.com

you can see there is 2x Sercan674 and i want remove all sercan674s except first one

Edited by Sercankd

Share this post


Link to post
Share on other sites

any help?

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

I was already thinking about to help... but looking at the provided source... I doubt you have a respectable reason for any help...

Edit: Typo

Edited by KaFu

Share this post


Link to post
Share on other sites

thats not a malicious code.its just registering some important accounts for me before registration closes for game server forever

Share this post


Link to post
Share on other sites

#8 ·  Posted (edited)

thats not a malicious code.its just registering some important accounts for me before registration closes for game server forever

Hmmm, okay, cheating @ games is not that malicious... not nice but not too bad either :)...

Threw something together from help-file, look at it and adopt something like it in your script...

$file = FileOpen("input.txt", 0)
$output_text = ""
; Check if file opened for reading OK
If $file = -1 Then
    MsgBox(0, "Error", "Unable to open file.")
    Exit
EndIf
; Read in lines of text until the EOF is reached
While 1
    $line = FileReadLine($file)
    If @error = -1 Then ExitLoop
    
    if Not StringInStr($output_text,stringleft($line,StringInStr($line," - "))) then
        $output_text &= $line & @CRLF
    EndIf
Wend
FileClose($file)

$file = FileOpen("output.txt", 1)
; Check if file opened for writing OK
If $file = -1 Then
    MsgBox(0, "Error", "Unable to open file.")
    Exit
EndIf
FileWrite($file, $output_text)
FileClose($file)
Edited by KaFu

Share this post


Link to post
Share on other sites

#9 ·  Posted (edited)

a bit confused, but works.

#include <array.au3>
#include <file.au3>
#include <guiconstants.au3>
#include <string.au3>
;~ #include <array.au3>

;---------------------------------------- ask user for txt file
$message = "Choose text file to make your Wordlist"
dim $chars
$var = FileOpenDialog($message, @DesktopCommonDir & "\", " Text file (*.txt)", 1)

If @error Then
    MsgBox(4096,"","No File chosen")
Else
;-------------------------------------------------------- open File
    $filename_user = StringReplace($var, "|", @CRLF)
    $file = FileOpen($filename_user, 0)

;-------------------------------------------------------- Check if file opened for reading OK
    If $file = -1 Then
        MsgBox(0, "Error", "Unable to open file.")
        Exit
    elseif $file <> -1 Then
        $chars = FileRead($file)
;-------------------------------------------------------- create array
        ToolTip("Create array")
        $array = StringSplit($chars, @CRLF)

;-------------------------------------------------------- order in array - delete dupes
        ToolTip("delete dupes - sorting...")
        _ArraySort($Array, 0, 0, 0, 0)
        
        ToolTip("delete dupes")
        $array = _ArrayUnique($array)
        
        ToolTip("delete dupes - sorting...")
        _ArraySort($Array, 0, 0, 0, 0)

;-------------------------------------------------------- write wordlist
        ToolTip("write wordlist")
        FileDelete(@ScriptDir & "\test_sorted.txt")
        $file_sorted = FileOpen(@ScriptDir & "\test_sorted.txt", 2)
        
        for $i = 1 to UBound($array) - 1
            FileWrite($file_sorted, $array[$i] & @CRLF)
        next
        
        FileClose($file)
        FileClose($file_sorted)
;-------------------------------------------------------- clean blank lines
    ToolTip("clean blank lines")
        $file = FileOpen(@ScriptDir & "\test_sorted.txt", 0)

        FileDelete(@ScriptDir & "\Wordlist_from_text.txt")
        $file_sorted = FileOpen(@ScriptDir & "\Wordlist_from_text.txt", 2)

; Read in lines of text until the EOF is reached
        While 1
            $line = FileReadLine($file)
            If @error = -1 Then ExitLoop
                
            $line1 = StringStripWS($line,1)
            $line1 = StringStripCR($line1)

            if $line1 <> "" Then
                FileWrite($file_sorted, $line1 & @CRLF)
            EndIf
        Wend


    EndIf
endif

ToolTip("")
FileClose($file)
FileClose($file_sorted)
FileDelete(@ScriptDir & "\test_sorted.txt")


Func _ArrayUnique($aArray, $iDimension = 1, $iBase = 0, $iCase = 0, $vDelim = "|")
Local $iUboundDim
;$aArray used to be ByRef, but litlmike altered it to allow for the choosing of 1 Array Dimension, without altering the original array
If $vDelim = "|" Then $vDelim = Chr(01); by SmOke_N, modified by litlmike
If Not IsArray($aArray) Then Return SetError(1, 0, 0);Check to see if it is valid array

;Checks that the given Dimension is Valid
    If Not $iDimension > 0 Then
        Return SetError(3, 0, 0);Check to see if it is valid array dimension, Should be greater than 0
    Else
;If Dimension Exists, then get the number of "Rows"
    $iUboundDim = UBound($aArray, 1);Get Number of "Rows"
    If @error Then Return SetError(3, 0, 0);2 = Array dimension is invalid.

;If $iDimension Exists, And the number of "Rows" is Valid:
        If $iDimension > 1 Then;Makes sure the Array dimension desired is more than 1-dimensional
            Local $aArrayTmp[1];Declare blank array, which will hold the dimension declared by user
            For $i = 0 To $iUboundDim - 1;Loop through "Rows"
            _ArrayAdd($aArrayTmp, $aArray[$i][$iDimension - 1]);$iDimension-1 to match Dimension
            Next
            _ArrayDelete($aArrayTmp, 0);Get rid of 1st-element which is blank
        Else;Makes sure the Array dimension desired is 1-dimensional
    ;If Dimension Exists, And the number of "Rows" is Valid, and the Dimension desired is not > 1, then:
    ;For the Case that the array is 1-Dimensional
            If UBound($aArray, 0) = 1 Then;Makes sure the Array is only 1-Dimensional
            Dim $aArrayTmp[1];Declare blank array, which will hold the dimension declared by user
            For $i = 0 To $iUboundDim - 1
            _ArrayAdd($aArrayTmp, $aArray[$i])
            Next
            _ArrayDelete($aArrayTmp, 0);Get rid of 1st-element which is blank
        Else;For the Case that the array is 2-Dimensional
            Dim $aArrayTmp[1];Declare blank array, which will hold the dimension declared by user
            For $i = 0 To $iUboundDim - 1
            _ArrayAdd($aArrayTmp, $aArray[$i][$iDimension - 1]);$iDimension-1 to match Dimension
            Next
            _ArrayDelete($aArrayTmp, 0);Get rid of 1st-element which is blank
        EndIf
    EndIf
EndIf

Local $sHold;String that holds the Unique array info
For $iCC = $iBase To UBound($aArrayTmp) - 1;Loop Through array
;If Not the case that the element is already in $sHold, then add it
    If Not StringInStr($vDelim & $sHold, $vDelim & $aArrayTmp[$iCC] & $vDelim, $iCase) Then _
    $sHold &= $aArrayTmp[$iCC] & $vDelim
Next

If $sHold Then
    $aArrayTmp = StringSplit(StringTrimRight($sHold, StringLen($vDelim)), $vDelim, 1);Split the string into an array
    Return $aArrayTmp;SmOke_N's version used to Return SetError(0, 0, 0)
EndIf

Return SetError(2, 0, 0);If the script gets this far, it has failed
EndFunc;==>_ArrayUnique

Pay attention !

script makes alphabetic order to result file.

m.

Edited by myspacee

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0