Jump to content
Sign in to follow this  
cloq

Help - Delete partial duplicate instances

Recommended Posts

cloq

Hi,

Having a hard time figuring out how to keep the first instance of of row of text (based on partial column string) and deleting all other similar instances.

I know how to do simple remove duplicates but this been giving me a headache.

I have attached an example csv file.

csv basically:

x,y,z,0

x,y,z,1

x,y,m,7

x,y,z,3

x,d,m,1

x,d,m,2

f,d,z,4

f,d,m,6

f,d,m,5

g,bb,m,d

g,bb,m,c

g,bb,m,c

g,bb,m,c

---------

what the output should be:

x,y,z,0 (first instance based on column 1)

f,d,z,4 (first instance based on column 1)

g,bb,m,d (first instance based on column 1)

Thanks for any assistance.

input.txt

Share this post


Link to post
Share on other sites
FireFox

Hi,

Here you go :

#include <Array.au3>

Global $aReadRow = StringSplit(FileRead("input.txt"), @CrLf, 1)

Global $aRowNoDupl[1][2], $sFirstRowContent, $iRowNoDuplUBound

For $iRowIndex = 1 To $aReadRow[0]
    $sFirstRowContent = StringLeft($aReadRow[$iRowIndex], StringInStr($aReadRow[$iRowIndex], ",") -1)
;~     $sFirstRowContent = StringLeft($aReadRow[$iRowIndex], 1) ;in case the 1st row content as always a lenght of 1

    If $iRowIndex = 1 Then
        $aRowNoDupl[0][0] = $sFirstRowContent
        $aRowNoDupl[0][1] = $aReadRow[$iRowIndex]
    Else
        If _ArraySearch($aRowNoDupl, $sFirstRowContent) = -1 Then
            $iRowNoDuplUBound = UBound($aRowNoDupl)

            ReDim $aRowNoDupl[$iRowNoDuplUBound +1][2]

            $aRowNoDupl[$iRowNoDuplUBound][0] = $sFirstRowContent
            $aRowNoDupl[$iRowNoDuplUBound][1] = $aReadRow[$iRowIndex]
        EndIf
    EndIf
Next

Global $aRowNoDuplCleaned[UBound($aRowNoDupl)]

For $iRowNoDuplCleanedIndex = 0 To UBound($aRowNoDupl) -1
    $aRowNoDuplCleaned[$iRowNoDuplCleanedIndex] = $aRowNoDupl[$iRowNoDuplCleanedIndex][1]
Next

_ArrayDisplay($aRowNoDuplCleaned)

Br, FireFox.


 

OS : Win XP SP2 (32 bits) / Win 7 SP1 (64 bits) / Win 8 (64 bits) | Autoit version: latest stable / beta.
Hardware : Intel(R) Core(TM) i5-2400 CPU @ 3.10Ghz / 8 GiB RAM DDR3.

My UDFs : Skype UDF | TrayIconEx UDF | GUI Panel UDF | Excel XML UDF | Is_Pressed_UDF

My Projects : YouTube Multi-downloader | FTP Easy-UP | Lock'n | WinKill | AVICapture | Skype TM | Tap Maker | ShellNew | Scriptner | Const Replacer | FT_Pocket | Chrome theme maker

My Examples : Capture toolIP Camera | Crosshair | Draw Captured Region | Picture Screensaver | Jscreenfix | Drivetemp | Picture viewer

My Snippets : Basic TCP | Systray_GetIconIndex | Intercept End task | Winpcap various | Advanced HotKeySet | Transparent Edit control

 

Share this post


Link to post
Share on other sites
Spiff59

#include <Array.au3>
#include <File.au3>
Global $aList

_FileReadToArray(@ScriptDir & "input.txt", $aList)
$idx = 0
For $x = 1 to $aList[0]
    $temp = "__" & StringLeft($aList[$x], StringInStr($aList[$x], ",") - 1)
    If Not IsDeclared($temp) Then
        $idx += 1
     $aList[$idx] = $aList[$x]
        Assign($temp, 0)
    EndIf
Next
ReDim $aList[$idx + 1]
$aList[0] = $idx
_ArrayDisplay($aList)

Edit: update array count ($alist[0])

Edited by Spiff59

Share this post


Link to post
Share on other sites
FireFox

oh, seems like my mind is too much complicated. I'm out on this.


 

OS : Win XP SP2 (32 bits) / Win 7 SP1 (64 bits) / Win 8 (64 bits) | Autoit version: latest stable / beta.
Hardware : Intel(R) Core(TM) i5-2400 CPU @ 3.10Ghz / 8 GiB RAM DDR3.

My UDFs : Skype UDF | TrayIconEx UDF | GUI Panel UDF | Excel XML UDF | Is_Pressed_UDF

My Projects : YouTube Multi-downloader | FTP Easy-UP | Lock'n | WinKill | AVICapture | Skype TM | Tap Maker | ShellNew | Scriptner | Const Replacer | FT_Pocket | Chrome theme maker

My Examples : Capture toolIP Camera | Crosshair | Draw Captured Region | Picture Screensaver | Jscreenfix | Drivetemp | Picture viewer

My Snippets : Basic TCP | Systray_GetIconIndex | Intercept End task | Winpcap various | Advanced HotKeySet | Transparent Edit control

 

Share this post


Link to post
Share on other sites
cloq

@Spiff59 and FireFox

Thank you! this gives me something to study and work with.

Much appreciated!

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×