Sign in to follow this  
Followers 0
billthecreator

How to delete strings within a string that are repeats

5 posts in this topic

#1 ·  Posted (edited)

i have a file with over 800 strings

how would i write a script that would delete the repeats

example:

This is String 1

This is String 2

This is String 3

This is String 4

This is String 1

This is String 3

This is String 8

and i want to remove the ones that are repeating.

also, it being over 800 lines, i started writing a for loop, but had no luck.

i also searched for something with deleting string repeats, and had no luck.

is there anyone who has done this before and would like to help me?

thanks.

Edited by billthecreator

[font=Microsoft Sans Serif]My Scripts: From Most recent to least.[/font]Countdown GUI | QLOCK TWO | FlipClock | Slot Machine My UDF:_GenerateRandomNoRepeat | _GuiSnap

Share this post


Link to post
Share on other sites



You should try out this guy's UDF, it looks like just what you need http://www.autoitscript.com/forum/index.php?showtopic=119596

Share this post


Link to post
Share on other sites

#Include <File.au3>

Local $aFile
_FileReadToArray(@ScriptDir & "\File.txt", $aFile)

_Array2DDblDel($aFile)

_FileWriteFromArray(@ScriptDir & "\FileNew.txt", $aFile, 1)


;----------------------------------------------------------------------------------------------------------------------
; Function  _Array2DDblDel(ByRef $ARRAY [, $CASESENS=0])
;
; Description - From an 1D/2D Array will delete double entries (2D -> combination by '[n][0]' to '[n][x]').
;    - Autodetection 1D/2D Array
;    - By using string, you can choose case sensitivity.
;
; Parameter  $ARRAY:   Array to sort
; optional $CASESENS:  Case sensitivity off[0] or on[1] (default 0)
;
; Return  Succes   ByRef Array without doubles
;        Count of doubles
;    Failure   0 and set @error = 1; no array
;
; Author  BugFix ([email="bugfix@autoit.de"]bugfix@autoit.de[/email])
;----------------------------------------------------------------------------------------------------------------------
Func _Array2DDblDel(ByRef $ARRAY, $CASESENS=0)
 Local $arTmp[1] = [''], $dbl = 0, $count = 0, $x, $l, $val, $valTmp, $i, $k
 If ( Not IsArray($ARRAY) ) Then
  SetError(1)
  Return 0
 EndIf
 Local $Ubound2nd = UBound($ARRAY,2)
 If @error = 2 Then
  For $i = 0 To UBound($ARRAY)-1
   $dbl = 0
   For $k = 0 To UBound($arTmp)-1
    Switch $CASESENS
     Case 0
      If $arTmp[$k] = $ARRAY[$i] Then 
       $dbl = 1
       $count += 1
      EndIf
     Case 1
      If $arTmp[$k] == $ARRAY[$i] Then
       $dbl = 1
       $count += 1
      EndIf
    EndSwitch 
   Next
   If $dbl = 0 Then
    If $arTmp[0] = "" Then
     $arTmp[0] = $ARRAY[$i]
    Else
     ReDim $arTmp[UBound($arTmp)+1]
     $arTmp[UBound($arTmp)-1] = $ARRAY[$i]
    EndIf
   Else
    $dbl = 0
   EndIf
  Next
 Else
  ReDim $arTmp[1][$Ubound2nd]
  $arTmp[0][0] = ''
  $x = 0
  For $i = 0 To UBound($ARRAY)-1
   $dbl = 0
   $val = ''
   $valTmp = ''
   For $l = 0 To $Ubound2nd-1
    $val &= $ARRAY[$i][$l]
   Next
   For $k = 0 To UBound($arTmp)-1
    For $l = 0 To $Ubound2nd-1
     $valTmp &= $arTmp[$k][$l]
    Next
    Switch $CASESENS
     Case 0
      If  $valTmp = $val Then
       $dbl = 1
       $count += 1
      EndIf
     Case 1  
      If  $valTmp == $val Then
       $dbl = 1
       $count += 1
      EndIf
    EndSwitch
    $valTmp = ''
   Next
   If $dbl = 0 Then
    If $x = 1 Then ReDim $arTmp[UBound($arTmp)+1][$Ubound2nd]
    For $l = 0 To $Ubound2nd-1
     If $arTmp[0][0] = '' Or $x = 0 Then 
      $arTmp[0][$l] = $ARRAY[0][$l]
      If $l = $Ubound2nd-1 Then $x = 1
     Else
      $arTmp[UBound($arTmp)-1][$l] = $ARRAY[$i][$l]
      $x = 2
      If $l = $Ubound2nd-1 Then $x = 1
     EndIf
    Next
   Else
    $dbl = 0
   EndIf
  Next
 EndIf
 $ARRAY = $arTmp
 Return $count
EndFunc ; ==>_ArrayDblDel


Programming today is a race between software engineers striving to
build bigger and better idiot-proof programs, and the Universe
trying to produce bigger and better idiots.
So far, the Universe is winning.

Share this post


Link to post
Share on other sites

You can try some cool stuff too.

$sString = FileRead("YourFile.txt")

For $sLine In StringRegExp($sString, "(.*?)\v+|", 3)
    StringRegExpReplace($sString, "\Q" & $sLine & "\E", $sLine)
    If @extended > 1 Then $sString = StringRegExpReplace($sString, "\Q" & $sLine & "\E\v+", @CRLF, @extended - 1)
Next

FileWrite("YourFile_New.txt", $sString)

♡♡♡

.

eMyvnE

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0