Sign in to follow this  
Followers 0
Kaeft

Remove Duplicates

5 posts in this topic

#1 ·  Posted (edited)

OK so I'm going to attempt to read through a word file and remove all of the duplicate words.

the words are like this:

games
yahoo
hotmail
lyrics
hotmail.com
yahoo.com
jokes
ebay
maps
games
warez
napster
www.hotmail.com
song+lyrics
www.yahoo.com
dictionary

Problem is there are 900 words. I am thinking that I should do a _FileCountLine, accompanied by a While loop, a StringReplace (to delete the line), and then _FileWriteToLine. Make it take the first line and then check every line in the file for the first line and delete the same. Then take the second line, and the third, and fourth, etc. Maybe through it into a function, idk....

That would take forever... Do you guys know of a quicker way?

Edited by Kaeft

Share this post


Link to post
Share on other sites



Ok so I just did a quick sort through an array and then write back to the file and it seems there is a duplicate, triplicate, or quadruplicate of every single word. That will be handy for testing I suppose. Anyone have an idea yet?

Share this post


Link to post
Share on other sites

The algorithm you want is fairly simple. You just need to think through it.

If the array you start with is already sorted, you can just loop through the whole array and write the first element into a new file, then keep skipping the next element if it is the same as the element you just wrote.

Share this post


Link to post
Share on other sites

I am still learning all the different functions of autoit, but I'm going to assume that the function I'm looking for is in the array section. Good to know. I'll tell you how it goes... tomorrow, lol it's getting late.

Share this post


Link to post
Share on other sites

Or use _ArrayUnique

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0