Jump to content
Sign in to follow this  
Delta

Comparing LARGE files

Recommended Posts

Delta

I was hoping someone could help me i have two LARGE files each is a list of about 10000 blacklisted sites. I want to combine this list but i don't want to make duplicate entries. Is it possible to compare each list in an autoit script and have it pull out the duplicate entries? or is there an easier way to remove them?


[size="1"]Please stop confusing "how to" with "how do"[/size]

Share this post


Link to post
Share on other sites
Das Ami

Read the files into 2 arrays and if-loop them, that's the only idea I have right now.

Share this post


Link to post
Share on other sites
Vindicator209

Somewhere along the lines of

$line=1
Do
$file1 = FileReadLine("file1.txt",$line)
$file2 = FileReadLine("file2.txt",$line)
$line = $line + 1
If $file1 = $file2 Then
FileWriteLine("NewFile.txt",$file1)
Else
FileWriteLine("NewFile.txt",$file1)
FileWriteLine("NewFile.txt",$file2)
EndIf
Until <===Some State Here

Then you can choose to delete the first 2 files..

havent tested that yet, but its something along those lines


[center]"When you look at old, classic games like Snake, you often put it off because it's such a simple game, but it's only when you actually try and create your own unique game from scratch, do you finally appreciate those games."[/center][center]Don't ask for answers if you haven't TRIED yet![/center][center]Most answers can be answered in the help file! Use it![/center]

Share this post


Link to post
Share on other sites
Delta

That wouldn't work because its only comparing the same line with each text file and both text files aren't in the same order.

Also i just found out there are comments in the second file. They look like this: [some comment here]. How would i mass remove the comments?

Edited by Deltaforce229

[size="1"]Please stop confusing "how to" with "how do"[/size]

Share this post


Link to post
Share on other sites
Das Ami

That wouldn't work because its only comparing the same line with each text file and both text files aren't in the same order.

Also i just found out there are comments in the second file. They look like this: [some comment here]. How would i mass remove the comments?

Stringinstr and Stringreplace.

http://www.autoitscript.com/forum/index.ph...ite=%2Bcomments

Share this post


Link to post
Share on other sites
Jos

One time job?

Download PSPad editor, copy the 2 files into one and do option Edit/Sort and tick remove duplicates...

:rolleyes:


Visit the SciTE4AutoIt3 Download page for the latest versions  - Beta files                                How to post scriptsource        Forum Rules
 
Live for the present,
Dream of the future,
Learn from the past.
  :)

Share this post


Link to post
Share on other sites
Delta

JdeB you rule!

*bows before the almighty JdeB*


[size="1"]Please stop confusing "how to" with "how do"[/size]

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.