Looking for a way to find duplicate pictures.

[John117]

Hey, I have about 5k pictures in one folder. about 2/3 of them are duplicates with different names. Picassa is now exhausted.

I need way to identify duplicates and then move them.

Here is what am thinking

match based on dimensions, dpi, and size.

They have different names, but may be the same picture.

I was also considering a pixel match based on 4 points. (like a square) in from the corners.

So, maybe 10% in then 20% in and so forth. -store these then compare to the next. . . .

Any better ideas?

Will post in examples when complete with credit for anyone that helps! Could make for a nice piece of software!

Edit: eye can spell!

Edited May 18, 2009 by John117

[AdmiralAlkex]

What format is the pictures in? Jpg, Bmp or.... ?

[John117]

  AdmiralAlkex said:

What format is the pictures in? Jpg, Bmp or.... ?

I would say 95% are jpg

[John117]

  Thanubis said:

How about using a MD5 Hash or CRC? Identical files will have the same MD5/CRC value.

http://www.autoitscript.com/forum/index.php?showtopic=76976

yeah, I was thinking about that using crc32 - I downloaded duplicate finder. but it found 18 dupes. I can spot that many in the first 30-40 files alone. Maybe limitation of the trial?

In either case, looking up post now . . .

[LurchMan]

if i remember right, if you change the filename then the MD5 / SHA1 Hash will be different...been awhile since i did this in college..

[John117]

  LurchMan said:

if i remember right, if you change the filename then the MD5 / SHA1 Hash will be different...been awhile since i did this in college..

odly enough, I found that some of the files, have different names and are 1-5kb different in size.

not sure how, much copy paste I guess . . .

[nitekram]

  John117 said:

odly enough, I found that some of the files, have different names and are 1-5kb different in size.

not sure how, much copy paste I guess . . .

does the time stamp do anything for you?

[KaFu]

For truly identical files give my program "SMF" a try ...

If the files are not identical but similar the only program I know to be capable of identifying those is d'peg (http://www.gotdupes.com/index.cfm?page=3495&pagename=d%60peg!). I guess you could perform something similar by calculating Color Checksums with the ImageMagick Suite.

Edited May 18, 2009 by KaFu

[LurchMan]

  Thanubis said:

No, the file name has no effect on the hashes. As long as the the file type and file content is the same, they will match regardless of the name.

couldnt remember for sure...i went to college for computer forensics but ended up doing programming once i got a job lol....

[John117]

  nitekram said:

does the time stamp do anything for you?

not sure, which timestamp? The created date is different, but the modified date is the same. Sometimes it was modified before it was created :-)

[John117]

Am currently running Dupdetector - it may do the trick! :-) cleaned about a thousand so far . . . we will see.

[junkew]

With logic in

http://www.autoitscript.com/forum/index.php?showtopic=66545

you could write a picture comparison in autoit

[AutoBert]

Hash (_Crypt_HashFile) all files and compare the hash.

Edited March 23, 2016 by AutoBert

[InunoTaishou]

Hopefully OP comes back to look at his 7 year old topic to see the solution.