Advanced Duplicate Find & Fix 3.8.2 (2014-01-27)

Download and get help for different MediaMonkey for Windows 4 Addons.

Moderators: Peke, Gurus

Bex
Posts: 6316
Joined: Fri May 21, 2004 5:44 am
Location: Sweden

Advanced Duplicate Find & Fix 3.8.2 (2014-01-27)

Post by Bex »

Advanced Duplicate Find & Fix

The screen shoots and the instructions are for the moment a bit outdated, please read the Update log for latest info!
You find download link and installation instructions at the bottom of this post.

This script is rather four scripts in one!
- Copy/Paste: Tags - Album Art - PlayStatistics, between any tracks
- Find More From Same - Custom Search
- Find Duplicate Tracks by metadata (tags) or Duplicate Tracks/Folders by Hash value (file finger print)
- Find UnAnalyzed tracks i.e. tracks with no Hash value

The main purpose of this script is of course to find duplicates but instead of just deleting them, the script allows you to merge their PlayStat so your PlayHistory, PlayCount, DateAdded and Playlist Entries are intact.
The script makes it also very easy to replace old low bitrate files with new rips in higher bitrates since it lets you Copy/Paste metadata (Tags, Album Art, PlayStat) from one set of files to another very easy.

Some screen shots:

In bottom of "File To Edit" node. You'll find this:
Image

The "UnAnalyzed Tracks" node expanded
Image

The "Folder With same Content" node expanded:
This node finds folders which contains the exact same music tracks.
(Album art and other files are not considered.)
This is useful if you downlo... Err rip the same CD multiple times.
Image

- "Tracks With same Content"
Image

When expanding the "Create Keep/Delete Suggestion" nodes.
You must define how to find the tracks to keep.
Image

Then you can check the result. If you sort the tracks and then want to get back the original sort order. Just right click on the nodes.
Image

If you find tracks you want to delete in the "Delete Suggestion" node.
Select them, right click and chose the first option from the menu.
Image

And like magic the selected database values are transferred to the corresponding track in the "Keep Suggestion" node
Image

- "Custom Duplicate Search"
When expanding the node. You'll get this.
Image

And then you get this:
Image

When expanding the "Create Keep/Delete Suggestion" nodes.
You must define how to find the tracks to keep.
Image

- Copy/Paste MetaData
You find this feature in the right click menu in the main window:
Image

These Database values and tags can be Copied/Pasted:
Image

There are three automatically detected modes in which metadata can be Copied/Pasted between tracks:
You can configure three default paste actions in every mode for faster pasting.
- ListToList
In this mode both Database values and tags can be copied
Very useful for copying tags between two identical albums.
E.g. When reripping an existing CD with a higher bitrate.
List A, track 1 goes to List B, track 1
List A, track 2 goes to List B, track 2
Etc...
Image


- OneToMany
In this mode only tags can be Copied/Pasted
Useful for fixing tags in incorrectly tagged albums.
In this mode, plus the special case OneToOne, the "Single Field Paste" option gets activated.
Image


- ManyToOne
In this mode only Database values can be transferred
Useful for consolidation PlayStat between Duplicates.
Image

If you make an impossible selection. This happens:
Image

- "Find More From Same - Custom Search"
This is useful for finding duplicates from a set of tracks or to make extra analyses of one or more tracks.
Note, you can edit the text in all text fields to further enhance your search.
Image

The result of the search is presented like this.
Note that you can choose to include the seeding track(s) in the search result.
Image


Special thanks to Trixmoto who has coded everything with the Copy/Paste Album Art functionality!

Enjoy!
/Bex
___________________________________________________________
Updates:
Ver 3.8.2 (2014-01-27)
-Fixed Search on FileName in Custom Duplicate Search didn't work properly
-Fixed "Single Paste" of Artist/Director didn't work
-Renamed "Single Paste" to "Single Field Paste"
-Added Collection Name to each Main Node of the script

Ver 3.8.1 (2012-11-10)
-Fixed SQL errors with "Create Keep/Delete Suggestion Nodes" in CDS when a tolerance was applied
-Fixed SQL errors in right click menu "Delete Selected and/or Transfer their Playstat"
-Fixed "Create Keep/Delete Suggestion Nodes" in "Tracks with same Content"-Node didn't respect "Filter by Playlist"

Ver 3.8.0 (2012-10-27)
- Moved the script from the main tree into every Collection
- Changed Filters to be Playlists instead
- Implemented individual tmp tables per collection so the script can be ran without interference between collections
- Replaced same/diff dropdowns with buttons instead (CDS)
- Implemented configurable "pages" to reduce the size of the form (CDS/FMFS)
- Added the Keep/delete node settings so they are remembered per preset (CDS)
- Fixed "in/Not in"-Playlist (previously filter) in Keep/Delete node didn't work properly (CDS/TSC)
- Added Date (Release Date) to Keep/Delete (TSC/FSC)
- Implemented individual Playlist filter per collection (TSC/FSC)
- Added right click short cut menus on the nodes to set "Filter by Playlist" (TSC/FSC)
- Fixed so the TSC Ok-List is individual per collection. Old values are transferred to Entire Library (TSC)
- Added search in Collection (FMFS)
- Fixed search for filename didn't work (FMFS)
- Fixed search didn't work properly for EQUALS, STARTS WITH, SMART CONTAINS for text fields (FMFS)
- Fixed form wasn't resizeable (FMFS)
- Renamed "Smart Contains" to "Contains Full word" (FMFS)
- Rearranged the form (Copy/Paste)
- Added all new MM4 fields (Copy/Paste)

Ver 3.7.4 (2010-08-21)
- Fixed "Find More From Same" didn't work properly

Ver 3.7.3 (2010-07-26)
- Fixed error when using "Delete from Library and Computer (Without Confirmation)" and "Move the "Pasted To tracks" to the "Copied from tracks" old path"

Ver 3.7.2 (2010-07-06)
- Fixed error when Pasting MetaData using Manual Selection

Ver 3.7.1 (2010-06-28)
- Fixed a rare error with updating the .ini file

Ver 3.7 (2010-06-27)
- Added Conductor and Grouping to both "Custom Duplicate Search" and "Find More From Same"
- Added possibility to configure shortcuts to both "Copy/Paste Metadata" and "Find More From Same"
- Added possibility to Move the "Pasted To tracks" to the "Copied from tracks" old path in Copy/Paste metadata
- Fixed bug in "Custom Duplicate Search" when adding a new preset

Ver 3.6.2 (2010-04-07)
- Fixed SQL-Bug when having single quote in custom field names

Ver 3.6.1 (2010-03-19)
- Fixed SQL-Bug in "Create Keep/Delete Suggestion Nodes"

Ver 3.6 (2010-02-28)
- Added "Paste Artist to other unselected Persons" in Copy/Paste Metadata
- Added new Filter-functionality in Custom Duplicate Search, It's now possible to find dupes from a set of tracks
- Added "Only display Group Nodes With a TrackCount of X" in Custom Duplicate Search (Idea by DasKraut)
- Added new criteria AlbumTrackCount in "Create Keep/Delete Suggestion Nodes" (Idea by RifRaf)
- Fixed Bug in "Delete Selected and/or transfer their playstat" in Track with same content
- Fixed "Show Select Transfer/Delete Actions"-setting wasn't working
- Fixed Error with fields containing double quotes in Find More From Same
- Changed how Ratings is displayed in the nodes since the previous way didn't work for some locales

Ver 3.5.2 (2009-07-10)
- Added possibility to hide fields in both "Custom Duplicate Search" and "Find More From Same"
- Redesigned the Option Sheet

Ver 3.5.1 (2009-07-06)
- Added Custom1-5 in both "Custom Duplicate Search" and "Find More From Same"
- Fixed a bug in "Custom Duplicate Search". It wasn't possible to use Size in Folder mode.
- Fixed a bug in "Find More From Same". Searching for blank values wasn't possible.

Ver 3.5 (2009-04-29)
- Added "Display result Grouped By Folders" in "Custom Duplicate Search" :D
- Fixed a bug in "Folders with same Content". It didn't find all folders that were identical :oops:
- Tuned some SQL-Statements in all Folder nodes
- Changed so used Filters are saved with FilterID instead of FilterName in the ini file
+ some minor fixes

Ver 3.4.1 (2008-02-24)
- Fixed FMFS data from first seeding track was used for all seeding tracks
- Fixed FMFS changed text wasn't considered

Ver 3.4 (2009-02-04)
- Added Filter to "Folders with Same Content" (Select filter in the options)
- Added Filter to "Tracks with Same Content" (Select filter in the options)
- Added menu-shortcut to the options (Right click on ADFF's Main Node, MM 3.1 only)
- Fixed AV-Error when closing MM (Happened only in MM 3.0)
- Tuned some SQL-statements. "Find More From Same" is now very fast (MM 3.1 only)
+ some smaller fixes


Ver 3.3.1 (2009-02-04)
- Updated "Find More From Same"
- - Added VBR/CBR
- - Fixed "Contains" didn't work properly
- - Fixed "Smart Contains" didn't work sometimes
- - Redesigned the Form
- - Changed layout in the form to better show how "AND/OR" radio-buttons work
- Updated "Create Keep/Delete Suggestion Nodes" in both "Custom Duplicate Search/Same Content"
- - Added VBR/CBR
- Updated "Custom Duplicate Search"
- - Added VBR/CBR
- - Redesigned the Form
- Updated "Copy/Paste Metadata"
- - Redesigned the Form
+ many more smaller fixes and tweaks

Ver 3.3 (2009-01-11)
- Totally Reworked "Find More From Same"
- - Added Unlimited numbers of Configurable AutoSearches, executed from right click menu
- - Added a lot of new fields
- - Added more Search Operators
- - Added AutoSearch name to the Selection Node
- - Added FMFS to right click menu in "Now Playing Main Window" (3.1 only)
- Reworked "Create Keep/Delete Suggestion Nodes" in both "Custom Duplicate Search/Same Content"
- - Added Previous Keep-Criteria is remembered
- - Added Highest/Lowest "Rating"
- - Added In/Not In "Filter"
- - Added "Manage Filter"-Button (3.1 only)
- - Added "Keep Selections" Node
- Updated "Custom Duplicate Search"
- - Added Unlimited numbers of Configurable Presets
- - Added Rating
- - Added Original Artist
- - Added "Manage Filter"-Button (3.1 only)
- Updated "Copy/Paste Metadata"
- - Added Unlimited numbers of Configurable "AutoActions"
- - Added "Copy/Paste Metadata" to right click menus in both "Now Playing" and "Now Playing Main Window"
+ many more smaller fixes and tweaks

Ver 3.2.3 (2008-10-09)
- Fixed error with "Starting a transaction while some query is open" (I hope)
- Fixed bug, Month and Day weren't pasted from the Date tag when using "OneToOne - Single Paste"
- Fixed so restart isn't needed when upgrading

Ver 3.2.2 (2008-07-03)
- Added possibility to include seeding tracks to the result in "Find More From Same - Custom Search"
- Fixed Tolerance mode and figures weren't read from saved layouts in "Custom Duplicate Search"

Ver 3.2.1 (2008-06-13)
- Fixed error when pasting Tags if "Don't paste empty Tags" was selected

Ver 3.2 (2008-06-06)
- Added Month and Day to the Date tag (Replaces the Year tag)
- Added OriginalMonth and OriginalDay to the OriginalDate tag (Replaces the OriginalYear tag)
- Increased the maximum Tolerance values for all fields
- Minor cosmetic changes

Ver 3.1.1 (2008-05-29)
- Fixed strange result when using <Different> together with tolerance

Ver 3.1 (2008-05-29)
- Reworked the "Custom Duplicate Search" node
- - Added Create Keep/Delete Suggestion Nodes
- - Added SampleRate
- - Added Tolerance in Length, Size and Bitrate
- - Added "Compare modes" in Length, Size and Bitrate
- Fixed bug when saving option sheet. (Some values were reset)
- Fixed add to OK-list didn't exist when in "Add All Tracks"-node
- Fixed Selection nodes were presented in reverse order

Ver 3.0.1 (2008-05-07)
- Fixed Tags weren't always pasted when Copy/Paste metadata (i.e. nothing happened)
- Minor text changes


Ver 3.0 (2008-05-05)
- Reworked the Copy/Paste functionality
- - Added Track#, Disc#, Custom 4-5 and Grouping Tags
- - Added Paste modes. (ListToList, OneToMany, ManyToOne)
- - Added Automatic detection of paste mode
- - Added Single Field Paste
- - Added Configurable default paste actions
- - Added Counts of copied tracks and when pasting selected tracks
- - Added reset option
- - Improved Delete Options
- - Improved error handling on impossible selections
- Reworked the Custom Duplicate Search Node
- - Added Search for Same or Different
- - Added Optionally Exclude unknown values
- - Added Optionally use Filters
- - Added Configurable default layouts
- - Improved various SQL statements
- - Improved Layout/Selection node
- - Improved error handling on impossible selections
- - Fixed node counts disappear when closing the node
- - Moved Add All tracks to own Node
- Added “Folders with Same Content” Node
- Improved “Tracks with Same Content” Node
- - Added “Create Keep/Delete Suggestion Nodes”
- - Added Automatic transferring of playstat upon deleting tracks in “Delete Suggestions” Node
- - Added Sort Order is remembered between Keep and Deletes Nodes
- - Added Right click menu “Sort Tracks in Node Order” on Keep and Deletes Nodes
- Improved “Unanalyzed Tracks” Node
- - Added Count of tracks and folders
- - Fixed so only unanalyzed tracks are added to main window
- Improved “Find More From Same” Functionality
- - Added possibility to edit the “search” text
- - Added PlayCount
- - Fixed so no seeding track is in the result in the “Search Result” Node
- - Fixed several fields didn't work as intended or returned wrong result
- - Fixed several SQL issues
- - Fixed cosmetic issues
- Removed all Drag & Drop Functionality (Use Copy/Paste instead)[/b]
- Created installer
___________________________________________________________
Download:
Latest version:
MM4 (Installer)
Download the script from its Addon Page. Please also rate the script, especially if you like it! :)

MM3 (Installer)
Advanced Duplicate Find & Fix 3.7.4



Installation Instructions:
MM3 (Installer)
Avoid "Product installation error"
- Vista Users:
- - To be able to install scripts you must Run MM as an administrator.
- - It means that you must right click the MM icon and select "Run as administrator" even if you are logged in as an administrator.
- All Users:
- - Check in your task manager that you only have one instance of MediaMonkey.exe running.

1. Download the mmip file and double click on it.
2. Restart MediaMonkey if needed
3. Go to the "Files To Edit"-Node. In there you will find the "Advanced Duplicate Find & Fix"-Node.
4. Right click on any tracks will give you access to "Copy/Paste Metadata" and "Find More from Same - Custom Search".


- Upgrade:
As above but if you wish to save the old version simply rename its extension to .BAK or similar before you run the installer
Last edited by Bex on Thu Jul 16, 2009 5:04 pm, edited 85 times in total.
Advanced Duplicate Find & Fix Find More From Same - Custom Search. | Transfer PlayStat & Copy-Paste Tags/AlbumArt between any tracks.
Tagging Inconsistencies Do you think you have your tags in order? Think again...
Play History & Stats Node Like having your Last-FM account stored locally, but more advanced.
Case & Leading Zero Fixer Works on filenames too!

All My Scripts
trixmoto
Posts: 10024
Joined: Fri Aug 26, 2005 3:28 am
Location: Hull, UK
Contact:

Post by trixmoto »

Looks extremely impressive - I'm looking forward to playing with this, and will report back soon! :)
Download my scripts at my own MediaMonkey fansite.
All the code for my website and scripts is safely backed up immediately and for free using Dropbox.
Mizery_Made
Posts: 2283
Joined: Tue Aug 29, 2006 1:09 pm
Location: Kansas City, Missouri, United States

Post by Mizery_Made »

Looks like a nicely done script, great idea too. Really expands on what the default node lacked and that was information.

I just installed it, well, made the file & restarted. Not sure that technically constitutes "Installing" haha. Anyway, I can't really push it to the edge though, atm, because I only seem to have 1 duplicate entry and that's because the song was on two different CD's and I have both in my library, thus it's fine.

Though, I look forward to giving it a whirl the next time I need to figure out the mess of some dups. :)
Teknojnky
Posts: 5537
Joined: Tue Sep 06, 2005 11:01 pm
Contact:

Post by Teknojnky »

861 duplicates! I have alot of work to do... :o

Very nice script, thanks Bex! Much appreciated! :D

edit:

First suggestion, I like the 'add to OK list' idea, but would be nice to have it on the node with #123 (2x) artist - track so you can "OK" all of the dupes in that node without having to add each one.
Bex
Posts: 6316
Joined: Fri May 21, 2004 5:44 am
Location: Sweden

Post by Bex »

Thanks guys!

@Trixi,
Awaiting report :wink:

@Mizery_Made,
Only two dups? Just add them to the OK-list since you want to keep them.

@Teknojnky,
861 dups :o
I had like 500 and thought I was in the top but you bet me big time!
I'm down to 400 now and eventually all will be OK'ed or deleted.

Regarding not be able to "Add to OK-List" from the "same content" subnode. It's intentional to force you to look at the tracks, so you dont miss anything... I might change it in the next version.
Advanced Duplicate Find & Fix Find More From Same - Custom Search. | Transfer PlayStat & Copy-Paste Tags/AlbumArt between any tracks.
Tagging Inconsistencies Do you think you have your tags in order? Think again...
Play History & Stats Node Like having your Last-FM account stored locally, but more advanced.
Case & Leading Zero Fixer Works on filenames too!

All My Scripts
trixmoto
Posts: 10024
Joined: Fri Aug 26, 2005 3:28 am
Location: Hull, UK
Contact:

Post by trixmoto »

How are these duplicates calculated? I've got none apparently!
Download my scripts at my own MediaMonkey fansite.
All the code for my website and scripts is safely backed up immediately and for free using Dropbox.
Bex
Posts: 6316
Joined: Fri May 21, 2004 5:44 am
Location: Sweden

Post by Bex »

Exactly the same as in the Duplicate Content node. Is that one also empty?
Advanced Duplicate Find & Fix Find More From Same - Custom Search. | Transfer PlayStat & Copy-Paste Tags/AlbumArt between any tracks.
Tagging Inconsistencies Do you think you have your tags in order? Think again...
Play History & Stats Node Like having your Last-FM account stored locally, but more advanced.
Case & Leading Zero Fixer Works on filenames too!

All My Scripts
Mizery_Made
Posts: 2283
Joined: Tue Aug 29, 2006 1:09 pm
Location: Kansas City, Missouri, United States

Post by Mizery_Made »

Bex wrote:@Mizery_Made,
Only two dups? Just add them to the OK-list since you want to keep them.
Yep, only two Dups. I only mentioned it since given I know both are "different," then I can't test the merging, etc features and thus help "test" for bugs and such.

Hmm, I was just thinking, since your "OK List" are just the duplicate items that YOU mark as being OK. So, (dunno if it'd be possible) but maybe an option in the right click menu or somewhere else where you could Select to "Hide" the OK list, and be able to uncheck the option if you need to double check the contents.

Iunno, may be impossible to implement, and it may not even be needed as the OK List only takes up ONE line when it's collapsed. Just trying to throw ideas out there though.
Bex
Posts: 6316
Joined: Fri May 21, 2004 5:44 am
Location: Sweden

Post by Bex »

Yeah, it's only one "line" so it's better to leave it as it is.
Thanks for the suggestion though!

If you have played both of your duplicate tracks you can merge them (without delete one of them) if you want, so all stats goes into one of them. That way the statistics will be more correct, at least for one....
Advanced Duplicate Find & Fix Find More From Same - Custom Search. | Transfer PlayStat & Copy-Paste Tags/AlbumArt between any tracks.
Tagging Inconsistencies Do you think you have your tags in order? Think again...
Play History & Stats Node Like having your Last-FM account stored locally, but more advanced.
Case & Leading Zero Fixer Works on filenames too!

All My Scripts
Spazz
Posts: 243
Joined: Tue Jan 11, 2005 9:49 pm

Post by Spazz »

One thing that I think would be interesting would be to link OK files so that if the dup is played the playcount data is matched on both files.
Image
Bex
Posts: 6316
Joined: Fri May 21, 2004 5:44 am
Location: Sweden

Post by Bex »

Well, I see why you want todo it but I think it would "damage" your playstatistics... or do you mean it would fix it?

The playhistory, which can be displayed in my "Play History Node" script, can not be duplicated. So it would be inconsistent in a way to give all copies of a song the same playcount.

Anyway, there are other scripts that could alter the playcount. You could try them if you really are sure that you are doing the right thing.
Advanced Duplicate Find & Fix Find More From Same - Custom Search. | Transfer PlayStat & Copy-Paste Tags/AlbumArt between any tracks.
Tagging Inconsistencies Do you think you have your tags in order? Think again...
Play History & Stats Node Like having your Last-FM account stored locally, but more advanced.
Case & Leading Zero Fixer Works on filenames too!

All My Scripts
Spazz
Posts: 243
Joined: Tue Jan 11, 2005 9:49 pm

Post by Spazz »

Well, either that or wipe the one you specifiy as a dup. Example being I have a song that is on the bands cd, and then 5 soundtracks. I would like to keep the main bands song up to date, for use in smart playlists. what would be best would actually be linking one song to multiple albums but until that happens it's all workarounds. :(
Image
Bex
Posts: 6316
Joined: Fri May 21, 2004 5:44 am
Location: Sweden

Post by Bex »

Yes, a link to all copies of a track would solve this issue. It can't be done with scripts though since the database structure has to be changed...
Advanced Duplicate Find & Fix Find More From Same - Custom Search. | Transfer PlayStat & Copy-Paste Tags/AlbumArt between any tracks.
Tagging Inconsistencies Do you think you have your tags in order? Think again...
Play History & Stats Node Like having your Last-FM account stored locally, but more advanced.
Case & Leading Zero Fixer Works on filenames too!

All My Scripts
Spazz
Posts: 243
Joined: Tue Jan 11, 2005 9:49 pm

Post by Spazz »

Actually, maybe an option to reset the song played data instead of deleting the file? That way you could periodically merge the dupes in. I've been considering setting up my smartplaylists to filter out dupes on compilations and soundtracks. Sounds like a project. That way the only way the duplicates would get played would be if you played the compilation as a whole and then you could merge the playcounts.

I think that makes sense, maybe just in my head though.
Image
Bex
Posts: 6316
Joined: Fri May 21, 2004 5:44 am
Location: Sweden

Post by Bex »

Spazz wrote:Actually, maybe an option to reset the song played data instead of deleting the file? That way you could periodically merge the dupes in. I've been considering setting up my smartplaylists to filter out dupes on compilations and soundtracks. Sounds like a project. That way the only way the duplicates would get played would be if you played the compilation as a whole and then you could merge the playcounts.

I think that makes sense, maybe just in my head though.
Well, that's what this script does. It doesnt delete the tracks unless you say so- Or don't I understand what you mean?
Advanced Duplicate Find & Fix Find More From Same - Custom Search. | Transfer PlayStat & Copy-Paste Tags/AlbumArt between any tracks.
Tagging Inconsistencies Do you think you have your tags in order? Think again...
Play History & Stats Node Like having your Last-FM account stored locally, but more advanced.
Case & Leading Zero Fixer Works on filenames too!

All My Scripts
Post Reply