Welcome Guest

Pages: 1
Find Duplicate Tracks Action
liamk23PostNovember 23, 2024, 09:58
Newbie
Posts: 20
Registered:
April 14, 2017, 15:43
Normal topicFind Duplicate Tracks Action

Hi, hope all is well.

I am looking to create an action, I am hoping you can help or give direction.

I have a master folder of music and a downloads folder (the downloads folder may have folders within it with more tracks in, which I would like included in the action, I don't want to maintain the folder structure)

I would like an action which duplicate checks all tracks within the downloads folder against tracks within my master music folder.
If a duplicate is found in the downloads folder then it is moved to a duplicates folder.
If a track is found to be a new track and not a duplicate then it is moved to a new tracks folder.

The duplicate check is to be based on a fuzzy % match with the artist, fuzzy % match with the title and a song length within 4 seconds.

As a bonus it would be nice if the action could also automatically create a copy of the file in a backup folder regardless if it is a duplicate or not.

Thank you 🙂

2MR2PostNovember 23, 2024, 11:03
Avatar photo
Administrator
Posts: 2167
Registered:
August 23, 2012, 19:27
Normal topicFind Duplicate Tracks Action

Oddly enough this is easier than it seems. The only change would be to base it on a 5 second difference as opposed to 4.

The Yate database viewer can search for duplicates based on fuzzy differences in the Title, Album, Artist and exact matches on Track, Disc and Year. It can also use a time difference of 5 seconds. All tests are optional.

So the process would be to create a database with sufficient information to do a 'search for duplicates'; search for duplicates and save the results and then run an action over the modified database to do the moving.

You can download an action to produce the database at:

https://2manyrobots.com/actions/Create-Database-Duplicate-Minimal-Database.zip

To produce the database do the following:

- start the Batch Processor Wizard from the Actions menu
- select the Create Database Duplicate Minimal Database action
- hit Next
- drag the root of your Master folder to the target
- drag the downloads folder to the target
- verify the folders via View Folders
- hit Next
- disable Auto save files
- enable Verbose log
- hit start
- go get a coffee

when the process finishes you should have the following files

~//Documents/Yate/DupsDB.csv
~/Documents/Yate/DupsDB.plist
~/Library/Application Support/Yate/BatchProcessorLog.txt

Send me all three files and I'll write the second action. If you'd prefer to not send me the database files just send the log. I'll tell you to how to generate the duplicates when I've done the second action.

liamk23PostNovember 23, 2024, 11:33
Newbie
Posts: 20
Registered:
April 14, 2017, 15:43
Normal topicFind Duplicate Tracks Action

Hi, I am hoping to make this process a one button press and it does it in the background as I will continuously use this action to bring in new tracks to the master library, maybe that is not possible? I ran the above and got the log below, I have opened the database and can see all the files listed:

Version 7.0.0.1 (12109.12098) [ARM64] (R)

Batch processing started Saturday 23 November 2024 at 16:18:13 Greenwich Mean Time

Root action: Create Database Duplicate Minimal Database
Root folder: /Users/liam/Music | DJ
Root folder: /Users/liam/Downloads/downloads

--------------------

Wrote database /Users/liam/Documents/Yate/DupsDB.csv

--------------------

Folders scanned: 158
Folders with audio: 140
Files processed: 18460

Run completed at Saturday 23 November 2024 at 16:18:53 Greenwich Mean Time
Run duration: 00:00:40

2MR2PostNovember 23, 2024, 12:32
Avatar photo
Administrator
Posts: 2167
Registered:
August 23, 2012, 19:27
Normal topicFind Duplicate Tracks Action

At this time I can't automate running the dup search on the database from an action. If I add that in 7.1 we could automate the entire process.

For now you'd have to produce the database, run the dup process and then the action.

If you want to proceed now, zip up and send me the two database files to support@2manyrobots.com so I can see what's going on and write the action.

I'll look at automating the dup process in 7.1

liamk23PostNovember 23, 2024, 13:44
Newbie
Posts: 20
Registered:
April 14, 2017, 15:43
Normal topicFind Duplicate Tracks Action

Sent, thank you

2MR2PostNovember 23, 2024, 15:58
Avatar photo
Administrator
Posts: 2167
Registered:
August 23, 2012, 19:27
Normal topicFind Duplicate Tracks Action

I've had a quick look and your criteria involving only Title and Duration will not work. You have a lot of tracks which have no Title metadata.

In my testing, I used Title (80%), Artist (70%) and Duration (5 seconds).

1790 tracks were identified and there were 839 unique groupings. Some tracks had more than one duplicate.

What do you want done when:

- all matches of a track are entirely in Downloads? Copy 1 I assume.
- all matches of a track are entirely in 'the other folder'? Eliminate duplicates?
- matches are split between Downloads and 'the other folder'? Copy none?

What criteria should I use to determine which copy of a file to keep? Most metadata of Title, Artist, etc.? Best bitrate?

I would like an action which duplicate checks all tracks within the downloads folder against tracks within my master music folder.
If a duplicate is found in the downloads folder then it is moved to a duplicates folder.
If a track is found to be a new track and not a duplicate then it is moved to a new tracks folder.

I can take this literally in which case I will ignore duplicates only in 'the other folder'.

I suggest that you play with the duplicates functionality to determine your match criteria. As I said, I used Title (80%), Artist (70%) and Duration (5 seconds)

- Open the produced database and from the context menu do a 'Search for Duplicates'.
- Enter your criteria and Start
- The database will have a Results column where the same value represents 'part of a group'

Let me know and I'll get you something over the next few days.

liamk23PostNovember 23, 2024, 16:30
Newbie
Posts: 20
Registered:
April 14, 2017, 15:43
Normal topicFind Duplicate Tracks Action

Thank you, much appreciated! My intention is to use this action as a kind of filter for new tracks to ensure they are not already in my library, in the hopes it saves me time by not tagging or listening to tracks I already have. The metadata I expect will be better than this going forward with more modern sources of music, a lot of these tracks are historic, its a good test as I expect this selection of files to be worst case.

- all matches of a track are entirely in Downloads? Copy 1 I assume. - Yes that's correct.
- all matches of a track are entirely in 'the other folder'? Eliminate duplicates? - Ignore duplicates found in master library.
- matches are split between Downloads and 'the other folder'? Copy none? - If a match is in main library then I am happy to discard any matches found in downloads to the duplicates folder.

I will have a play with the duplicates function as you describe. Please don't spend lots of time on this, I am happy to tinker and figure things out if needed. Thank you as always.

2MR2PostNovember 23, 2024, 16:34
Avatar photo
Administrator
Posts: 2167
Registered:
August 23, 2012, 19:27
Normal topicFind Duplicate Tracks Action

No problem. The action will not be too difficult.

Pages: 1
Mingle Forum by Cartpauj | Version: 1.1.0beta | Page loaded in: 0.041 seconds.