If you are seeing 'adding' then you are more than likely attempting to load your entire collection into the UI. Your Mac will run out of memory long before Yate loads 130k tracks. When draining the Mac that much many applications will come unstable and it is virtually impossible to test for for the condition.

The Invalid Tag Size is possible if one of the files has a corrupted tag. I'd be surprised if in a 130k tracks you didn't find a few of these.

This is not the way to go. As you have found out the UI cannot handle the load. However, Yate has a Batch Processor which can run over your entire collection and only load a folder at a time.

What you want to do can be done with actions but there is no way to do this directly in the UI.

You can write an action which will run through the Batch processor and produce the fingerprint, extract metadata from AcoustID, attempt to match and extract metadata from MusicBrainz. I can guarantee you that because of the nature of acoustic fingerprinting and matching, some results will be totally incorrect. Have many will be inaccurate depends on the contents of folders (single album, multiple albums or singles). Further with 130k tracks many will not be matched at all. It might pay to do the operation without committing the results, saving them to temporary locations in the tracks and/or producing a track database where you could look at the results. Looking at the results does not sound attractive with 130k tracks. Audio fingerprinting, unlike human fingerprinting is not exact. Quite often many tracks have the same fingerprint and this can lead to false matches.

Fingerprinting if it works will get you MusicBrainz IDs which will enable you to extract artwork from a few sources, if it can be found. Unfortunately, quite often Google is the best source for artwork.

Assuming that some sort of success could be attained, Yate could find the duplicates in a produced Track database (doing this is a one line action).

I don't know the quality of the metadata you currently have in the files. It is quite easy to produce a track database which will describe the metadata in each track in your collection. You could also display the number of artwork items in each file. To do this you would create an Export set describing the fields that you are interested in. You do this in Preferences-Export Sets. You more than likely want to include at a minimum the following fields:
Album, Artist, Album Artist, Genre, Title, Artwork Count and File Path. You must have File Path.

You can then create a one line action with a 'Create Track Database' statement, using the Export set that you created.

You would then run the action through the Batch Processor. When done, you will have a database file which you can open in Yate or any other application capable of reading a CSV file. If your metadata is decent, you could always attempt the duplicate detection without any other processing.

You can of course go any way you like. Personally I would not attempt to 'blindly' update the metadata of 130k tracks.

It is tough for me to even give sound advice at this point. There are a lot of unknowns. (again, folder contents and structure). There are applications out there which claim to 'fix' a collection with a single button press. I guess it works for some people but I've also heard horror stories. It's not the quantity of songs which worries me ... it is inaccurate results.

Unfortunately most music programs work best with a folder structure which represents the metadata of the tracks, Most people use some variant of:

music/artist/album/tracks

That being said some people separate multi-cd albums in separate folders and many people vary the metadata which appears in the artist and album path components.

Yate cannot handle 130 tracks in a single folder in any execution mode including Batch Processing. Every file has to be loaded, the metadata extracted, the artwork stored, etc. The Mac will start failing memory allocations at some point regardless as to how much physical memory you have. Unlike the old days, most modern applications do not test for memory failures as almost every single line of code allocate memory in some form or another. I have seen user load north of 5k tracks at a time and I have loaded close to that in my own testing. However, the Mac will become unresponsive dues to the amount of swapping which can occur.

The Batch Processor loads a single folder at a time, typically recursively processing folders containing a collection. This s the advantage, you can process 100k+ files by only loading 10-20 at a time.

Yate contains an action system...scripting. You can write actions do do almost anything, including fingerprinting, etc.

Here is another problem, If all your tracks are in one folder, it will be almost impossible to make good guesses regarding matched AcoustID information as there are virtually no album associations in your collection that can be used to verify what is happening. The algorithms which try to group tracks into albums when matching AcoustID data would take ages to run with a set of 130k files, eve if they would load.

The beneficent of a Track Database would be the ability to see what you have. However, we cannot even generate that as no mode in Yate will load the 130k files. If you randomly split the folder into reasonable chunks (probably 1000 or less), you should be able to attempt matches as single tracks...ignoring albums.

I can't see anything in Yate working until that folder is split up.

Sorry, I wish I had a better answer for you.

You answered while I was typing another reply. 🙂

A number of points. First there is probably a way to get Yate to read the folder and either move the files (without your intervention), or to do the fingerprinting, AcoustID extraction and to create a database file.

I'll come to the method (or at least the first step) at the end of this reply.

I can manage the files but unless you already have some albums with consistent album metadata, there will be little chance of grouping the tracks into albums without errors. Many songs appear on multiple albums, compilations, etc. By having albums to associate with, the task is manageable, but we may not have that. As far as if your tracks will be able to extract any data at all from AcoustID, there is no way to know without trying. Fingerprints are user contributed and there is no way to know what is in there without looking.

If we can pass the next test, it should be possible to enumerate the tracks, one at a time. The process of attempting to get where you want to be will require a number of steps and I can't guarantee the results.

Hopefully there is something meaningful in the tracks, as I assume that you have some means of identifying songs now so that you can listen to them.

Let's try this:

Download the following file:

https://2manyrobots.com/actions/GetPathContents.zip

Do the following:

- unzip the file
- start Yate
- set Yate's Open Mode to Playlist. File menu>Open Mode>Playlist Mode
- drag a single track from anywhere into the main window's file list area or use the File>Open command to open a single track. Any track will do. Yate needs at least one loaded file in order to run an action.
- from the Yate menu do an Import Preferences on the unzipped GetPathContents.plist file
- if not already selected, select the single track in the main window.
- From the Actions menu choose User Defined>Get Path Contents
- you will be prompted for a folder
- select the folder which contains the 130k tracks
- depending on where the files are stored the enumeration may take a while. When the action finishes and the spinner disappears from the main window, your Documents folder will contain a file named FileList.txt
- please email the file to support@2manyrobots.com

If the file contents are as I expect they should be it will be possible to think about moving forward, without you splitting anything manually.