Welcome Guest
Pages: 1
Comparing Two Databases
PhukkinPostApril 19, 2015, 12:57
Newbie
Posts: 17
Registered:
March 28, 2014, 05:21
Normal topicComparing Two Databases

Hey 2MR2,

I was wondering if there is any way of using Yate to compare a csv file against my primary database and highlight/generate a list of any that do not have a match in the primary database? I only need to match Album/Artist fields. Tried looking around for a simple app to do this but turned out a bit harder than I thought it would be.

Thanks in advance
2MR2PostApril 19, 2015, 13:01
Avatar photo
Administrator
Posts: 2364
Registered:
August 23, 2012, 19:27
Normal topicRe: Comparing Two Databases

I need some clarification.

What is your primary database?

Is your csv file a Yate track or album database?
PhukkinPostApril 19, 2015, 23:05
Newbie
Posts: 17
Registered:
March 28, 2014, 05:21
Normal topicRe: Comparing Two Databases

Sorry, my primary database is a database I generated using Yate of all my artists and their albums, which I then edited in excel to have only the artist and album columns (thought itd be easier to compare that way). I then want to check this against another CSV, not yate generated but has the same two columns, album and artist. I then want to compare the two and highlight/export a second list of whatever album/artist combinations on the secondary csv that are not present on the primary database (or highlight the ones are present, it doesn't matter as long as I can identify what isn't present in the primary database).

The goal of this is to compare my library against other music library's and/or collections list (for example checking my library against a list of the 1001 albums you must hear before you die collection).

This would be a massive timesaver for me as I am trying cross check my database against others by manually searching each title, which is very time consuming.

Thanks again for any help 🙂
2MR2PostApril 19, 2015, 23:25
Avatar photo
Administrator
Posts: 2364
Registered:
August 23, 2012, 19:27
Normal topicRe: Comparing Two Databases

Well, it can be done although it will probably be slow. I'd probably run it as a dummy batch run so that you don't get a spinning beachball while the comparison is being done.

I can look into writing an action which would take the two files and produce a third containing the missing items. I'm trying to get v3.6 out within a week so you may have to wait for it to ship before I finish the action.

It would probably make the action run must faster if you could use Excel to output the files as tab separated values as opposed to comma separated values. This would remove the need of parsing double quotes and commas. Would that be okay for you? If it is please send me two test files to support@2manyrobots.com and I'll use those to test the action. I will not hard-code the action to the files you send. 🙂
PhukkinPostApril 20, 2015, 01:59
Newbie
Posts: 17
Registered:
March 28, 2014, 05:21
Normal topicRe: Comparing Two Databases

That'd be fantastic. And excel would be perfectly fine, or plain text, or anything I can concert a CSV to is fine. I'm not expecting perfect results, as despite my heavy scrutiny I'm sure there are some misspellings and variations in my database. One thing I will ask, is it possible to somehow alter the matching criteria so a double colon (:) will return a positive if a dash is present in it's place (-)? Because when formatting my files I change all double colons (usually they are in the album field) to space-dash-space to prevent file renaming errors with invalid characters. This isn't a major issue if it is difficult, as I can just create an action to filter any of these out of the database and check them manually, but would definitely be handy 🙂 thanks for all your help, I'll send through the files when I finish work. Would you prefer them as excel spreadsheets or CSV?
2MR2PostApril 20, 2015, 07:50
Avatar photo
Administrator
Posts: 2364
Registered:
August 23, 2012, 19:27
Normal topicRe: Comparing Two Databases

The :: conversion is not a problem. The best format for the files is tab delimited. If not possible comma delimited. Excel would be a problem as Yate can't process the binary data.
2MR2PostApril 21, 2015, 18:50
Avatar photo
Administrator
Posts: 2364
Registered:
August 23, 2012, 19:27
Normal topicRe: Comparing Two Databases

Don't worry about tab delimited files as opposed to CSV files. It is no longer an issue 🙂 CSV files will work just fine.
2MR2PostApril 28, 2015, 07:36
Avatar photo
Administrator
Posts: 2364
Registered:
August 23, 2012, 19:27
Normal topicRe: Comparing Two Databases

Hi. v3.6 is out and based on the new CSV support added to the action system I wrote an action which seems to do the job. The action uses CSV files and not tab-delimited which I previously requested. I need some 'live' data to test it on as there are many ways to tune it further...including using the new fuzzy compares. BTW, you no longer have to use Excel to remove a column. v3.6 allows to to create the CSV files as standalone without field requirements.

Whenever you get a chance send me the two files and I'll test it out further.
Pages: 1
Mingle Forum by Cartpauj | Version: 1.1.0beta | Page loaded in: 0.044 seconds.