Writing new ID3 Tags [#6218]

[renowden]

Hi - I would like to write a script which would replicate the existing ReplayGain and RGAD tags, currently created by MM, and create an iTunNORM tag (part of a COMM tag) so that iPods, Sonos and other such devices would recognise them. I know that MP3Tag can do this but I would rather stick to one piece of software. There is not a big problem doing the arithmetic but can anyone tell me if there is a mechanism to get right down deep and write tags directly to the music files in MM or any other way to trigger a new tag?

[jiri]

MM interface doesn't directly allow ID3 tag custom modifications. I think that the best solution in this particular case would be to add it to the wish-list so that this feature is added directly to MM and based on configuration, iTunNORM is either created, left as is, or deleted, according to user's preferences.

Jiri

[trixmoto]

I thought it was on the cards for MM4 that we would be able to read/write custom tags via the scripting APIs? I've been waiting for this for my PUID scripts.

[nohitter151]

it's tracked at http://www.ventismedia.com/mantis/view.php?id=6218

[jiri]

After reviewing the potential usage of such a feature, I decided that we could try to implement a pretty generic scripting interface that would let script developers to implement features like this. I have proposed a simple interface in http://www.ventismedia.com/mantis/view.php?id=6218 that would handle all the tag reading and writing. I'd be interested in feedback from the interested developers...

Thanks,
Jiri

[ZvezdanD]

Yeah, great for that, but where do you think we scripters should store those tags? Into 5 existing Custom fields? Such solution would just cause the appearance of many different "standards" of storing data in MM database clashing each other. There is already bunch of scripts that are using one or more custom fields exclusively. You should first ensure some official way to store several different unrelated data into the same field or you should add more new custom fields to the Songs table. How about my suggestion which is related to this: Several new custom fields in Songs table that will not be connected to any tags needed?

Would your solution allow us to read non-supported frames from ID3 tags like TPE4 for Remixers and other wanted tags?

[Lowlander]

The problem would be displaying this info to the user. In other words if scripts add custom tags they should also be able to display them in the File Listing and add them for editing to the Properties dialog.

[ZvezdanD]

The problem would be displaying this info to the user. In other words if scripts add custom tags they should also be able to display them in the File Listing and add them for editing to the Properties dialog.

MM4 already has added an option for scripters to customize the Properties dialog. However, you are right about the main tracklist (and Now Playling panel as well). My suggestion with the new custom fields not referenced to file tags enables that such fields could be addressed by Summary column adjusting its mask (although I suggested adding several Summary columns on some another place).

[jiri]

Zvezdan, this feature would be useful in many cases when new DB fields aren't even needed - one of the examples would be the OP iTunNORM tag handling, others could involve handling of some specific fields, like prevention of rating storage to tags, etc.

That said, yes, I agree that some more fields (apart from the fixed Custom1-5) could be useful. However, given the complexity of all the related tasks (to avoid cluttering the DB, resulting in very slow MM, ...) I don't see it as the top priority at this moment.

Jiri

[ZvezdanD]

I am sorry, but I don't understand what is a point of reading/writing some tags if they are not stored into the database and cannot be displayed in the main tracklist.

With a little more effort and very little intervention on the database you could extend your suggestion to allow not only manipulation of the new fields when reading/writing tags from files, but also in many other situations as I described at the previous link. OK, that my suggestion assumed that you add several new custom fields to the Songs table which you obviously don't want. However, I have two new proposals and I will try to explain DB and API approach using two new custom fields: Country and Language, both of them could be multi-item, e.g. Country could be USA;Mexico and language could be English;Spanish.

1. proposal: storing multiple user fields into some existing unused field from the Songs table, e.g. Remixer. Syntax of the data stored in that field in DB could be e.g.
"Country:USA;Mexico/Language:English;Spanish" with the named parameters. Your suggested new GetFieldValue/SetFieldValue methods could be used to get those parts using API, e.g. GetFieldValue("Country") and SetFeildValue("Country", "USA;Mexico"). That is not very elegant solution because it requires parsing of that field, but that task would be left to you MM developers to do it internally and scripters would have a simple API interface to deal with those new fields.

2. proposal: is much more elegant, but it requires one new table added to the database, let's call it Tags. With that suggestion you would store the new custom fields into the List and ListsSongs tables beside of the existing Tempo, Mood, Occasion and Quality.
Songs:
ID = 123

ListsSongs:
ID = 101, IDSong = 123, IDListType = 5 (or higher), IDList = 456
ID = 102, IDSong = 123, IDListType = 5, IDList = 457
ID = 103, IDSong = 123, IDListType = 6, IDList = 458
ID = 104, IDSong = 123, IDListType = 6, IDList = 459

Lists:
ID = 456, IDListType = 5, TextData = "USA"
ID = 457, IDListType = 5, TextData = "Mexico"
ID = 458, IDListType = 6, TextData = "English"
ID = 459, IDListType = 6, TextData = "Spanish"

Here is the only new thing, the Tags table:
ID = 11, IDListType = 5, TagName = "Country"
ID = 12, IDListType = 6, TagName = "Language"

The API in this proposal would be the same as in the previous one with just two new methods.

As you could see, both proposals ensures almost unlimited number of new fields. The benefit is that we scripters would have the official way to use them, and we would have simple API interface with just two new methods already suggested by you.

I didn't mention anything about reading/storing of those fields into files, that is something that you should decide what would be the best. Maybe you could implement both of my proposals in the same way as you already store e.g. the same Tempo string into the Songs table and the Lists/ListsSongs tables. In that case you could read/write all those fields from/to the one new unused tag frame like TPE4.

The user presentation of those fields would be somewhat more problematic. The scripters could modify the Properties dialog to allow editing of those fields, but their display in the main tracklist is not currently possible. However, you could add one new field to the masks syntax corresponding to the DB field that I suggested in the first proposal and the users could use that field in the Summary column, specifying the mask e.g. <CustomTag>. Even better, you could enhance that to <CustomTag:Country> with the specified name of the new field. Such mask's field could be used everywhere where you use the masks, e.g. for Auto-organize, not limited only for specifying the Summary column.

I have proposals added to Mantis.

[booblers]

I am sorry, but I don't understand what is a point of reading/writing some tags if they are not stored into the database and cannot be displayed in the main tracklist.

Being stored in the database would make some data more useful (so too with displaying the extra stored tags). But you asked what the point would be for the read/write capability without display/db storage? Here are some examples:

1) Audio Fingerprints. Note that these are different than the associated IDs (a PUID is not the same as the MusicIP fingerprint) - these are long strings that the user would basically never have a reason to need to see in a UI or work with in a database (provided there is a scripting method to access them). These typically need to be generated once and then never interacted with by the user again - certainly not with a UI. This would be true for echoprints, acoustid fingerprints, and MusicIP fingerprints.

2) Audio Fingerprint IDs: see above. There is no real reason a user would ever need to know what the ID for an associated Fingerprint on given track is. Again - this data just needs to be stored in the file and available for reading/writing.

3) Other static unique identifiers. These include things like a MusicBrainz RecordingID/ArtistID/ReleaseID/etc. Now the MusicBrainz site does have a front end and users might be able to make some use of this information through a UI but mostly they just want it to "work." Very few people would be interested in having this data available in a UI. This information would be very useful for storing in tags in order to perform very fast auto-tagging in the future that is tied to canonical data. So for instance a user could switch to or add metadata from Discogs or MusicBrainz or etc without fear that the recording was misidentified.

4) A lot of data that is currently stored in Custom1 or etc tags that the user doesn't actually need to see visually. For instance: I'm sure the vast majority of people using AutoRateAccurate don't care what the string looks like.

5) Any future metadata/tag that players use but that humans never really work with. Renowden's request at the start of the thread is a good example. How many users need to interact with the iTunNORM tag in a UI? Probably not many - but it would be nice to be able to generate those tags with a script from ReplayGain nonetheless. This is an example of a situation where it would be nice for database capability so that these values wouldn't have to be generated for an entire library over and over without performing a read operation on every file.

None of this is to say that UI/database storage for arbitrary tags is bad or that I can't think of examples where it would be necessary - but you asked what good it would be to add read/write without them.

For the life of me I can't figure out why the capability would be event-based rather than a simple method attached to a SongData object or attached to a TaggedFile object which is then attached to a SongData object - but that's another issue I suppose.

I should note that I agree with you that without some kind of db/display support scripters will implement things in haphazard and idiosyncratic ways - but at least there IS a way to do it. We can make our own tables and ui elements and do those things if necessary right now. What we *cannot* do is make that information static in the file.

[booblers]

Sorry - just so you don't think I'm being a downer Z here are some examples of situations where database/ui capability would add to what I said above and make MM up-to-date:

Currently MM approaches metadata in a "string" fashion. An "artist" is just a string that is in the Artist or AlbumArtist tag/db field. The only exception is ";" to add multiple values to a field/tag. This is a very 2003 approach to metadata. There are hundreds of albums where what appears in the "AlbumArtist" tag should be handled very differently (other tags as well).

MusicBrainz (and other metadata sites) have all switched to a much more robust system. Hence the need to add these new unique identifiers to the tags in files. What having them available in the database would do is allow for much more accurate results from searches. Many artists have gone by different "names" over the years and sometimes their releases have "display names" in certain fields and etc. Searching for these using MM is a real problem and requires that the user either normalize all fields (and give up the "display name") or stick with the display name and lose the ability to properly search for an "object."

A real world example (there are millions): "Sean Combs" aka "Puff Daddy," aka "P. Diddy" released his 1995 album under "Puff Daddy." Lately he's releasing under "P. Diddy" - Currently one must either pick a common name or give up the ability to relate them to each other in MediaMonkey. Both are bad options.

This is true in more mundane ways as well - everything from the "(feat. <some artist>)" which often appears in Title tags/fields to simple "Artist & Artist" strings.

The basic problem is, as I said above, that MediaMonkey approaches metadata as if it were a bunch of strings rather than a bunch of related objects which also happen to have display strings associated with them. One can see why: the application as it ships has no way of knowing how to create the objects and relationships for the user. But, of course, the users *could* do this (with scripts or manually) if the capability was made available.

It would make for much, much more powerful 3rd party applications and scripts. AutoDJs could be significantly more accurate, for example, and then there are things like chart scrapers, and other music acquisition/resolution functions which would all see enormous benefit.

As I've said in other places: MM is stuck in a very old way of approaching audio in this respect... but at least Jiri appears to be looking into adding the very basic tools necessary for developers to help catch MM up.

[ZvezdanD]

Being stored in the database would make some data more useful (so too with displaying the extra stored tags). But you asked what the point would be for the read/write capability without display/db storage? Here are some examples:

OK , I see your point, thanks for the detail explanation. I take back my question about the need to display all tags into the GUI, but I still cannot see a point not to store tags into the database. At least such approach would speed-up access to those tags since searching of the database is much faster then searching and reading those tags from files.

4) A lot of data that is currently stored in Custom1 or etc tags that the user doesn't actually need to see visually. For instance: I'm sure the vast majority of people using AutoRateAccurate don't care what the string looks like.

Actually, I already have a request for MM developers to allow us to use one or more fields which would not be related to any tag in file, it is already mentioned in this thread twice: Several new custom fields in Songs table that will not be connected to any tags needed. As I said, I agree with you that such data should not be displayed in GUI, but they still need to be stored into the database.

This is an example of a situation where it would be nice for database capability so that these values wouldn't have to be generated for an entire library over and over without performing a read operation on every file.

Exactly my point.

For the life of me I can't figure out why the capability would be event-based rather than a simple method attached to a SongData object or attached to a TaggedFile object which is then attached to a SongData object - but that's another issue I suppose.

How would you know when some tag needs to be read/written if you are not informed with the suggested event? It is like the OnTrackProperties event which informs scripts that some tag/field is updated by user and that they should do something appropriately to that.

We can make our own tables and ui elements and do those things if necessary right now. What we *cannot* do is make that information static in the file.

Yes, we can do our own tables, but not UI elements. We cannot display some new field in the main tracklist. We cannot search specifically for new fields using the Search box. We cannot use Auto-Organize with the new fields, nor any other operation in the program which is based on masks like e.g. <Artist>.

When I say "new fields", I am not thinking only about some Audio Fingerprints and similar data, but also about another examples like the same artist with multiple names, artist with "feat." in names...

[booblers]

I don't disagree with you - the more functionality the better, as I noted in my next post to you.

How would you know when some tag needs to be read/written if you are not informed with the suggested event? It is like the OnTrackProperties event which informs scripts that some tag/field is updated by user and that they should do something appropriately to that.

I wasn't clear. Yes, the events are in some cases necessary. What I don't understand is why it sounds like that's the only place we will be able to actually write the tags.

Currently if you wish to change a tag you get a SongData object and then just change the field eg: SongData.ArtistName = "foo" then SongData.WriteTags (or similar).

It looks like this will not be possible in the proposed system. In that system one wouldn't be able to directly do something like:

SongData.TaggedFile.SetFieldValue( field: string; value: string)

In other words: it looks like the TaggedFile object will not exist outside of the event as far as I can tell. Unless I'm just misunderstanding. (See Jiri's post here: http://www.mediamonkey.com/forum/viewto ... 40#p327848 )

It sounds from the proposal like one must register an event, wait for the event concerning the file we wish to change to fire that event (by manually calling WriteTags) and then process the event to make our actual changes.

Is that more clear? In other words: I don't understand why the TaggedFile object shouldn't exist outside of the event for easier access in many applications (every application I can think of for which I would want to use the object).

If I want to read a tag from a file or write one, why must it be done through an event?

Don't get me wrong - as I've said before - I'll use the system if that's all we're given but it seems odd to me.