Newly-imported tracks default to ratings "bomb" [#2147]

[Melodian]

I'm having difficulty understanding how MM's "ratings" work, due to some recent bizarre behavior when scanning tracks. When I put new tracks into an existing folder and re-scan with MM (using the "insert" key), all the new tracks (and ONLY the new tracks) that have been added since the last scan come up with the "bomb" icon displayed in the ratings field. I would expect this field to be empty unless I specifically assigned a rating to that track. To remove the "bomb" I select all the tracks containing them and press "alt-minus". Oddly, sometimes MM will whip through the files with great speed and accomplish this in a matter of a few seconds, but most often it's a VERY slow process which takes perhaps 15 to 30 seconds per track, sometimes longer. As the number of marked tracks increases, the total time to complete the entire process seems to increase exponentially to where it might take an hour or more to remove the "bombs" for several hundred files. It seems like MM is trying to re-write the entire ID3.v2 tag for each file, but I thought the rating was stored in MM's database and not in the file tag itself.

If it wasn't for the fact that I use the ratings field when auditioning new tracks to mark selected ones for inclusion in my podcast, I wouldn't even bother with the ratings data at all.

Can anyone clarify for me exactly how and where the rating information is stored, why newly imported tracks are rated as "bombs" and why it might be taking so long to remove them? Is there something I can do with a bulk tag editor to pre-process the files before importing into MM? Any ideas or suggestions welcome -- thanks.

[Lowlander]

Either the files already had the rating set or you have a rating script working on the files.

And yes ratings are stored in the tags: http://www.mediamonkey.com/sw/webhelp/f ... erties.htm

[Melodian]

Lowlander, thanks for the info. So if I understand this right, MM is looking for a POPM "Popularimeter" tag field in the file to determine the rating as it scans it into the database? I've found these values to be:

0x00 - no stars
0x01 - 0x1F - 1 star
0x20 - 0x5F - 2 stars
0x60 - 0x9F - 3 stars
0xA0 - 0xDF - 4 stars
0xE0 - 0xFF - 5 stars

From what I can tell, POPM is a fairly new definition contained in the ID3.v2.3 spec, and older tag editors/catalogers don't support it. So if I'm using a bulk-tagger on my new music files before scanning them in with MM, is it likely not going to write a POPM tag at all? In this case, would MM find a value of 0x00 (no stars) and interpret this as a "bomb"? How is a rating of zero different from lack of any rating at all? (i.e., alt-0 versus alt-minus)

I'm still confused, but think I may be slowly grasping some of this. Any further help appreciated.

BTW - no scripts at all are active.

[Lowlander]

Try scanning in files before using another program to tag them and see if the bomb rating still shows up. I don't know what the value for bomb would be, also MediaMonkey uses a range value I believe.

[gk]

In this case, would MM find a value of 0x00 (no stars) and interpret this as a "bomb"?

This is, unfortunately, exactly right - sadly, MediaMonkey is in violation of the official id3.org specification for how the ID3v2 'POPM' frame should be used:
http://www.id3.org/id3v2.3.0#head-2452e ... ff67970852

4.18. Popularimeter

The purpose of this frame is to specify how good an audio file is. Many interesting applications could be found to this frame such as a playlist that features better audiofiles more often than others or it could be used to profile a person's taste and find other 'good' files by comparing people's profiles. The frame is very simple. It contains the email address to the user, one rating byte and a four byte play counter, intended to be increased with one for every time the file is played. The email is a terminated string. The rating is 1-255 where 1 is worst and 255 is best. 0 is unknown. If no personal counter is wanted it may be omitted. When the counter reaches all one's, one byte is inserted in front of the counter thus making the counter eight bits bigger in the same away as the play counter ("PCNT"). There may be more than one "POPM" frame in each tag, but only one with the same email address.

<Header for 'Popularimeter', ID: "POPM">
Email to user <text string> $00
Rating $xx
Counter $xx xx xx xx (xx ...)

As you can see from the spec, the POPM value of 0x00 is reserved to indicate that the rating is 'unknown'; i.e., not rated.
MM encodes a rating of 'worst' (bomb) as 0x00 when it should actually be using 0x01, per the spec.

A related issue is that, instead of setting POPM value to 0x00 to indicate 'not rated', MM (and J. River Media Center also) deleted the entire POPM frame.
Although this tends to have the desired effect, why not follow the spec?

I see no advantage to 'going your own way' on stuff like this. For example, one side effect of deleting the POPM frame entirely when a rating is removed is that this changes the size of the mp3 file, which would NOT occur if only the byte for the rating value were changed to 0x00. If syncing, for example, were to be dependent on a change in file size, this would be true when a rating is changed to 'not rated' but not true if rating were changed from one value to another - unnecessarily inconsistent, in my opinion.

A couple more instances where MM is out of compliance:

1. MM encodes the 'highest' rating (5 stars) with POPM value of 0xFC, rather than 0xFF, which the spec calls for. It is quite likely that another application, which followed the spec, would look at a '5 star' mp3 file, rated by MM and display it as 4.5 stars, since it is clearly tagged as 'less than highest', which is obviously NOT what the hapless MM user intended when choosing the '5 star' rating!

2. Assuming we vow to simply avoid using the blatantly non-compliant MM 'bomb' rating, the 'lowest' available rating (0.5 stars) is assigned a POPM value of 0x0D by MM, when a value of 0x01 should be used.

Why does MediaMonkey make things so confusing when the spec is so clear?

[gk]

To remove the "bomb" I select all the tracks containing them and press "alt-minus". Oddly, sometimes MM will whip through the files with great speed and accomplish this in a matter of a few seconds, but most often it's a VERY slow process which takes perhaps 15 to 30 seconds per track, sometimes longer. As the number of marked tracks increases, the total time to complete the entire process seems to increase exponentially to where it might take an hour or more to remove the "bombs" for several hundred files. It seems like MM is trying to re-write the entire ID3.v2 tag for each file

Again, per my prior post, this horribly inefficient situation is entirely due to the fact that MM ignores the ID3v2 specification, which states that, the POPM frame rating field should simply be set to 0x00, when a rating is removed, which would be very fast since it only requires writing one byte of data to the file.

Instead, in DELETING the entire POPM frame to accomplish this simple change, MM is not only rewriting the 'tag' data, but, since the tag data appears at the beginning of the mp3 file, it is re-creating the entire mp3 file, music data and all - no wonder this takes a long time!

Now, to explain your observation that "sometimes MM whill whip through the files ...", I'm not absolutely certain, but I think this may be true where the ID3v2 tag uses 'padding' bytes to maintain a consistent size for the tag - whether or not the original tag had padding would depend on the application which most recently edited the tags.

If you want to test this out, try this experiment:
Pre-process the files you import into MM with an application like Mp3 Tag/Studio which will do the following:
1. add padding to the id3v2.3 tag data
2. set the POPM rating field to 0x00 (unrated)
[Tech NOTE: if you rewrite the entire POPM frame, as hex data, make sure you include the null-terminator for an empty Email field, after the POPM header and before the rating field, as follows: 50 4F 50 4D 00 00 00 02 00 00 00 00]

Even though MM will still be inefficient since it will delete the POPM frame when you delete the 'bomb', hopefully it will not have to re-write the music data to do this since the tag padding ensures that the music data begins at the same offset into the file, even when the entire tag is re-written.

[nohitter151]

Perhaps the reason is because no one else follows the spec either! What good is going with the "Standard" when no other programs do it that way! Users don't care about what some "standard" is, they care about compatibility with other programs. By following the true ID3 standard, MM is no longer compatible with WMP and Windows explorer.
see http://www.mediamonkey.com/forum/viewto ... ng#p226576

[gk]

Perhaps the reason is because no one else follows the spec either!

That is not true.
J. River Media Center follows the id3v2 spec, on this, and many other points.
Here's a summary of the current situation, in regard to the POPM rating field.

Code: Select all

Rating                      worst       best    un-rated        
-----------------------------------------------------------
id3v2.3 spec                0x01        0xFF    0x00 (or no POPM)
JRiver Media Center v15     0x01        0xFF    0x00 (or no POPM)
MediaMonkey v3.2.2.1300     0x00        0xFC    no POPM

If MediaMonkey wants to "go its own way" on how to store ratings, then it should do either of two things:

1. Honestly and openly, tell your users exactly what you are doing with POPM and how it differs from the spec - we deserve this.
Silently breaking the rules is much worse than breaking the rules, since it introduces corruption into our data, without our consent.

2. Don't use the POPM frame. If you don't like the spec, use a COMM frame, or a custom frame of your own design.

POPM has a clear definition. By calling your data 'POPM' data and privately changing the semantics from the published spec, it is like saying: "From now on, we at MediaMonkey are going to use the word 'five' to mean the number 4: 2+2=5. Voila!"

If this doesn't bother you, then consider the following situation.

Suppose an MM user spends many hours rating their files, then decides to switch to J. River, or another id3v2-compliant program (hopefully there will be more, as time goes on).

When the user leaves MM, they will have the follow conditions:
A. 'worst' rated files will have 0x00 ('bomb') ratings in the POPM frame.
B. unrated files will have no POPM tag at all.

After importing these files to the compliant player, unrated files and 'worst' files will all look the same: zero stars.
At this point, unless the user promptly notices the problem and takes action to correct it, data corruption can occur.

This situation could be called "lock-in" - MediaMonkey is steering user data AWAY from compliance with an open standard.

I can work around data corruption when I know about it - but if Melodian hadn't reported this problem (and me doing a little bit of research), I would have never known what was going on with my data. I think we are owed an apology for the lack of transparency in how MediaMonkey has behaved.

[Lowlander]

I would assume that being compatible with WMP and Windows is more important than JRiver (and thus also the standard) as the former has a much larger market share thus would cause problems for more users if it would be incompatible. Of course ideally MediaMonkey would follow the standard, but this would be of negative impact on more users than the current implementation. This is going by the fact that I've seen more complaints about rating discrepancies between MediaMonkey and WMP/WinExplorer than between JRiver and MediaMonkey here on the forum.

There is the following in the bug tracker, not sure what it means though: http://www.ventismedia.com/mantis/view.php?id=2147
Bug: http://www.ventismedia.com/mantis/view.php?id=6660

[gk]

I would assume that being compatible with WMP and Windows

As far as I can tell, WMP doesn't even use the POPM tag so there are no compatibility issues there. I don't know what you're talking about.

[nohitter151]

I would assume that being compatible with WMP and Windows

As far as I can tell, WMP doesn't even use the POPM tag so there are no compatibility issues there. I don't know what you're talking about.

Yes it does, at least certainly it can read that field, so I don't know where you got that info. The point is, different software uses different standards, and probably the most important are Windows Explorer and Windows Media Player, especially since MM is a windows only program. So it makes more sense to adhere to their standard than whatever others are out there.

[gk]

I would assume that being compatible with WMP and Windows

As far as I can tell, WMP doesn't even use the POPM tag so there are no compatibility issues there. I don't know what you're talking about.

Yes it does, at least certainly it can read that field, so I don't know where you got that info.

I got the info by testing it:
1. open an mp3 file in a text editor - verify it has no POPM frame
2. open the same file in WMP and rate it (try different ratings if you like)
3. open the file in a text editor again and see what happened: still no POPM frame
4. google the subject, etc. - according to all documentation I could find, WMP never writes rating data to the mp3 file; it stores it in the database

The point is, different software uses different standards, and probably the most important are Windows Explorer and Windows Media Player, especially since MM is a windows only program. So it makes more sense to adhere to their standard than whatever others are out there.

That would be fine if MediaMonkey were simply honest about it and called the data it writes something else than POPM.
POPM has a very clear definition which, in part, says that 0x00 rating means "not defined" and 0x01 means "worst".

Why does MM have to hijack the spec? This is simply evil.

[nohitter151]

As far as I can tell, WMP doesn't even use the POPM tag so there are no compatibility issues there. I don't know what you're talking about.
I got the info by testing it:
1. open an mp3 file in a text editor - verify it has no POPM frame
2. open the same file in WMP and rate it (try different ratings if you like)
3. open the file in a text editor again and see what happened: still no POPM frame
4. google the subject, etc. - according to all documentation I could find, WMP never writes rating data to the mp3 file; it stores it in the database

Yes, but it READS the POPM tag from mp3 files. Verify it this way: tag an mp3 file in MM. Open that file in WMP. See that it is rated.

Same applies for windows explorer.

Why does MM have to hijack the spec? This is simply evil.

I don't know how many times I have to explain it. MM is not "evil", it just uses an implementation of POPM that makes the most sense in terms of compatibility. If it follows the "true" POPM standard, then the ratings won't match between Windows Explorer/WMP and MM! Then users see this and think MM is buggy because the ratings don't match!

[gk]

Why does MM have to hijack the spec? This is simply evil.

I don't know how many times I have to explain it. MM is not "evil", it just uses an implementation of POPM that makes the most sense in terms of compatibility.

I understand the point about trying not to look bad in a buggy Microsoft world, but that does not excuse lying or hijacking the spec.

There is only one 'implementation' of POPM, which is defined by id3.org, just as there is only one definition for the number 4.
When MusicMonkey tells users it is "following the POPM standard" and write a rating of "worst" as 0x00, this is evil.

POPM defines semantics which say that 0x00 MUST ONLY MEAN "not-rated" and 0x01 MUST ONLY MEAN 'worst'

WMP may read POPM and interpret it incorrectly but that is just a bug - neither Windows Explorer nor WMP claim to write POPM data and force their bugginess on the rest of the world; MM does.

[Lowlander]

But from a user aspect it creates a lot of issues as everybody is using Windows (and Windows Explorer) and many use WMP both of which have this faulty implementation. This is why it makes sense to do this despite that it's non-standard as this works out better for a larger number of MediaMonkey users. I wish it were another way and this is why a entry into Mantis was made to see if this deserves re-evaluation, but what Microsoft does is important as that's what many people use.
The fact that this isn't documented is something that was mentioned in the Bug report as well. Furthermore if I'm not mistaken you can disable MediaMonkey from writing to the POPM field as to not affect your files as described here: http://www.mediamonkey.com/support/inde ... ticleid=62

Also take in account that you're the first one to have an issue with the choice to be compatible with Microsoft instead of the standard. Again, I wish it were different and everybody would follow the standards, but this isn't the case and thus a choice had to be made and by the amount of posts on this the right one it seems.

PS. In the following it mentions that the WMP implementation of POPM is used: http://www.mediamonkey.com/sw/webhelp/f ... erties.htm