[REQ] backup and restore auto-playlists

by **ZvezdanD** » Sun Nov 27, 2022 9:11 am

If you apply the Properties dialog box to retag group of tracks, it is not one transaction. Each affected track is updated in a separate transaction. It is one transaction only if you explicitly rename an album name using the Albums node in the Media Tree or, as you mentioned, using the Album panel.

Your example is assuming that all selected tracks are from a single album, and that only the album tag is affected. But, what if you have selected multiple tracks from several albums? Or, what if you have selected all tracks from the same artist and you change its name? Did you test an auto-playlist with "Artist" "is" "something" criterion and what happens with its ID when you change its name? Or, album artist, or composer, or genre or any other multi-item field that has its relational table? What if you retag not only album, but other tags as well at the same time using the same Properties dialog?

I suppose it would consume too much CPU time if the program needs to check what you have changed using the Properties dialog and how these modifications are affecting the database, just to be able to update IDs in the auto-playlists and collection rules.

However, all of our guessing are pointless. As I said, I am not the right person for this conversation and you should report it to those who are responsible.

by **Barry4679** » Sun Nov 27, 2022 4:32 am

ZvezdanD wrote: ↑Sat Nov 26, 2022 6:41 am The behavior that you are describing is intended, the program behaves like that since the beginning.

You say that like it is a benefit, or should be expected, from a User perspective.
I doubt that it is "intended". At best it is something that they didn't think through, but maybe it was an intentional shortcut, because they didn't place too much importance upon protecting the contents of auto playlists.

ZvezdanD wrote: ↑Sat Nov 26, 2022 6:41 amYou would think that if you select all tracks from an album and if you bulk retag these tracks that the operation is done in a single step. ----- [snip] ---- The program cannot know what you did with it, and hence it will not alter the IDs in auto-playlists (and collections with the similar criteria).

Yes I understand what happens at the database level.

But the user did just one transaction, ie select all tracks from an album, and retag them all as a group.

If the MM Devs cared as much about the sanctity of a playlist as you do, they could have detected that all tracks for the album were about to be modified .. they could have done an album rename in these situations, so that a the album was not a assigned a new album ID. ...

Same thing when an albumartist tag is bulk changed, or a new albumartist tag added. The fact that this operation reassigns a new album ID, and breaks auto playlists, seems even less excusable to me .... And I am convinced that neither of these things would be expected or understood by the majority of Users.

BTW this may be what the new json data is for.. ?. ... Maybe they are planning a SQL Delete trigger to repair auto playlists that become broken like this??

ZvezdanD wrote: ↑Sat Nov 26, 2022 6:41 am If you think that such behavior is wrong, you should report it as a bug to the MM developers.

They can see this thread.
I am not going to report it, because as already discussed I don't hold auto playlists as being too important, and there is a lot of other things that I would rank ahead of this.

ZvezdanD wrote: ↑Sat Nov 26, 2022 6:41 am However, I would do the next test. What happens if you change an album name in the Albums node of the Media Tree, instead of selecting all tracks from it and using the Properties dialog to retag the tracks? Maybe I will test it latter.

There is no need to do that test. As I have said twice already in this thread, a rename from the Album or Artist Detail panel doesn't cause a problem.

by **ZvezdanD** » Sat Nov 26, 2022 6:41 am

A simple migration facility using JSON data wouldn't be better at all if it gives wrong data. You cannot get correct data without using IDs. Would you be OK if you get something that you don't expect?

The behavior that you are describing is intended, the program behaves like that since the beginning. You would think that if you select all tracks from an album and if you bulk retag these tracks that the operation is done in a single step, but in fact you will get executed a series of operations. For example, when you select all tracks from "The Dark Side of the Moon" and change its name to 'Dark Side of the Moon'" in the Properties dialog box, the program will first add a new album by that new name with the new ID, then:
- it will reassign the first track from that old album to the new one -> the old album will have one track less, but it will still contain the remaining tracks;
- then it will reassign the second track from the old album to the new one -> the old album will have two tracks less, but it will still contain the remaining tracks;
- and so on, until the last track is reassigned and the old album is left without any track. I don't know what happens in that case and cannot test it right now, but I expect that the program removes the old album record from the Albums table. The program cannot know what you did with it, and hence it will not alter the IDs in auto-playlists (and collections with the similar criteria). If you think that such behavior is wrong, you should report it as a bug to the MM developers.

However, I would do the next test. What happens if you change an album name in the Albums node of the Media Tree, instead of selecting all tracks from it and using the Properties dialog to retag the tracks? Maybe I will test it latter.

by **Barry4679** » Fri Nov 25, 2022 10:10 pm

ZvezdanD wrote: ↑Fri Nov 25, 2022 11:04 amI am not sure that understand your mentioning of "break playlist criteria links when a user does some simple retagging" and "the JSON data in the source MM database would still be correct, but the ID has been broken". Could you please give a more detailed explanation or steps to reproduce?

I explained this before.

My general observation is that you are taking great care to migrate something where MM5 itself does a poor job to keep valid ... ie. you are somewhat impacted by Garbage In == Garbage Out, admittedly only in some corner cases. ... This is a comment about MM5, not what you are planning.

And, ironically, in at least one case you would have got a more accurate result if you had used the JSOn fields instead of the (potentially broken) IDs

Steps below.

But first, I am not trying to diss what you are planning to do (or ? have already done?). I am just making the general points

that a simple migration facility using JSON data would be a lot better than having no migration facility

and that a "simple migration facility would OK for my own personal Use Case

is OK to stop talking about this now? I get it that you don't agree

Here are some steps showing how MM5 fails to protect autoplaylists. ... these are just my observations ... maybe there is some corrective mechanism using the JSON data, but I Haven't been able to trigger it, so I doubt that it exists.

create autoplaylist where criteria is Album - "The Dark Side of the Moon" ... select the album from the list provided by the MM5 autoplaylist wizard
then (later) select all tracks in the album, and retag to change, or add, an albumartist tag ... eg. add Roger Waters ==> MM5 it creates a new row in the albums table, with a new IDAlbum
MM5 does not alter the autoplaylist ... the playlist is now broken
ironicaly the JSON field is still accurate, so in this case at least you would have got a function playlist if you migrated using the JSON field

Or create autoplaylist where criteria is Album - "The Dark Side of the Moon" ... select the album from the list provided by the MM5 autoplaylist wizard
then (later) select all tracks in the album, and retag to change the album name to 'Dark Side of the Moon'" ==> MM5 doesn't just alter the album name .. it creates a new row in the albums table, with a new IDAlbum
MM5 does not alter the autoplaylist ... the playlist is now broken ... ie. is not protected by the fact that it used IDs
in this case you would no better off if you used the JSON data
nb. MM5 does reuse the AlbumID if the retagging is made from the Album detail panel, .. but not done from the main panels ... so this is "correct" way to do this retagging, but who knows that??

And the same thing happens when artist names are altered ==> new ID ID ==> autoplaylists are potentially broken.
My guess is that this is just a sub-set of situations where autoplaylists appear to become trashed.

And then there is the case of a DB REBUILD (ie. using the new MM5 DB Management option) ... that reassigns all album IDs, and probably other IDs also ... I haven't tested it, but I wonder if autoplaylists that are based on those Ids are protected?

by **ZvezdanD** » Fri Nov 25, 2022 11:04 am

Again, this is not a "higher goal", it is a necessary goal if you want to get reliable data. If you don't use correct IDs, you will get mismatched data. The JSON field cannot help with that, it is just an additional complication for the add-on.

As I said before, the correct synchronization of data "is already implemented with my add-on. It could find the corresponding IDs just fine, even if you want to export/import data from/to non-related databases." The new version should just additionally synchronize JSON field (most probably only these rules in records that contain IDs, like the mentioned "Playlist") and that is it.

I am not sure that understand your mentioning of "break playlist criteria links when a user does some simple retagging" and "the JSON data in the source MM database would still be correct, but the ID has been broken". Could you please give a more detailed explanation or steps to reproduce?

by **Barry4679** » Fri Nov 25, 2022 6:14 am

ZvezdanD wrote: ↑Thu Nov 24, 2022 6:17 am This is not about "adequate" or "superior". It is about "functioning" and "reliability". To be able to export/import auto-playlists, the IDs in QueryData field should be synchronized between the backup and current databases, as it is already implemented in MM4 version. If that part is not done correctly, or even skipped, the data will not be synchronized, so you could get e.g. "Eminem" instead of "Pink Floyd", as my experiment proved.

Using the JSON field is useless in that part, it just complicates things. And I don't want to leave that field non-synchronized or even empty because that could cause who knows what kind of trouble. It was added in MM5 with a reason and I am sure that the program is using it for something.

I am just reporting about what I would be needing or wanting from a playlist import utility. You have a different higher goal in mind, so that is good. I presume that we see and use playlists differently, and place a different degree of importance upon them.

It is good that you aim to take more care of playlists than MM does itself. ... ie. as reported in my earlier post, they are OK to break playlist criteria links when a user does some simple retagging, despite the fact that the playlists are ID based.

Also it is ironic that your meticulous approach presumably would have a worse outcome where a user has retagged a track to add or change an albumartist tag. ... ie. the JSON data in the source MM database would still be correct, but the ID has been broken ... that's what I see.

by **ZvezdanD** » Thu Nov 24, 2022 6:17 am

This is not about "adequate" or "superior". It is about "functioning" and "reliability". To be able to export/import auto-playlists, the IDs in QueryData field should be synchronized between the backup and current databases, as it is already implemented in MM4 version. If that part is not done correctly, or even skipped, the data will not be synchronized, so you could get e.g. "Eminem" instead of "Pink Floyd", as my experiment proved.

Using the JSON field is useless in that part, it just complicates things. And I don't want to leave that field non-synchronized or even empty because that could cause who knows what kind of trouble. It was added in MM5 with a reason and I am sure that the program is using it for something.

by **Barry4679** » Thu Nov 24, 2022 1:00 am

ZvezdanD wrote: ↑Wed Nov 23, 2022 10:50 am The JSON data would not "greatly ease the task of migration", it is opposite. The MM5 version of the add-on now should synchronize data not only in one field, but in two. It would be even more complicated than before.

There is probably no need to recreate the JSON data. Probably you could leave it empty. Nothing seems to use it when running, or altering an auto playlist.

I am not in a position to argue with you, because you are the expert here.
I know that you are thinking of a "superior" migration utility, while I am thinking of just an "adequate" utility.
"Adequate" in the sense that it would do a good enough job, although it had trouble where the user had made changes to tags referenced in criteria clauses.
I still think that the new JSON field enables creation of an less complex "adequate" migration facility, if there were use for such a thing.

But this is whole bag of worms anyway. ...ie. using IDs only gives partial protection.
eg. if I select an album, and retag the artist name, this creates an new row in the artist table, with a new ID ===> auto playlist is broken. The user needs to know that they can't do that, they need to go to the detail panel for the artist, and do the rename there.

If I retag the albumartist for all tracks for an album, it creates a new Album ID ====> auto playlists are broken here,

Also if the user does a database rebuild (new in MM5), it re-assigns all album IDs, so maybe this ==> breaks all auto playlists containing album equals criteria ... I am guessing here ... I tested the above cases, but not this.

by **ZvezdanD** » Wed Nov 23, 2022 10:50 am

The JSON data would not "greatly ease the task of migration", it is opposite. The MM5 version of the add-on now should synchronize data not only in one field, but in two. It would be even more complicated than before.

by **Barry4679** » Wed Nov 23, 2022 5:35 am

ZvezdanD wrote: ↑Wed Nov 23, 2022 3:07 am Besides, how would you export/import criterion like: "Playlist" "is" "something"? Did you look at its JSON representation? In that case it is like this: {"field":"","operator":"=","value":"7"} where the "7" is the ID of the specified playlist. And who knows what else is still not represented in JSON directly, but by IDs.

I didn't try yet, but I suspect that it is not enough to just export/import the QueryDataJSON field for auto-playlists to work correctly. I suppose that you need to have exported/imported the QueryData field as well with all its corresponding IDs.

I only reviewed it until I got bored. I didn't test it exhaustively.
I didn't test "playlist" inclusion criteria, and as you say, I now see that that does contain an ID.
I noticed what I was hoping to see, in the QueryDataJSON column, and didn't look too closely at anything else.

ZvezdanD wrote: ↑Wed Nov 23, 2022 3:07 am EDIT: I just tried a very simple test. I replaced the artist name in JSON field with another artist name using an external SQLite program with closed MM, without changing anything to the QueryData field, and when I started MM it showed tracks from the old artist. So, it seems that MM doesn't use QueryDataJSON field for its display at all, and that IDs are still the only way how auto-playlists work.

Even if MM5 cannot run the queries using JSON data, that data would greatly ease the task of migration IMO ... unless someone, like you, already had all the code required to rebuild everything by comparing paths, and deducing IDs for the new target database, etc etc

I would guess that they could alter the JSON field to use playlist title instead of its ID. ... It seems to have a bug there that could be fixed anyway, because that field name is blank.

I suppose a migration using JSON data method would have an issue in databases where there was many tag duplicates ... ie. multiple tracks which have the same tag value, where that tag value is used as a selection criteria ... I wouldn't think that would be a show stopper though.

by **Peke** » Wed Nov 23, 2022 4:01 am

Hi,
Zvezdan is right here we often use IDs from DB to ensure reference to specific field of search/criteria in Auto-Playlist as like Zvezdan pointed. One of reasons is that you can rename Playlist and reference in other playlist would be broken or MM would need to go thru all other playlists and change it. Which would be performance hit, while ID stays same.

by **ZvezdanD** » Wed Nov 23, 2022 3:07 am

Yes, I am thinking about MM4 version of the database. MM5 has added the QueryDataJSON field that you are talking about, which didn't exist before, but if you look at the QueryData field you will see that the conditions still use IDs instead of real values. Something like this:
DBField=Artists.Artist
Condition=301
Value=943

Besides, how would you export/import criterion like: "Playlist" "is" "something"? Did you look at its JSON representation? In that case it is like this: {"field":"","operator":"=","value":"7"} where the "7" is the ID of the specified playlist. And who knows what else is still not represented in JSON directly, but by IDs.

I didn't try yet, but I suspect that it is not enough to just export/import the QueryDataJSON field for auto-playlists to work correctly. I suppose that you need to have exported/imported the QueryData field as well with all its corresponding IDs.

EDIT: I just tried a very simple test. I replaced the artist name in JSON field with another artist name using an external SQLite program with closed MM, without changing anything to the QueryData field, and when I started MM it showed tracks from the old artist. So, it seems that MM doesn't use QueryDataJSON field for its display at all, and that IDs are still the only way how auto-playlists work.

by **Barry4679** » Tue Nov 22, 2022 9:04 pm

ZvezdanD wrote: ↑Tue Nov 22, 2022 10:26 am No, the export/import of auto-playlists is not that simple as you think and its complexity is not overstated. That is because the rules for auto-playlists are in almost all cases related to some kind of ID that is specific to the particular database. Not a database ID, but a data ID.

For example, if I have an auto-playlist with a criterion: "Artist" "is" "Pink Floyd", that criterion is not stored in the database as such.

As I said originally I may be over simplifying things, but I did try before making my original post, to force an autoplaylist to be dependent upon row ids. I still don't see any evidence of this. Not even in the example you quote.

Maybe you are thinking of MM4? Maybe this has changed in MM5?

Example this query is stored in the MM5 db as shown below. Pink Floyd was selected from the autplaylist's dropdown list, ie. I did not type the name as text:

Code: Select all

{"conditions":{"autoSortString":"","data":[{[b]"field":"genre","operator":"=","value":"Classic Rock"[/b]}, {[b]"field":"albumArtist","operator":"=","value":"Pink Floyd"[/b]}, {"field":"lastPlayed","operator":"> (days ago)","value":"7"}]},"limit":"files","limitValue":50,"limitOrder":"sort order","matchAllCriteria":true,"sortOrders":[{"field":"title","ascending":true}],"collection":"Entire Library"}

The relevant part is {"field":"albumArtist","operator":"=","value":"Pink Floyd"} .. ie. no IDs which are specific to the database where it was created. And the difficulty of exporting this, appears to me, to be overstated by people like LowLander and Peke.

ZvezdanD wrote: ↑Tue Nov 22, 2022 10:26 am As I said several times already, my Restore/Synchronize Database add-on could export/import auto-playlists in MM4 (and static playlists as well) and its port for MM5 is planned.

Who knows what anyone says here, or what they actually replying to?
It is so confusing when LowLander jerks everybody around, by relocating posts into other threads, and or other forums.
I know that in many cases it helps discoverability, but he overdoes it sometimes, which really pisses me off TBH

Rob_S wrote: ↑Tue Nov 22, 2022 7:18 pm As to addon vs core, I don't care too much who supplies it, but it is not something I think I should pay extra for, and sadly for now even that option does not exist.

I have a view on this.
I think it is to everybody's' benefit if there is a healthy active community of developers are working to extend MediaMonkey.
I don't think that anyone would expect to be paid for small utilities, that someone made for their own benefit, and were happy to share.

But I think that it is OK, and is important, to pay for addons containing significant amounts of work and skill, which are well supported. ZvezdanD is a clear example of this.

Then there is question about whether something is "core".
I have a very long list of disappointments about what did not make its way the "core" of the new generation of MM5.
I am not convinced that export|import of auto playlists is one of those cases. I am OK for things like that to be add-ons.

Once someone has spent the effort to make an add-on, and it has become successful, and the author is serving customers well ... then Ventis should be careful to treat that Developer well.

As far as I can see, they are largely respectful of this.

by **Rob_S** » Tue Nov 22, 2022 7:18 pm

Pardon me, you are right, that bit of clarification was added by someone else. But I suspect was intended by OP , as the import of regular playlists appears to be flawless and reliable.

As to addon vs core, I don't care too much who supplies it, but it is not something I think I should pay extra for, and sadly for now even that option does not exist.

by **Lowlander** » Tue Nov 22, 2022 7:04 pm

As mentioned above by Rusty, there is no plan for a MediaMonkey native solution to transferring AutoPlaylists as AutoPlaylists to a different database. This is being left up to an Addon to accomplish this.

[REQ] backup and restore auto-playlists

Post a reply

Expand view Topic review: [REQ] backup and restore auto-playlists

Re: [REQ] backup and restore auto-playlists

Re: [REQ] backup and restore auto-playlists

Re: [REQ] backup and restore auto-playlists

Re: [REQ] backup and restore auto-playlists

Re: [REQ] backup and restore auto-playlists

Re: [REQ] backup and restore auto-playlists

Re: [REQ] backup and restore auto-playlists

Re: [REQ] backup and restore auto-playlists

Re: [REQ] backup and restore auto-playlists

Re: [REQ] backup and restore auto-playlists

Re: [REQ] backup and restore auto-playlists

Re: [REQ] backup and restore auto-playlists

Re: [REQ] backup and restore auto-playlists

Re: [REQ] backup and restore auto-playlists

Re: [REQ] backup and restore auto-playlists