We need a 'Sync-Cache' or a 'Shadow-Library' [#7636]

Post a reply

:D :) :( :o :-? 8) :lol: :x :P :oops: :cry: :evil: :roll: :wink:

BBCode is ON
[img] is ON
[flash] is OFF
[url] is ON
Smilies are ON

Topic review

Expand view Topic review: We need a 'Sync-Cache' or a 'Shadow-Library' [#7636]

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

by Peke » Wed Apr 06, 2011 4:48 pm

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

by bobthegoat2001 » Sun Mar 27, 2011 9:56 pm

I haven't heard anything from this topic in a while? Are their still any plans in implementing this?

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

by Lowlander » Thu Jul 02, 2009 11:34 am

Tag syncing between copies of the same track can be done with the Advanced Duplicate Find & Fix script.

And yes I do fully support this request!

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

by phantomwhale » Thu Jul 02, 2009 8:39 am

Wholly agree - currently I am trying to manage both Lossless and Compressed formats in one library, and it come with a lot of painful syncing of metadata and non-transferable playlists etc... (see http://www.mediamonkey.com/forum/viewto ... =1&t=40932)

It seems that the auto-conversion option would take away all the trickiness of this, but at the price of having to encode / re-encode every time I copy a rack onto my iPod (or other removeable device). As I change the content of my devices quite often, the price of all this encoding is too high, esp. compared to using a small chunk of drive space to cache the encoded tracks.

I would love to see a basic version of this cache implemented somehow.

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

by bobthegoat2001 » Thu Jul 02, 2009 6:28 am

That's good, and I'm sure that'll help. But I still stand beside what I said. Even though that will help the speed, I think it would still be faster if it were to directly copy the already converted MP3's to the player.

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

by Lowlander » Wed Jul 01, 2009 10:50 pm

MediaMonkey 3.1 has multi-core support, it should be faster.

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

by bobthegoat2001 » Wed Jul 01, 2009 10:42 pm

I think the main use of Media Monkey is meant to sync your music to your portable (at least for me). So this option would really help me out. To me this is an obvious update and I'm surprised it hasn't been implemented yet.
DWSR wrote:The big issue for me is #1, however. I believe that a multi-threaded encoding process will drastically reduce the amount of time that it will take to sync PAPs, while providing no increased disk usage.
I have an Intel Quad Core Q6700 and 6gb of ram and it still takes at least 2 days to sync my library to my Archos 5 120gb MP3 player. That's 2 days that I can't use my MP3 player, which I use while at work and that sucks when I can't use it. I think the payoff of time/memory would be way worth it. For it to take minutes vs days to sync is way worth the extra 50gbs of hard drive space it would take to store an extra set of lossy MP3's. Hard drives are dirt cheap these days too, I can get a 1tb (one terabyte!) hard drive for $79, for one that's not on sale.

So please include this feature. It would really, really, really save me time.

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

by Lowlander » Fri Apr 24, 2009 8:54 am

The sync would be designed to be intelligent that it would choose that copy that matches the conversion rules so no unnecessary conversion is applied. The ability to use a Virtual Copy as sync base would not sync those not in the Virtual Copy, but this would be just an option for defining which tracks should be synced.

The ability to define player rules would probably not require to use 2 separate playlists. At least as I envision it the playlist would contain tracks in your library (not a specific copy although rules allowing for this should be added to auto-playlists). The player rules would allow you to define that MediaMonkey should use the best available copy (highest bitrate) should be used whereas the sync conversion rules could define another copy to be used.

As you see this is a fairly extensive proposal that would require quite a bit of work, testing and user feedback, but if it is implemented would be a great addition to MediaMonkey and probably be something that sets it apart from other media software.

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

by MoDementia » Fri Apr 24, 2009 3:17 am

I do think the best option is to "duplicate" the Virtual CD environment for this task too

A couple of points I find with my script:

I don't have a completely lossless library so whether these still need to be added to the virtual copy without conversion or the sync can work backwards as well as forwards
i.e. I don't have a sync version so use the original and apply any conversion rules.
where as playback is I have the original online, play that.

I sync a playlist(s) so these need to obey the play/sync rules too?

currently my script requires 2 sets of playlists, 1 play, 1 sync

Virtual Copy

by Lowlander » Wed Apr 22, 2009 10:22 am

As mentioned there are currently 2 methods available that can serve the purpose in limited situations. You can either use the Virtual CD for managing one copy of the songs. This allows you to maintain a local copy of networked drives or a backup of files while MediaMonkey treats them as one and the same. You can sync to any location on your PC using the USB_Mass plugin for syncing which allows for syncing files while converting the.

The following is a proposal based on this topic and elaborates more on this. It solves many scenarios that different users might have for needing a copy of the original songs. Main concerns that need to be addressed are automation of copying files and maintaining consistent tags across copies. And as such I propose the following:

Split the current Virtual CD functionality into 2 parts:
Virtual CD would be used exclusively for copying content of CD's and maintaining a relationship with the original CD (already implemented).

Virtual Copy would be used to maintain copies of songs in the library (Sync-Cache or Shadow-Library). I propose that the Virtual Copy has the following capabilities:
  • Allow multiple Virtual Copies
  • Rule based auto Virtual Copies
    • Rule to define which tracks are added to the Virtual Copy with ability to select filter and playlists (like sync profile)
    • Rule to define what conversion should be applied to Virtual Copy (like sync profile)
    • Rule to define mask used to store tracks
    • Rule to define max size the Virtual Copy can used (can be absolute max size or by defining how much free space should be left on hard drive)
    • Allow tag switching for Virtual Copy (this has been proposed for the Sync Profile as well and is useful for tag incompatible software/hardware)
    • Custom Analyze Volume level to be stored in tags or auto-apply Level Volume (latter like sync profile)
    • Define rule for deletion or prompt on deletion what to do with Virtual Copies
  • Original and Virtual Copies will be presented like one and the same (like Virtual CD)
  • Auto-Maintain tags synced across original/copies and option to manually sync tags to all copies (apply tag switch rules)
  • Auto-Copy to Virtual Copy (like Auto-Organize)
  • Scheduled copy to Virtual Copy (like backup software)
  • Allow Virtual Copy locations to be offline (currently MediaMonkey will complain if Virtual CD location is inaccessible)
Add option to define which copy (original/Virtual Copy) should be used for playback or even better a preference order that MediaMonkey uses until it finds an accessible copy.

  • Add option to define Virtual Copy as tracks to be synced.
  • Add intelligent auto-selection of original/Virtual Copy to prevent unnecessary conversion.
  • Add manual selection of which original/Virtual Copy should be used as source files for sync
Auto-Playlist/Advanced Search
  • Add option to define location (original/Virtual Copy) as rule (not to be confused with path)
  • Add preference order (original/Virtual Copy) for use in combination with status = available.
  • Allows to maintain library on network and maintain local copy for when not on network
  • Allow to keep a cache for files used to sync so conversion is not required when syncing, which is useful for those who often change which tracks are synced
  • Allows to keep a copy of library for those devices on the network that don't support certain file types, have problems with tags or require lower bitrate for smooth playback
  • Maintain backups from within MediaMonkey (only protects against drive failures if copy is on different drive and doesn't protect against mistakes made inside MediaMonkey).
  • and others
I think that this covers the concerns that have been raised in this topic related to keeping copies of songs and automates the process so it's seamless to the user, what do you think?

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

by VRN » Tue Apr 21, 2009 4:47 pm

Has anything significantly moved on this topic?


Re: We need a 'Sync-Cache' or a 'Shadow-Library'

by Guest » Fri Apr 10, 2009 2:11 am

I really need this feature. Right now managing a dual flac + mp3 library is a real PITA.

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

by adb » Wed Oct 08, 2008 6:10 pm

After finally realising why the sound output, streaming MP3 192 vbr, through my EVA8000 is crap (apparently caused by the terrible DAC in the EVA), I realised I needed to upgrade my 1991 amp to go optical... So, I got an Onkyo 805 as the sales guy told me the DAC was good, but I then decided that I ought to upgrade my existing MP3 192 vbr collection, so started looking at FLAC.

I agree with Wolfgang that this concept would be a good idea. I have approx 70GB MP3, so going to FLAC will be about 300GB. As I have 1.3TB NAS, I don't care about having an extra 70GB in these files ready to put on my 160GB iPod - in fact I would then upgrade that shadown library to be 320vbr. The saving of being able sync quickly to the iPod using the MP3s would be great. It's more important to be able to manage tags in a single place and let MM manage the two formats. Maybe it should not be so much as a sync-cache, but a lossless/lossy management by MM.

I'd like the ability to store the shadow on a different path, e.g. currently on my NAS I have \\media\music\MP3 and \\media\music\FLAC, but I'd want to change that to store the MP3 on a different NAS volume so that EVA does not pick up duplicates.

Maintaining the lossy format could be done at sync time, or any time the lossless form is changed, but I'd prefer the latter as it would make backups more consistent knowing that the lossy and lossless formats are in sync.


Re: We need a 'Sync-Cache' or a 'Shadow-Library'

by VRN » Tue Sep 16, 2008 3:19 am


As I see it there are several ways of addressing the problem at hand. One may favour one solution over another depending on whether money, time, storage, network or processor speed is(are) the limited resource(s).

For my part, I have enough disc space to host both the "originals" in FLAC and irreversibly compressed copies thereof (MP3s). What I don't have a lot of is time and processing power. Accordingly, I favour having a "master library" in FLAC and an identical "slave library" in MP3. The "master library", I would use to make changes (additions, deletions, tag amendments, cover art etc.) to my FLAC files. These changes should be automatically reflected in the MP3s and the "slave library" I would use for sync'ing to mobile devices. Now the advantage of having the "slave library" addressing a collection of physical MP3-files from a "cache" or "shadow" rather than the FLACs would be that sync'ing to a new device wouldn't require that all FLACs would have to be converted on-the-fly (again). Accordingly, "first-time" sync'ing would be much faster. There might also be occasions where you quickly need an MP3 copy of some of your songs and don't want to wait for conversion.

Moreover, where the lossless version in FLAC feeds my SONOS players in my living room, kitchen etc; the MP3 version could be streamed to cheaper streaming clients (typically not supporting FLAC) in the bath, garage, basement, toilet etc., i.e. everything available everywhere at any time.

Best regards,