We need a 'Sync-Cache' or a 'Shadow-Library' [#7636]

Any ideas about how to improve MediaMonkey? Let us know!

Moderator: Gurus

I like to be able to maintain copies of my music managed by MediaMoney

You may select 1 option

 
 
View results

wolfzell
Posts: 155
Joined: Fri Apr 23, 2004 8:42 am
Location: Germany
Contact:

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

Post by wolfzell »

DWSR wrote:How are you going to address the problem of the second library growing and taking up a large amount of disk space?
Simple: I do not see this as a problem at all.

192kBit/s MP3-versions of FLAC files need about 25% of the space of the originals. So for example with my collection of 20.000 FLAC files that need 400GB, my manually managed MP3-copy of the library that I am using for syncing right now needs 87GB. I do not consider this as a "large amount" in relation to the original disk space. Considering the cost of hard disk space I really do not understand where the problem is. I'd rather spend another 100$ on a new disk once than wait 1-2 days for my 32GB-MP3-player to sync everytime I want to do it.

Not to start talking of the necessity to have the PC running for 2 days, wasting energy. Or what if the PC encounters a problem in these 48 hours? Will I have to start the sync again? And when will I be able to actually use my MP3-player if it is connected to the PC, syncing all the time? :)

bye
Wolfgang
nohitter151
Posts: 23640
Joined: Wed Aug 09, 2006 10:20 am
Location: NJ, USA
Contact:

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

Post by nohitter151 »

Doesn't this script basically fulfill your needs?
MediaMonkey user since 2006
Need help? Got a suggestion? Can't find something?

Please no PMs in reply to a post. Just reply in the thread.
VRN
Posts: 26
Joined: Wed Apr 09, 2008 11:40 am
Contact:

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

Post by VRN »

Hi,

As I see it there are several ways of addressing the problem at hand. One may favour one solution over another depending on whether money, time, storage, network or processor speed is(are) the limited resource(s).

For my part, I have enough disc space to host both the "originals" in FLAC and irreversibly compressed copies thereof (MP3s). What I don't have a lot of is time and processing power. Accordingly, I favour having a "master library" in FLAC and an identical "slave library" in MP3. The "master library", I would use to make changes (additions, deletions, tag amendments, cover art etc.) to my FLAC files. These changes should be automatically reflected in the MP3s and the "slave library" I would use for sync'ing to mobile devices. Now the advantage of having the "slave library" addressing a collection of physical MP3-files from a "cache" or "shadow" rather than the FLACs would be that sync'ing to a new device wouldn't require that all FLACs would have to be converted on-the-fly (again). Accordingly, "first-time" sync'ing would be much faster. There might also be occasions where you quickly need an MP3 copy of some of your songs and don't want to wait for conversion.

Moreover, where the lossless version in FLAC feeds my SONOS players in my living room, kitchen etc; the MP3 version could be streamed to cheaper streaming clients (typically not supporting FLAC) in the bath, garage, basement, toilet etc., i.e. everything available everywhere at any time.

Best regards,

>V<
adb
Posts: 64
Joined: Thu Mar 09, 2006 1:46 pm
Contact:

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

Post by adb »

After finally realising why the sound output, streaming MP3 192 vbr, through my EVA8000 is crap (apparently caused by the terrible DAC in the EVA), I realised I needed to upgrade my 1991 amp to go optical... So, I got an Onkyo 805 as the sales guy told me the DAC was good, but I then decided that I ought to upgrade my existing MP3 192 vbr collection, so started looking at FLAC.

I agree with Wolfgang that this concept would be a good idea. I have approx 70GB MP3, so going to FLAC will be about 300GB. As I have 1.3TB NAS, I don't care about having an extra 70GB in these files ready to put on my 160GB iPod - in fact I would then upgrade that shadown library to be 320vbr. The saving of being able sync quickly to the iPod using the MP3s would be great. It's more important to be able to manage tags in a single place and let MM manage the two formats. Maybe it should not be so much as a sync-cache, but a lossless/lossy management by MM.

I'd like the ability to store the shadow on a different path, e.g. currently on my NAS I have \\media\music\MP3 and \\media\music\FLAC, but I'd want to change that to store the MP3 on a different NAS volume so that EVA does not pick up duplicates.

Maintaining the lossy format could be done at sync time, or any time the lossless form is changed, but I'd prefer the latter as it would make backups more consistent knowing that the lossy and lossless formats are in sync.

Antony
Guest

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

Post by Guest »

I really need this feature. Right now managing a dual flac + mp3 library is a real PITA.
VRN
Posts: 26
Joined: Wed Apr 09, 2008 11:40 am
Contact:

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

Post by VRN »

Has anything significantly moved on this topic?

>V<
Lowlander
Posts: 48374
Joined: Sat Sep 06, 2003 5:53 pm
Location: MediaMonkey 5
Contact:

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

Post by Lowlander »

No
Lowlander (MediaMonkey user since 2003)
Lowlander
Posts: 48374
Joined: Sat Sep 06, 2003 5:53 pm
Location: MediaMonkey 5
Contact:

Virtual Copy

Post by Lowlander »

As mentioned there are currently 2 methods available that can serve the purpose in limited situations. You can either use the Virtual CD for managing one copy of the songs. This allows you to maintain a local copy of networked drives or a backup of files while MediaMonkey treats them as one and the same. You can sync to any location on your PC using the USB_Mass plugin for syncing which allows for syncing files while converting the.

The following is a proposal based on this topic and elaborates more on this. It solves many scenarios that different users might have for needing a copy of the original songs. Main concerns that need to be addressed are automation of copying files and maintaining consistent tags across copies. And as such I propose the following:

Split the current Virtual CD functionality into 2 parts:
Virtual CD would be used exclusively for copying content of CD's and maintaining a relationship with the original CD (already implemented).

Virtual Copy would be used to maintain copies of songs in the library (Sync-Cache or Shadow-Library). I propose that the Virtual Copy has the following capabilities:
  • Allow multiple Virtual Copies
  • Rule based auto Virtual Copies
    • Rule to define which tracks are added to the Virtual Copy with ability to select filter and playlists (like sync profile)
    • Rule to define what conversion should be applied to Virtual Copy (like sync profile)
    • Rule to define mask used to store tracks
    • Rule to define max size the Virtual Copy can used (can be absolute max size or by defining how much free space should be left on hard drive)
    • Allow tag switching for Virtual Copy (this has been proposed for the Sync Profile as well and is useful for tag incompatible software/hardware)
    • Custom Analyze Volume level to be stored in tags or auto-apply Level Volume (latter like sync profile)
    • Define rule for deletion or prompt on deletion what to do with Virtual Copies
  • Original and Virtual Copies will be presented like one and the same (like Virtual CD)
  • Auto-Maintain tags synced across original/copies and option to manually sync tags to all copies (apply tag switch rules)
  • Auto-Copy to Virtual Copy (like Auto-Organize)
  • Scheduled copy to Virtual Copy (like backup software)
  • Allow Virtual Copy locations to be offline (currently MediaMonkey will complain if Virtual CD location is inaccessible)
Playback
Add option to define which copy (original/Virtual Copy) should be used for playback or even better a preference order that MediaMonkey uses until it finds an accessible copy.

Sync
  • Add option to define Virtual Copy as tracks to be synced.
  • Add intelligent auto-selection of original/Virtual Copy to prevent unnecessary conversion.
  • Add manual selection of which original/Virtual Copy should be used as source files for sync
Auto-Playlist/Advanced Search
  • Add option to define location (original/Virtual Copy) as rule (not to be confused with path)
  • Add preference order (original/Virtual Copy) for use in combination with status = available.
Benefits
  • Allows to maintain library on network and maintain local copy for when not on network
  • Allow to keep a cache for files used to sync so conversion is not required when syncing, which is useful for those who often change which tracks are synced
  • Allows to keep a copy of library for those devices on the network that don't support certain file types, have problems with tags or require lower bitrate for smooth playback
  • Maintain backups from within MediaMonkey (only protects against drive failures if copy is on different drive and doesn't protect against mistakes made inside MediaMonkey).
  • and others
I think that this covers the concerns that have been raised in this topic related to keeping copies of songs and automates the process so it's seamless to the user, what do you think?
Lowlander (MediaMonkey user since 2003)
MoDementia
Posts: 1321
Joined: Thu Jun 15, 2006 3:26 pm
Location: Geelong, Victoria, Australia
Contact:

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

Post by MoDementia »

I do think the best option is to "duplicate" the Virtual CD environment for this task too

A couple of points I find with my script:

I don't have a completely lossless library so whether these still need to be added to the virtual copy without conversion or the sync can work backwards as well as forwards
i.e. I don't have a sync version so use the original and apply any conversion rules.
where as playback is I have the original online, play that.

I sync a playlist(s) so these need to obey the play/sync rules too?

currently my script requires 2 sets of playlists, 1 play, 1 sync
Lowlander
Posts: 48374
Joined: Sat Sep 06, 2003 5:53 pm
Location: MediaMonkey 5
Contact:

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

Post by Lowlander »

The sync would be designed to be intelligent that it would choose that copy that matches the conversion rules so no unnecessary conversion is applied. The ability to use a Virtual Copy as sync base would not sync those not in the Virtual Copy, but this would be just an option for defining which tracks should be synced.

The ability to define player rules would probably not require to use 2 separate playlists. At least as I envision it the playlist would contain tracks in your library (not a specific copy although rules allowing for this should be added to auto-playlists). The player rules would allow you to define that MediaMonkey should use the best available copy (highest bitrate) should be used whereas the sync conversion rules could define another copy to be used.

As you see this is a fairly extensive proposal that would require quite a bit of work, testing and user feedback, but if it is implemented would be a great addition to MediaMonkey and probably be something that sets it apart from other media software.
Lowlander (MediaMonkey user since 2003)
bobthegoat2001
Posts: 31
Joined: Thu Sep 20, 2007 12:28 am
Contact:

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

Post by bobthegoat2001 »

I think the main use of Media Monkey is meant to sync your music to your portable (at least for me). So this option would really help me out. To me this is an obvious update and I'm surprised it hasn't been implemented yet.
DWSR wrote:The big issue for me is #1, however. I believe that a multi-threaded encoding process will drastically reduce the amount of time that it will take to sync PAPs, while providing no increased disk usage.
I have an Intel Quad Core Q6700 and 6gb of ram and it still takes at least 2 days to sync my library to my Archos 5 120gb MP3 player. That's 2 days that I can't use my MP3 player, which I use while at work and that sucks when I can't use it. I think the payoff of time/memory would be way worth it. For it to take minutes vs days to sync is way worth the extra 50gbs of hard drive space it would take to store an extra set of lossy MP3's. Hard drives are dirt cheap these days too, I can get a 1tb (one terabyte!) hard drive for $79, for one that's not on sale.

So please include this feature. It would really, really, really save me time.
Last edited by bobthegoat2001 on Fri Jul 03, 2009 1:24 am, edited 1 time in total.
Lowlander
Posts: 48374
Joined: Sat Sep 06, 2003 5:53 pm
Location: MediaMonkey 5
Contact:

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

Post by Lowlander »

MediaMonkey 3.1 has multi-core support, it should be faster.
Lowlander (MediaMonkey user since 2003)
bobthegoat2001
Posts: 31
Joined: Thu Sep 20, 2007 12:28 am
Contact:

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

Post by bobthegoat2001 »

That's good, and I'm sure that'll help. But I still stand beside what I said. Even though that will help the speed, I think it would still be faster if it were to directly copy the already converted MP3's to the player.
phantomwhale
Posts: 16
Joined: Thu Jul 03, 2008 3:36 am
Contact:

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

Post by phantomwhale »

Wholly agree - currently I am trying to manage both Lossless and Compressed formats in one library, and it come with a lot of painful syncing of metadata and non-transferable playlists etc... (see http://www.mediamonkey.com/forum/viewto ... =1&t=40932)

It seems that the auto-conversion option would take away all the trickiness of this, but at the price of having to encode / re-encode every time I copy a rack onto my iPod (or other removeable device). As I change the content of my devices quite often, the price of all this encoding is too high, esp. compared to using a small chunk of drive space to cache the encoded tracks.

I would love to see a basic version of this cache implemented somehow.
Lowlander
Posts: 48374
Joined: Sat Sep 06, 2003 5:53 pm
Location: MediaMonkey 5
Contact:

Re: We need a 'Sync-Cache' or a 'Shadow-Library'

Post by Lowlander »

Tag syncing between copies of the same track can be done with the Advanced Duplicate Find & Fix script.

And yes I do fully support this request!
Lowlander (MediaMonkey user since 2003)
Post Reply