Only tiny percentage of songs read into collection

Report bugs & feature requests for MediaMonkey Server and learn about the newest builds.

Moderators: drakinite, Gurus

sparhawk
Posts: 10
Joined: Tue Sep 06, 2011 3:35 pm

Re: Only tiny percentage of songs read into collection

Post by sparhawk »

I've restarted MediaMonkeyServer multiple times to try to get the count up, as each time I restart the service, it will sometimes find more songs before halting. Here's the comparison between MMS and MM5.

MMS
Feb 24 21:37:10 mms mm-server[664]: 131312 files have been read from database in 10553 ms
Feb 24 21:37:10 mms mm-server[664]: Scan music directory: file:/mnt/music/Alt, Folk, Pop 1, 46059 files to process
Feb 24 21:37:55 mms mm-server[664]: Scan of repository /mnt/music/Alt, Folk, Pop 1 has been finished in 56.28 seconds
Feb 24 21:37:55 mms mm-server[664]: Scan music directory: file:/mnt/music/Alt, Folk, Pop 2, 11842 files to process
Feb 24 21:38:21 mms mm-server[664]: Scan of repository /mnt/music/Alt, Folk, Pop 2 has been finished in 25.40 seconds
Feb 24 21:38:21 mms mm-server[664]: Scan music directory: file:/mnt/music/Rock 1, 4741 files to process
Feb 24 21:38:22 mms mm-server[664]: Scan of repository /mnt/music/Rock 1 has been finished in 1.41 seconds
Feb 24 21:38:22 mms mm-server[664]: Scan music directory: file:/mnt/music/Rock 2, 49709 files to process
Feb 24 21:39:15 mms mm-server[664]: Scan of repository /mnt/music/Rock 2 has been finished in 52.55 seconds
Feb 24 21:39:15 mms mm-server[664]: Scan music directory: file:/mnt/music/Industrial 1, 7912 files to process
Feb 24 21:39:18 mms mm-server[664]: Scan of repository /mnt/music/Industrial 1 has been finished in 3.46 seconds
Feb 24 21:39:18 mms mm-server[664]: Scan music directory: file:/mnt/music/Industrial 2, 11049 files to process
Feb 24 21:39:28 mms mm-server[664]: Scan of repository /mnt/music/Industrial 2 has been finished in 9.86 seconds

MM5
Alt, Folk, Pop 1 -> 72,644
Alt, Folk, Pop 2 -> 55,036
Rock 1 -> 61,173
Rock 2 -> 53,430
Industrial 1 -> 13,113
Industrial 2 -> 11,049 (hey, this one matches!!!!)

All I can see from the logs is that it fails on MPC files, which is fine. But it might also be causing it to halt? Hard to tell. If you want me to attach logs again, just let me know which one(s).
Ludek
Posts: 4610
Joined: Fri Mar 09, 2007 9:00 am

Re: Only tiny percentage of songs read into collection

Post by Ludek »

Hi,
good catch with the MPC files!

I am seeing the hung with MPC too, looking into it...
Ludek
Posts: 4610
Joined: Fri Mar 09, 2007 9:00 am

Re: Only tiny percentage of songs read into collection

Post by Ludek »

Re the MPC: Actually seeing that I have already reported it here 4 years ago:
https://github.com/Borewit/music-metadata/issues/165

See the last comments where:
1) I'have just found that two files were actually corrupted (03, 06) having each 1.8 GB in size
2) it still takes 57 seconds to scan just the 10 files , so still quite slow
Reply from Borewit: As I explained, the the Musepack, stream-version 7 parsing indeed very inefficient.

Maybe you could also comment there with your MPC samples?
sparhawk
Posts: 10
Joined: Tue Sep 06, 2011 3:35 pm

Re: Only tiny percentage of songs read into collection

Post by sparhawk »

Any way to force a collection's rescan again to get those MPC errors? The only time they appeared was during initial scan. Restarting the server or choosing to rescan the collection from the web interface doesn't do it.

What I want to send you is the last MPC file from each scan before it stopped adding files, as that is probably the culprit.

I suppose if I have to, I can delete the collection and re-add it.
Ludek
Posts: 4610
Joined: Fri Mar 09, 2007 9:00 am

Re: Only tiny percentage of songs read into collection

Post by Ludek »

I think that currently only the new files (not in the database yet) are scanned during re-scan, this is being tracked as https://github.com/mediamonkeyserver/mms/issues/38

Until this is fixed then test it by moving the MPC files to a new folder and add this testing folder to a testing collection to see which MPC makes the issue.
sparhawk
Posts: 10
Joined: Tue Sep 06, 2011 3:35 pm

Re: Only tiny percentage of songs read into collection

Post by sparhawk »

I'm seeing two different types of errors with MPC files

Code: Select all

Music metadata parsing error: Error: Unexpected data-type: 3 (/mnt/music/Rock 1/Alan Davey/[2001] Al Chemical's Lysergic Orchestra/Alan Davey - Al Chemical's Lysergic Orchestra - 07 Synthia.mpc)

Code: Select all

Music metadata parsing error: RangeError [ERR_INVALID_OPT_VALUE]: The value "4098121093" is invalid for option "size" (/mnt/music/Rock 1/Amen/[1999] Amen/Amen - Amen - 03 Drive.mpc)
Here are three MPC files that appear to stop the scanning process for me:
https://www.dropbox.com/s/vhskudvs1pgn4k6/mpc.zip?dl=0
Ludek
Posts: 4610
Joined: Fri Mar 09, 2007 9:00 am

Re: Only tiny percentage of songs read into collection

Post by Ludek »

Thanks, added it as https://github.com/Borewit/music-metadata/issues/1041
for Borewit to analyze.
allwi
Posts: 67
Joined: Sun Jan 22, 2012 7:34 pm
Location: Sydney NSW

Re: Only tiny percentage of songs read into collection

Post by allwi »

Is there any process on this? I was hoping that a new version on a different platform (Linux) may solve the issue but when I installed on the weekend MMS from the source, it showed v 0.43 and worse, the scanning only discovered 2,046 of my 20k files :(
drakinite
Posts: 792
Joined: Tue May 12, 2020 10:06 am
Contact:

Re: Only tiny percentage of songs read into collection

Post by drakinite »

allwi wrote: Sun May 01, 2022 9:35 pm Is there any process on this? I was hoping that a new version on a different platform (Linux) may solve the issue but when I installed on the weekend MMS from the source, it showed v 0.43 and worse, the scanning only discovered 2,046 of my 20k files :(
Looks like Borewit fixed it, but we haven't updated the version of music-metadata on the repo. Go to package.json (in the MMS file root) and change "music-metadata": "^7.11.7", to "music-metadata": "^7.12.3", - then do npm install again and see how it works. :slight_smile:

side note: I wish phpbb had inline code blocks. :pensive:
Image
Student electrical-computer engineer, web programmer, part-time MediaMonkey developer, full-time MediaMonkey enthusiast
Post Reply