MonkeyMatch 0.5.56 - Find & Fix Similar Spelling (7/7/13)

[Scottes]

OK, it's been 5 weeks since I made this available, and I've only gotten one report of anyone using it. And it's not a good report, since it doesn't work at all for him...

Is anyone using this? Successfully or not? Any comments, recommendations, wishes, anything? I'll even take more bug reports...


Also, is anyone willing to make their database available to me for download? That would give me the chance to see if things are working OK, and fix some bugs that I am sure will appear.

[hintergrundrauschen]

Also, is anyone willing to make their database available to me for download? That would give me the chance to see if things are working OK, and fix some bugs that I am sure will appear.

See PM.

[Scottes]

I will be working on your database over the weekend. Thanks!

[zuilserip]

Hi Scottes,

First, thank you again for developing this. This is a hugely helpful script and fills a really important need in terms of allowing us to create consistency for our databases. I am very excited about its potential and would love to help in any way I can (i.e., testing, making my DB available, sharing logs, ideas, etc.)

Unfortunately I have not been able to get the program to work yet. I had installed an earlier version of the program a few weeks ago and was able to run it ok - it found 210 artist matches, and select the preferred alternatives - but was not able to save my choices (i.e., the save button was grayed out). I also tried to use the 'auto save' option but that did not seem to do anything either(I continued finding the same 210 matches afterwards).

Today I downloaded the most recent version of the program and made sure I installed it with admin rights. This version is crashing with a message that 'Index was outside the bounds of the array' message before I even get to the main screen. This sounds similar (the same?) to the other problem reported before. I also have a fair number of foreign characters in my Database (about 88K songs).

Here are the details of the crash:

Error message:
“Unhandled exception has occurred in your application. If you click Continue, the application will ignore this error and attempt to continue. If you click Quit, the application will close immediately.”

Details:

See the end of this message for details on invoking
just-in-time (JIT) debugging instead of this dialog box.

************** Exception Text **************
System.IndexOutOfRangeException: Index was outside the bounds of the array.
at MonkeyMatch.Form1.GetMatches(List`1 SubSet, List`1 SuperSet)
at MonkeyMatch.Form1.btnFindMatches_Click(Object sender, EventArgs e)
at System.Windows.Forms.Control.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ButtonBase.WndProc(Message& m)
at System.Windows.Forms.Button.WndProc(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

[Scottes]

I'm definitely not recognizing how that error could happen, inside that function. Is there any way you could upload your mm.db file so I could debug it?

I fixed Claude's issue yesterday, but your problem is different for sure. I have managed to get through Claude's database for Artists and Albums but he has 55,000 songs and I am seeing memory issues when matching Songs. This is also not the problem you're seeing.

I have tomorrow almost completely dedicated to working on these issues, and whatever else I can find. Having your database would allow me to fix at least one more issue.

[Scottes]

it found 210 artist matches, and select the preferred alternatives - but was not able to save my choices (i.e., the save button was grayed out). I also tried to use the 'auto save' option but that did not seem to do anything either(I continued finding the same 210 matches afterwards).

Just to make sure... The Save Changes button will be grayed out unless one of the names has changed, indicated by the text turning to Red. To do this, click on the Preferred button for your correct entry (the background will turn to Green), then double-click on the name you want changed (the text will turn Red). Then then Save Changes button will become enabled.

[hintergrundrauschen]

...he has 55,000 songs and I am seeing memory issues when matching Songs. This is also not the problem you're seeing.

Hehe... not your fault. Matching the songs obviously is an n*n size problem...

[dtsig]

OK, it's been 5 weeks since I made this available, and I've only gotten one report of anyone using it. And it's not a good report, since it doesn't work at all for him...

Is anyone using this? Successfully or not? Any comments, recommendations, wishes, anything? I'll even take more bug reports...


Also, is anyone willing to make their database available to me for download? That would give me the chance to see if things are working OK, and fix some bugs that I am sure will appear.

Scottes, I am just now trying it out with limited testing. Although the majority of the changes are simple capitalization .. there currently have been no problems to report on. So far this is working as advertised. Thank you very much

Edited: Info on setup, Win7 Ultimate, 8gig, 144+tracks.

[Scottes]

...he has 55,000 songs and I am seeing memory issues when matching Songs. This is also not the problem you're seeing.

Hehe... not your fault. Matching the songs obviously is an n*n size problem...

Yeah, basically. My 27,000 is fine, Your 55,000 runs our of memory. Twice the size requires 4x the iterations.

All lot of it has to do with the number of similar names. You have - to put it mildly - A LOT of similar Song names. I ran MonkeyMatch, set at Moderately Accurate, against your songs and I ran out of memory at 5,400 matches - and it was only 1/4 the way through the songs. And that took an hour and 18 minutes...

The error that you ran into, the row index array error, took me only a few minutes to fix. Then I ran through your Artists and Albums to see if there were any more issues. It got through them without issue, so all your foreign language ( ) music did not cause any problems. You could use the program right now against Artists and Albums - but I would not recommend it against Songs until I fix the memory issue.

I am currently working on some major rewrites to move to multi-threading for speed, and then I have to figure out how to deal with the memory issue.


For now, here is the latest version, with that bug fixed.
http://www.itsanadventure.com/MonkeyMat ... h_0549.zip

[rovingcowboy]

I am currently working on some major rewrites to move to multi-threading for speed, and then I have to figure out how to deal with the memory issue.


For now, here is the latest version, with that bug fixed.
http://www.itsanadventure.com/MonkeyMat ... h_0549.zip

scottes try to set a priority in your script so your script runs in high priority. high is all that is allowed to use in windows after xp. so it should be faster if you set it to run in high. also might set a limit on the cpu cycles to a high amount, if you can? making the script move faster should free up the memory issues.

[dtsig]

Scottes; I might have posted a little fast. I went though many updates. Stopped the process and started again. But did not have a problem.

I then did another scan and started to correct items. It then simply froze .. not completing the update of artist Al Stewert (about 15 albums). There was no way to stop it. It did not respond to anything then MMW stopped responding. Using Task Manager I stopped processes and rebooted.

I have been trying to monitor the memory usage when MM is running and haven't seen any big memory usage but will continue to try and track down what is causing the issue.

[dtsig]

Here are memory usages for several passes .. no problems here but as i note at the end it does get slower and slower and slower

All during 'Find Matches' it stayed at about
Commit = 22,484
Working = 32,472
Shareable = 22,684

Click Preferred then dclick one to change
Commit = 22,500
Working = 32,564
Shareable = 22,740

Click Save Changes and back to next group
Commit = 23,972
Working = 34,488
Shareable = 23,248

click perferred and dclick one to change
Commit = 23,192
Working = 33,704
Shareable = 23,280

Click Save Changes and back to next group
Commit = 23,972
Working = 34,488
Shareable = 23,248

At this point it appears that it has stopped working ... after several minutes waiting it returned with 'new match'

Commit = 25,036
Working = 35,612
Shareable = 23,296

Sitting for a few minutes to see what happens with memory

Commit = 24,980
Working = 35,580
Shareable = 23,296

click perferred and dclick one to change
Commit = 23,992
Working = 34,592
Shareable = 23,296

Click Save Changes and back to next group
Commit = 24,664
Working = 35,308
Shareable = 23,312

Again, to get to this point it took more and more time to do the updates. Even though for the last few we were only talking about a total of 5 albums max each. It should have been much quicker

Thoughts on what to check next ...

[dtsig]

Request: It would be nice to have an alternate EDITLINE that you could type in the correct information. There was some old tracks where none of the items listed were quite correct. I didn't see a way to change to the correct spelling. Did i miss it somewhere?

[Scottes]

Request: It would be nice to have an alternate EDITLINE that you could type in the correct information. There was some old tracks where none of the items listed were quite correct. I didn't see a way to change to the correct spelling. Did i miss it somewhere?

Right-click on the name and choose Edit.


Thanks for the memory tracking. The issue I am seeing is weird. I do not think it's running out of global memory - my machine has 32GB, yet MonkeyMatch started throwing "out of memory" errors at a point when I still had 23Gb free. And it happens when I am trying to create an integer array of 20 entries... I might have to fire up some debugging memory profiler to find it.

Or just go on the track where I optimize to use less memory. Which I should do anyway, so...


The issue where it stops responding for several minutes is another thing that should get fixed when I go multi-threaded. Right now everything is stuck in one thread, so the GUI can't update when the matching is tearing up all the CPU that the single thread can use. The GUI should be on a separate thread from that CPU-intensive matching process so that it can always respond. And the matching process should run in numerous threads to speed it up.


BTW, you should not have to reboot. Run Task Manager, kill MonkeyMatch, then I have found that it's a good idea to restart MediaMonkey. While developing this I would frequently lock up both MonkeyMatch and MediaMonkey, but I never had to reboot.

[Scottes]

scottes try to set a priority in your script so your script runs in high priority.

No, it's all because it's single-threaded. It sucks up all of the CPU it can, it's just limited to how much CPU it can grab. On my system it takes up 12.5% of the CPU while it's in the matching phase.