I think those might be two different issues. One is that changes in the sources list are not loaded when opening the dialog, the other is that all changes made to the list are completely ignored when auto-tagging files. Well, either that or I just misunderstand how the setting is supposed to work, which is definitely a possiblity.

My expectation was that, by disabling all auto-tag sources except dummy, auto-tag would not work since all sources mediamonkey could pull data from have been disabled. Tagging still works fine after doing so, so either the dummy tagger is providing data, or the sources I disabled were not actually disabled. The same seems to be true for the re-ordering sources, it doesn't seem to have any effect at all.

I can reproduce, it's a minor visual bug as it does show as disabled on the main options window: https://www.ventismedia.com/mantis/view.php?id=18146

Hi,
when reordering or disabling lyrics or auto-tag metadata sources in build 2424, the changes are correctly reflected on the settings page. If the SET SOURCES button is clicked a second time, all changes are set to their default settings, the previous changes are ignored.

Update:
It looks like changes to the auto-tag sources are ignored by the auto-tagging framework. I disabled all sources except dummy and still got auto-tag results.