Corrupted FLAC files after tag edit

[kdo]

I was just wondering if there was more to it - i.e. if there was a characteristic look to the waveform in this particular case of FLAC file truncation, that's all.

- Tim

Ah, alright. Then you should look at the waveforms of two consecutive FLAC tracks.

Normally you should see the last sample of one track (its amplitude value) not too far away from the first sample of the next track, and also overall curve should be more or less smooth in the transition between tracks (if you zoom in on the region of, say 10-20 samples).

Now, imagine you cut off a piece of the second track at some random position. The curve of the first track will no longer match the second one, and there will be a jump of amplitude. The bigger the jump, the more audible it is.

[tbessie]

Thanks for all your suggestions - very helpful indeed!
I will do some experiments and report back here.

- Tim

[tbessie]

Does anyone know if this pre-truncation problem of FLACs has been verified by Jiri or anyone else but kdo? I still haven't had time to test it on my FLAC files myself, but will soon.

- Tim

[cd_paul]

I have looked at some of my 'repaired' files and I can confirm that what KDO says is correct, unfortunately. In all cases audio samples ( > 1000 ) are lost at the beginning of the WAV files created from the FLAC's.

OK, these samples are often silence but not always. Also, the number of samples is small. So, for some this may not represent a big problem. I opted for overwriting the FLAC when I did the conversion and I want accurate files. So, for me I'll have to re-rip. As my CD's and me are 6000 miles apart I have no choice but to wait before doing anything.

When I read the posts suggesting this problem I stopped 'repairing' my files. I still have a number of the 'original' corrupted files. So, a 'repair' utility would be very useful. Can anybody a MM confirm that a true repair is possible or not given the nature of the original corruption. Also, does anybody have the required time to write the appropriate utility?

[GWS]

Hi there,

Ran into exactly the same issues:

Auto TAG ftom AMAZON messed up my FLAC file(s). Did not test other tagging but I assume the same applies there. I was running 2.5.5.996 and have been changing in_flac.dll and in_flac_codec.dll multiple times in attempts to fix the issue so I got lost there.

I wrote scripts to decode the flacs to WAV and recode them again. Indeed the first time the .flac is smaller then the corrupted .flac file but redoing all of this multiple times does not make the result smaller every cycle. Therefore I conclude that the first recode solves the issue without audio loss. Also I could not hear any loss at start of the file.


Remedy:
On another PC I installed 2.5.5.998 as clean install and tested updating form Amazon (same server same files) with success (no corruption) later on I copied these files from c:\program files\MediaMonkey\*.* /s to my system with the errors and ran the same tests again with succes!

Used flactester.exe to determine if there were any errors. Normally if there a error Flactester lost sync on 0m0s and skipped rest of the file.

Now I still have hundreds of files to fix but that should be possible doing a recode etc. It is a lot of work though and therefore I would strongly urge to have the following option in MM3

For coding/recoding/tagging (everything that changes a .flac file) I would very much appreciate a "paranoia" flag that will enforce verifying the files immedialtely after the change is applied. (flac -t <filename> after very change should do the trick) Off course rollback if any error occurs should be the default.

There is far too much time and effort involved in creating the library to risk messing things up without knowing it.

GWS

[Guest]

Hello:

I am a "Gold" paying subscriber to MediaMonkey and I have used this software with great success for a long time. Having said that, I have over 450GB of FLAC files on my server and the majority of these files have had their tags edited by MediaMonkey since late last year. I recently rebuilt 3 of my computers due to their not having been able to play the FLAC files. At the time, I believed that my codec installations had become corrupt. Just today I finally learned that the FLAC files themselves have been corrupted by MediaMonkey.

I want to emphasize that this is a very serious issue and I strongly suggest that the MediaMonkey team provide a simple repair utility which could automatically repair the headers on the FLAC files damaged by MediaMonkey.

So now a few comments:

1. Can the team build and release a utility which can identify and then fix the corrupt headers? I don't want to have to re-encode all of my files again (even from FLAC to FLAC). This would represent nearly a full week of non-stop processing on my server to resolve.

2. This thread should be a "sticky" and should be posted at the front of the forums. I also feel strongly that an alert should be on the home page of the MediaMonkey web site. I imagine many users don't know why their FLAC files suddenly won't play in any software but MediaMonkey. They are likely going crazy trying to solve the problem as I have been for some time.

3. Some people have mentioned that using MediaMonkey to re-encode their files resulted in a new file of smaller size than the original and theorized that some data was lost in the re-encoding process. I have used DBPOWERAMP software version 11.5 using FLAC 1.1.2 codecs (this is the pre version 12 which uses FLAC 1.1.3 instead) and the resulting file sizes were exactly the same size as the original but played normally.

4. In some testing today, I noticed that in many cases going from the older FLAC 1.1.2 to new 1.1.3 that the files sizes are changing (and in many cases ending up larger than the original). This is a FLAC codec issue but seems odd. Why is this?

5. Has the very latest build of MediaMonkey been fully test and confirmed NOT to cause new problems?

Thanks.

Andrew

[kdo]

3. Some people have mentioned that using MediaMonkey to re-encode their files resulted in a new file of smaller size than the original and theorized that some data was lost in the re-encoding process. I have used DBPOWERAMP software version 11.5 using FLAC 1.1.2 codecs (this is the pre version 12 which uses FLAC 1.1.3 instead) and the resulting file sizes were exactly the same size as the original but played normally.

You do compare the size (duration) of the audio, not the size of the FLAC files themselves, right?
i.e. compare number of wav samples in the original untouched file vs. number of wav samples in the corrupted and re-encoded file

[GWS]

I did the re-encode with Flac 1.1.4 and this solved my issues. Also all files as far as I can see now were smaller.

I can see two reasons for that:

I did the de-and-encode process flac->WAV->flac.
Old flac version is not 1.1.4 in almost all cases (most were done with 1.1.2) I have lost some tagging info the process and

1.1.4 most likley does a better job in compression. For some files I don't know the setting but I know that a lot of them were compressed level 8 with MM and the resulting file after my process using flac 1.1.4 was smaller.

I requested MM to have a "paranoia" flag to have at leat tested all files immediately after every change (encode or tag). An MD5 or CRC checker could also be of help for this.

MM is farout the best tool for the media managment job from a functional point of view. However if it possibly messes up the files without us knowing it it not worth a dime..... and we are better of not using it.

I am pointing to MM in my case but it is still possible that other tools like EAC or flac.exe itself have the same issue in some scenarios. This does not mean that MM should do the job including testing so we are sure MM did not mess up the files/library. For now I run a flactester.exe on the full tree with all my media files every now and then but this is consuming lots of resources. I am working on a script using FileDir.exe and flac -t to verify all changed *.flac files every day. Per day that should not be a lot of work.

Hope to post the script here soon so you can all use it.


GWS

[kdo]

I did the re-encode with Flac 1.1.4 and this solved my issues. Also all files as far as I can see now were smaller.

Again, sorry for re-iterating what I already wrote above, but this is just to make it totally clear:

* comparing file size of the re-encoded FLACs is irrelevant to the problem.

* one need to compare number of wav samples in the original and corrupted/re-encoded file


GWS, it's not quite clear (to me) what did you do. Did you actually compare the number of samples?

[GWS]

KDO

I was giving some explanations on why sizes differ and audio still could be in tact.

Since I don't know how to compare the # of samples I did not do that. Likeley it is through flac -a ???

After all I did there is no audible difference in the file but that does not proof a bit or two (maybe four bytes) fell over. My guess is they are smaller because of better compression in a newer version or different compression level settings (Can you know the compression level from the file if you would like to verify this?)

In another threat someone describes a more thourough investigation and some results. See
http://www.mediamonkey.com/forum/viewtopic.php?t=15400

A number of causes and solutions have been given now but the problem stands that this can still happen to anyone withuot knowing files are being messed up in the background.

I hope someone can come up with a small tool to test for and fix affected files the easy way without recode etc.
MM should not touch tagging in files created with a higher FLAC version than it is using.

Actually I think MM should make it an issues to stay up to date and support 1.1.4 since files created with that release are hitting the net. If the latter is too much for version 2.5.5 than a (suggested) recode should help when tagging these files.

Huge libraries have been affected by this problem and I would like to see a remedy and probably some other people too.

Trust me I really appreciate all the hard labour that went into MM and it is the best tool in the market as far as I can see. It would be a pity to lose the customer confidence because of this issue.

Regards,

GWS

[tbessie]

I agree -- I would like to see more of the MediaMonkey developers chime in with their opinion as to whether they can verify this or not.

It seems they're fully focused on MM 3.0 now, though, so they don't have much time for going back to fix problems caused by earlier releases of MM.

What I'd really like is some kind of acknowledgement, if true, that pre-MM2.5.5.998 versions DID cause this problem, and telling us what actually -- programatically -- was going wrong, so we could decide if re-ripping, though a terrible fate, would be something we want to do.

- Tim

[Q-Brick]

Hi,

and telling us what actually -- programatically -- was going wrong

as I have written in the other thread

When MM updates the Vorbis-Comments, the length of the padding was not calculated correctly.
In an example-file I see that the padding length was 4 bytes to long, so that the first 4 bytes of the header of first audio-frame were overwritten with zeros.

So, what could be recovered of the FLAC-files, depends on how many times you have updated the metadata. Each time you update the metadata you lose another 4 Bytes. >:-(

If you are lucky, you just loose 4 Bytes, which could be reconstructed (constant) ...

Q-Brick[/quote]

[tbessie]

So, what could be recovered of the FLAC-files, depends on how many times you have updated the metadata. Each time you update the metadata you lose another 4 Bytes. >:-(

If you are lucky, you just loose 4 Bytes, which could be reconstructed (constant) ...

Q-Brick

Not so lucky - I generally set up my tags after ripping through successive waves of mass-retagging (first normalizing genres, then albums, then individual tweaks), so it's likely I'm missing up to 50 or more bytes from the audio frames. Which sucks and really annoys me - why spend days and days and days (taking place over many months, actually) ripping with EAC to FLAC for an exact copy of the CD, tweaking tags to be perfect and consistent, and then discovering I've lost audio data, however so small?

It sucks big time.

Edit: What I'd *actually* like to see, once again if it is 100% certain this defect existed and has damaged files, is for the developers to -announce- "Yes, we fucked up, we are sorry" and maybe also "... the developer who didn't completely unit- & regression- test his code will be spanked with a copy of the Apple Human Interface Guidelines Manual", or something along those lines. :-/

- Tim

[AerosSaga]

I agree, I edited lots of my flac files pre 998, and need to know what to do now to correct/insulate the damage done. I am very upset by this...

[ABS]

QBRICK -

Are you sure about your statement? Are you a developer for this software or do you have access to the source code? If not, then what test did you do to confirm this behavior? In some other posts, which I think may have been from the development team there was no indication of the actual music data having been lost. Please provide more details - that is what we are all looking for.

DEVELOPERS - Why no response? There are a lot of us with very large collections who are now trying to cope with this serious issue. Please provide us with some of the details we need to make informed decisions about how to handle this problem. We do NOT have access to the source code so we need you to HELP us, please.

Andrew