Space vs.   Characters

Get answers about the current version of MediaMonkey 5

Moderator: Gurus

Kilo
Posts: 50
Joined: Sat Jan 21, 2023 1:55 am

Space vs.   Characters

Post by Kilo »

Hi -

I've recently moved over from using iTunes to manage my MP3 library to MMW5. I am definitely loving the switch and should have done it a long time ago. After using MediaMonkey for awhile now though, I am having an issue where my tags in my MP3 files have the   character in the tag instead of a normal space character. Since I am also using MediaMonkey to autoname my files, it is also putting   in the filename. This is definitely something I haven't run into before in iTunes and I did a test on a MP3 file that had   for a space that was tagged in MediaMonkey. iTunes cleans   up into an actual space character when I update the tag. It is extremely hard to figure out where this is happening in my tags since visually they are both blank spaces. Right now I have to dump tags and my playlists into HTML to see if it using   anywhere. Is there a way to prevent MediaMonkey from using   in the tag/filenames and/or have MediaMonkey clean/remove the character similar to how iTunes does it?

For some more background on this issue, I also use MediaMonkey to sync my library and playlists to a USB flash drive to play in my car. I noticed that some tracks weren't appearing in my playlists and after looking into my tags and playlists in MP3TagEditor and MediaMonkey, I outputted my playlists into HTML and found out about the   character. My car's media player is likely just escaping the character and not reading the MP3 file because of this.

This is on 5.0.4.2690.
Barry4679
Posts: 2408
Joined: Fri Sep 11, 2009 8:07 am
Location: Australia
Contact:

Re: Space vs.   Characters

Post by Barry4679 »

Kilo wrote: Sat Jan 21, 2023 2:51 am It is extremely hard to figure out where this is happening in my tags since visually they are both blank spaces. Right now I have to dump tags and my playlists into HTML to see if it using   anywhere.
You should be able to get visibility of the problem as follows:
  • click the magnifying glass icon at the right end of the toolbar
  • type the characters nbsp
  • that should show you a list of all tracks that use   in any tag
As you say   generates a space when viewed by a html viewer, but when you view the tag in the main MM5 panel you should see the character   in its escaped form... you are not seeing that? Where in MM5 are you looking?

If you had just a few incidents you could fix them by manually retagging using MM5.

I am not sure that it would be generally helpful if MM5 auto-rename automatically unescaped any escaped character that it as going to use for a file name.
Want a dark skin for MM5? This is the one that works best for me .. elegant, compact & clear.
Kilo
Posts: 50
Joined: Sat Jan 21, 2023 1:55 am

Re: Space vs.   Characters

Post by Kilo »

I'm not seeing the character in the escaped form. The tags don't actually have 'nbsp' in the tag or filename. If I use Alt code 255 for the actual nbsp character, it doesn't return anything in the search either.
Peke
Posts: 17457
Joined: Tue Jun 10, 2003 7:21 pm
Location: Earth
Contact:

Re: Space vs.   Characters

Post by Peke »

Hi,
Unfortunately String replace within tag fields do not exist as an add-on for MM5. But with little thinkering with https://www.mediamonkey.com/docs/api/ , https://www.mediamonkey.com/wiki/Import ... s_(Addons) and using https://developer.mozilla.org/en-US/doc ... replaceAll it should be easily made by any javascript developer.
Best regards,
Peke
MediaMonkey Team lead QA/Tech Support guru
Admin of Free MediaMonkey addon Site HappyMonkeying
Image
Image
Image
How to attach PICTURE/SCREENSHOTS to forum posts
Barry4679
Posts: 2408
Joined: Fri Sep 11, 2009 8:07 am
Location: Australia
Contact:

Re: Space vs.   Characters

Post by Barry4679 »

Kilo wrote: Mon Jan 23, 2023 3:04 am I'm not seeing the character in the escaped form. The tags don't actually have 'nbsp' in the tag or filename. If I use Alt code 255 for the actual nbsp character, it doesn't return anything in the search either.
That makes it harder.
Do you know which tags contain the control character?
This may not help you, but you you identify tracks like this:

If you install the MM5 SQL Editor addon, the following query, as an example, finds any track where the Comment tag contains an embedded non-breaking space ( ). See here.
I updated the Comment tag of that example track to contain three characters; 'a' followed by a nonbreaking space followed by a 'c'

Code: Select all

select artist,album,songtitle,comment from songs
where comment like '%'||char(160)||'%'


ie. 160 being the unicode value of   character.
Want a dark skin for MM5? This is the one that works best for me .. elegant, compact & clear.
Peke
Posts: 17457
Joined: Tue Jun 10, 2003 7:21 pm
Location: Earth
Contact:

Re: Space vs.   Characters

Post by Peke »

Kilo wrote: Mon Jan 23, 2023 3:04 am I'm not seeing the character in the escaped form. The tags don't actually have 'nbsp' in the tag or filename. If I use Alt code 255 for the actual nbsp character, it doesn't return anything in the search either.
Can you post screenshot of one file properties showing " " and point where it is located?

Maybe even a DL link to one file example in PM to me for analyze?
Best regards,
Peke
MediaMonkey Team lead QA/Tech Support guru
Admin of Free MediaMonkey addon Site HappyMonkeying
Image
Image
Image
How to attach PICTURE/SCREENSHOTS to forum posts
Lowlander
Posts: 56492
Joined: Sat Sep 06, 2003 5:53 pm
Location: MediaMonkey 5

Re: Space vs.   Characters

Post by Lowlander »

There is this Replace Addon for MediaMonkey 5: https://www.mediamonkey.com/addons/brow ... -string-3/
Kilo
Posts: 50
Joined: Sat Jan 21, 2023 1:55 am

Re: Space vs.   Characters

Post by Kilo »

Barry4679 wrote: Mon Jan 23, 2023 8:22 am
Kilo wrote: Mon Jan 23, 2023 3:04 am I'm not seeing the character in the escaped form. The tags don't actually have 'nbsp' in the tag or filename. If I use Alt code 255 for the actual nbsp character, it doesn't return anything in the search either.
That makes it harder.
Do you know which tags contain the control character?
This may not help you, but you you identify tracks like this:

If you install the MM5 SQL Editor addon, the following query, as an example, finds any track where the Comment tag contains an embedded non-breaking space ( ).
I updated the Comment tag of that example track to contain three characters; 'a' followed by a nonbreaking space followed by a 'c'

Code: Select all

select artist,album,songtitle,comment from songs
where comment like '%'||char(160)||'%'


ie. 160 being the unicode value of   character.
Thank you very much for the suggestion! I am familiar with SQL so that is a nifty little tool to use to find things like this. It appears that it's only the song title that is effected by this. I modified the query a little bit to pull up the songs.

Code: Select all

select artist,songtitle,album from songs
where songtitle like '%'||char(160)||'%'
Peke wrote: Mon Jan 23, 2023 8:31 am
Kilo wrote: Mon Jan 23, 2023 3:04 am I'm not seeing the character in the escaped form. The tags don't actually have 'nbsp' in the tag or filename. If I use Alt code 255 for the actual nbsp character, it doesn't return anything in the search either.
Can you post screenshot of one file properties showing " " and point where it is located?

Maybe even a DL link to one file example in PM to me for analyze?
I will PM you a link with one MP3 file with the filename and song title tag that have this issue.
EDIT-I can't PM since I don't have the board rights to use PM yet.
Lowlander wrote: Mon Jan 23, 2023 11:28 am There is this Replace Addon for MediaMonkey 5: https://www.mediamonkey.com/addons/brow ... -string-3/
This is great. I did a test on an effected MP3 file and it replaced the character as expected. I am a bit weary to run it against all my tracks but it worked with the one file.

I will add that the MediaMonkey global search and advanced search does not return the nbsp character, which is odd. Since the nbsp character is Alt Code 0160, searching for that character doesn't return anything. Strange.
Last edited by Kilo on Mon Jan 23, 2023 4:17 pm, edited 1 time in total.
Peke
Posts: 17457
Joined: Tue Jun 10, 2003 7:21 pm
Location: Earth
Contact:

Re: Space vs.   Characters

Post by Peke »

Hi,
Kilo wrote: Mon Jan 23, 2023 4:15 pm I will PM you a link with one MP3 file with the filename and song title tag that have this issue.
EDIT-I can't PM since I don't have the board rights to use PM yet.
No need I completely forgot that MPG updated plugin for MM5
Best regards,
Peke
MediaMonkey Team lead QA/Tech Support guru
Admin of Free MediaMonkey addon Site HappyMonkeying
Image
Image
Image
How to attach PICTURE/SCREENSHOTS to forum posts
Barry4679
Posts: 2408
Joined: Fri Sep 11, 2009 8:07 am
Location: Australia
Contact:

Re: Space vs.   Characters

Post by Barry4679 »

Kilo wrote: Mon Jan 23, 2023 4:15 pm Thank you very much for the suggestion! I am familiar with SQL so that is a nifty little tool to use to find things like this. It appears that it's only the song title that is effected by this.
That's good.
How many tracks were affected? Too many to fix manually via the MM5 track Properties panel?

If there were many you could try fixing as follows:
  • change your sql query to update one of the Customx columns, for those tracks, to some unique identifier
  • create a MM5 Collection with criteria to select just these tracks
  • then use MM5 Auto-Organise Files facilty to rename those tracks to some filename that doesn't include the songtitle
Humans may need the songtitle in the track, but all browser and music clients (that I know of) use the tag metadata when browsing or filtering .
eg. all of my file names are just <trackNumber>.flac ... eg. 01.flac ... I have 1000's of tracks, which all have this same track name.
I do this to maximise the amount of tracks that my Sonos equipment can hold in their tiny memory chips, ie. reduce path length .. but the point is that there is no downside ... the MM5 database knows the track names, and I never need to see or interact with individual file names.

This would make your playlist issue go away wouldn't it?

I tried this:
  • I replaced a space with this control character to title of one of my tracks
  • when I MM5 search using the assumed blank in MM5, the track is not found (unless I set MM5 search to use "respect diacritics/Match within words" ... (see Tools>Options>Library>Search) .. The track was "For Your Love", and I replaced the 1st space with a non-breaking space control character ... the search term was For Your ... ie. using the assumed normal space character ==> that's not good
  • used MM5 Auto-Organise to rename the track to something that included the file name
  • I added the track to a static MM5 playlist, ... MM5 could play the track via the playlist OK
  • I exported the playlist to a .m3u file ... I can see that the playlist contains the control character (hex C2A0) ... so it will be unplayable by any player which has unescaped the control character, ie. your car presumably
This is the only problem that this has caused for you? ... ie. in your car
I can't fully reconcile what I see. I am guessing that maybe Windows and Linux handles non-breaking space characters natively, but you car thing doesn't ... you got any other theories


Do you know more about how|where iTunes santised these control characters in your tracks?
Do you know how they got into the tags?
Am still not sure what, if anything, should be fixed.
Was it some exotic tool or music repository l that caused this? Or is an Apple thing?
What do you think?
Want a dark skin for MM5? This is the one that works best for me .. elegant, compact & clear.
Kilo
Posts: 50
Joined: Sat Jan 21, 2023 1:55 am

Re: Space vs. &nbsp; Characters

Post by Kilo »

Barry4679 wrote: Mon Jan 23, 2023 11:30 pm
Kilo wrote: Mon Jan 23, 2023 4:15 pm Thank you very much for the suggestion! I am familiar with SQL so that is a nifty little tool to use to find things like this. It appears that it's only the song title that is effected by this.
That's good.
How many tracks were affected? Too many to fix manually via the MM5 track Properties panel?

I tried this:
  • I replaced a space with this control character to title of one of my tracks
  • when I MM5 search using the assumed blank in MM5, the track is not found (unless I set MM5 search to use "respect diacritics/Match within words" ... (see Tools>Options>Library>Search) .. The track was "For Your Love", and I replaced the 1st space with a non-breaking space control character ... the search term was For Your ... ie. using the assumed normal space character ==> that's not good
  • used MM5 Auto-Organise to rename the track to something that included the file name
  • I added the track to a static MM5 playlist, ... MM5 could play the track via the playlist OK
  • I exported the playlist to a .m3u file ... I can see that the playlist contains the control character (hex C2A0) ... so it will be unplayable by any player which has unescaped the control character, ie. your car presumably
This is the only problem that this has caused for you? ... ie. in your car
I can't fully reconcile what I see. I am guessing that maybe Windows and Linux handles non-breaking space characters natively, but you car thing doesn't ... you got any other theories


Do you know more about how|where iTunes santised these control characters in your tracks?
Do you know how they got into the tags?
Am still not sure what, if anything, should be fixed.
Was it some exotic tool or music repository l that caused this? Or is an Apple thing?
What do you think?
Thanks for the tip regarding the library search. Mine was set to the default 'ignore diacritics/match first word'. Once I changed that, I was able to search for the Alt Code 0160 and it pulled up the tracks that the SQL query found, which was 18 or 19. Not too many. To fix it, I selected the tracks the search found and used the Replace String addon to replace the nbsp character with a space and used auto organize to fix the filenames. All done.

I am fairly certain that iTunes just trims the tags which is why I've never ran into this before. Most programming languages' native trim function that I know of removes nbsp characters and can remove excess spaces, like HTML and JavaScript. I am kind of scratching my head why tags aren't trimmed in MediaMonkey like in iTunes but perhaps there are more use cases that require them to not be trimmed. Just seems like a good small data cleanup/QC feature to me.

As far as where they came from, all of these tracks are CDR/white label tracks, which means they have no official release source so it's from SoundCloud, podcasts, or YouTube. Since these MP3s will have no tags/incorrect tags, I copy and paste the artist/track title from the source, from SoundCloud for example, and the autotag goes from there and I fill in the rest. Whatever I am copying from has these bad characters in the text so they get put into the tag.
Barry4679
Posts: 2408
Joined: Fri Sep 11, 2009 8:07 am
Location: Australia
Contact:

Re: Space vs. &nbsp; Characters

Post by Barry4679 »

Is good that you are all fixed now.
Kilo wrote: Tue Jan 24, 2023 1:44 am
I am fairly certain that iTunes just trims the tags which is why I've never ran into this before. Most programming languages' native trim function that I know of removes nbsp characters and can remove excess spaces, like HTML and JavaScript. I am kind of scratching my head why tags aren't trimmed in MediaMonkey like in iTunes but perhaps there are more use cases that require them to not be trimmed. Just seems like a good small data cleanup/QC feature to me.
I guess the issue is when to do this.

You wouldn't want them to do it when the tracks were scanned into MM5. ... The tracks may already have file names with nbsp characters. There may be other music apps that index those tracks, and which were accepting of the nbsp characters in file names or tags ... They (MM) can't have a situation where someone trials MM5, and it "corrupts" their music collection.

It is also dodgy if someone runs Auto-Organise, which should just change track paths, and MM5 alters the tags.
Nor is it good if this operation renames files to something that doesn't accurately reflect the tags.
Same thing when it generates playlists.

Also I wonder if all environments allow blanks in file names. I don't think so. In these cases the nbsp character serves to enhance readability, while ensuring that the OS can still find the file Ok.

The other side of the argument is that nbsp characters cause the default MM5 Search|Filter mechanism to appear buggy, because what looks like a blank space is not what it appears.

I suppose there could be something that detects nbsp characters in file names or tags, when the tracks are scanned into MM. The User could be notified, and have the option to sanitise the situation by removing all nbsp characters, except in tags like lyrics, or comments, etc.

BTW I found one track in my collection that has a track name with a nbsp character.

I will wait a little to see if anyone else chips in. The next step would be to get the request submitted in their tracking system for consideration.
Want a dark skin for MM5? This is the one that works best for me .. elegant, compact & clear.
Peke
Posts: 17457
Joined: Tue Jun 10, 2003 7:21 pm
Location: Earth
Contact:

Re: Space vs. &nbsp; Characters

Post by Peke »

Kilo wrote: Tue Jan 24, 2023 1:44 am Most programming languages' native trim function that I know of removes nbsp characters and can remove excess spaces, like HTML and JavaScript. I am kind of scratching my head why tags aren't trimmed in MediaMonkey like in iTunes but perhaps there are more use cases that require them to not be trimmed. Just seems like a good small data cleanup/QC feature to me.
MM as written in Delphi have extensive trim functions, but answer to your "scratching" is simpler than you think, some users used Space or &nbsp; Character for padding ;) Actually we had much more reports than not being able to trim &nbsp; Character from metadata when we tried to trim things more automatically.
Best regards,
Peke
MediaMonkey Team lead QA/Tech Support guru
Admin of Free MediaMonkey addon Site HappyMonkeying
Image
Image
Image
How to attach PICTURE/SCREENSHOTS to forum posts
Post Reply