OCR ID3 tagging improvements - DONE!

[José the Bronx Rican]

I've been meaning to ask - since my only experience is with a Creative Zen - just what systems these different players use to sort and playlist tracks. It's important for me to know before I move on with tagging my own stuff, which WILL happen no matter what, because every brand plays with the tags their own way. For example, album art size must be 200x200 before Zen recognizes it. Also, Zen doesn't sort by "year," although I'm sure others do. On that note, here's my suggestion: fix the songs with missing "year" tags. I'm not sure if the year given is necessarily the year the song was posted, but it makes sense to do that.

[Native Jovian]

Nowadays, I've seen OSes & programs smart enough to understand that filenames with Final_Fantasy_9 come before Final_Fantasy_10, so I think it's OK, but I'll keep an eye on that. As far as the actual tags, don't worry Mattias, Roman numerals where appropriate.

As far as this issue, I'd suggest using digits in the filename but Roman numerals in the ID3 tag. A quick check shows that Windows XP is in fact smart enough to know that 1 < 2-9 < 10+, but the original Final Fantasy isn't tagged as "Final Fantasy 1" so it gets all messed up. It classes numbers as higher than letters alphabetically, so FF1 is AFTER all the other Final Fantasy games. Also, Final Fantasy Tactics and Final Fantasy Mystic Quest get all mixed in with the Final Fantasy 1 games because it sorts all those mixes in with the FF1 songs at "T" and "M" respectively. That leads to another question. While filenames are smart enough to order themselves like that, are MP3 players? Maybe we'd be better off sticking with digits the whole way through to make those sorts of sorting issues nonexistent in any medium.

Another random issue I noticed! There are two Chrono Trigger mixes (KnightsComeMarchingHome and Downwind) that get messed up in order because, unlike any other remix that I'm aware of, they're listed as coming from multiple games. Knights is listed as "Chrono Trigger & FF9", while Downwind is "Chrono Trigger & Xenogears". There are other mixes from multiple unrelated games and they've never done this, so it should probably be standardized to listing the filename only under the "main" game.

OCR mix ID could easily go under track number.... however, this would mean that playing album "http://www.ocremix.org" would automatically sort by mix ID rather than by title, so that's something we'd probably want to avoid unless there was another easy way to sort by game, and I'm not aware of one. Perhaps put it under Disc? My iPod doesn't sort by disc number (something that has annoyed me intensely, actually), just track number within a particular album... Not an ideal solution, but the only one I can think of at the moment.

Either Album Artist/Band or Publisher seems to be a logical place to list the game the mix comes from, but I don't think that there's an option to sort by those fields in most MP3 players, so it'd be a bit pointless as far as that goes. Methinks the best solution for that is to just keep things as-is and have the game be the first part of the title.

[Moguta]

As far as this issue, I'd suggest using digits in the filename but Roman numerals in the ID3 tag. A quick check shows that Windows XP is in fact smart enough to know that 1 < 2-9 < 10+, but the original Final Fantasy isn't tagged as "Final Fantasy 1" so it gets all messed up. It classes numbers as higher than letters alphabetically, so FF1 is AFTER all the other Final Fantasy games. Also, Final Fantasy Tactics and Final Fantasy Mystic Quest get all mixed in with the Final Fantasy 1 games because it sorts all those mixes in with the FF1 songs at "T" and "M" respectively. That leads to another question. While filenames are smart enough to order themselves like that, are MP3 players? Maybe we'd be better off sticking with digits the whole way through to make those sorts of sorting issues nonexistent in any medium.

Yeah. While I don't feel too strongly about this really, roman numerals seem to be overly aesthetic. For example, you'll get FF IX coming before FF V. And many things sort by tags (media player's media libraries, iPods). And the original FF songs get thrown everywhere, between & around the numbered games depending on the title. I guess that last issue could be solved by seperating the game name from the track title by a ": " or " - ", but then I've noticed older tracks seem to treat the game name as part of the remix title. Not sure there's any perfect solution here.

OCR mix ID could easily go under track number.... however, this would mean that playing album "http://www.ocremix.org" would automatically sort by mix ID rather than by title, so that's something we'd probably want to avoid unless there was another easy way to sort by game, and I'm not aware of one. Perhaps put it under Disc? My iPod doesn't sort by disc number (something that has annoyed me intensely, actually), just track number within a particular album... Not an ideal solution, but the only one I can think of at the moment.

Arg, are you serious? Can you not manually make the iPod sort by title name inside an album? If not, it seems it would be extremely cumbersome to find a specific OCR song on your iPod. Maybe there is no place in the ID3 tags for the mix ID after all. *facepalm*

Concerning album art, there was a recent topic that seems a pretty good candidate. That is, unless Dave has a graphical idea of his own.

Larry, as far as additional ID3 fields that OCR should use, I think we've got 'em all covered so far. That's why you're not seeing any more posts on that issue.

[SoulinEther]

Correct me if I'm wrong, but right now, track number is used to identify which game of a series the remix is from? Like Castlevania 3 remixes would get a 3 in the track number? Or was that just my media player's fabrication and it's trying to trick me?

Isn't that kind of a waste? There should be something else being put there. And if not, what is being used there really?

Edit: my media player is a deceitful bastard. So... can't something be put there? I want to suggest remix #, but... I think that would be annoying for people whose media players / mp3 players automatically sort an album by track numbers - I personally find it handy that I can find songs from particular games easily because the songs are sorted in alphabetical order by title.. dunno.

[Liontamer]

fix the songs with missing "year" tags

I dunno where the issue came from, but I believe it's all set. I already knew about it though, but thanks anyway for pointing it out.

As far as this issue, I'd suggest using digits in the filename but Roman numerals in the ID3 tag.

That's already the new standard, and I've carried that over to the older stuff.

the original Final Fantasy isn't tagged as "Final Fantasy 1" so it gets all messed up. It classes numbers as higher than letters alphabetically, so FF1 is AFTER all the other Final Fantasy games. Also, Final Fantasy Tactics and Final Fantasy Mystic Quest get all mixed in with the Final Fantasy 1 games because it sorts all those mixes in with the FF1 songs at "T" and "M" respectively.

Nah, it's fine. There'll never be a 100% accurate system re: filename/tag sorting. We're OK with what's in place now.

Knights is listed as "Chrono Trigger & FF9", while Downwind is "Chrono Trigger & Xenogears".

Way, way ahead of you.

OCR mix ID could easily go under track number.... however, this would mean that playing album "http://www.ocremix.org" would automatically sort by mix ID rather than by title, so that's something we'd probably want to avoid unless there was another easy way to sort by game, and I'm not aware of one. Perhaps put it under Disc? My iPod doesn't sort by disc number (something that has annoyed me intensely, actually), just track number within a particular album... Not an ideal solution, but the only one I can think of at the moment.

Isn't it just as simple as sorting by filename or track title? If not, we may need to reconsider the idea. Someone run a test.

Either Album Artist/Band or Publisher seems to be a logical place to list the game the mix comes from, but I don't think that there's an option to sort by those fields in most MP3 players, so it'd be a bit pointless as far as that goes. Methinks the best solution for that is to just keep things as-is and have the game be the first part of the title.

Yep. And there's definitely not even the tiniest bit of reconsideration about the filename or track title styles. Those styles help popularize the mixes the same way the URL as the album popularizes the site. We'd be stupid to change them; they're the way they are for a good reason.

Concerning album art, there was a recent topic that seems a pretty good candidate. That is, unless Dave has a graphical idea of his own.

Yeah, I figure it's Dave's call.

[Native Jovian]

Nah, it's fine. There'll never be a 100% accurate system re: filename/tag sorting. We're OK with what's in place now.

Well, all you need to do to fix all that is change everything to "Final Fantasy 1" instead of plain "Final Fantasy" and all that goes away. FF1 gets sent to the front of the list, and Tactics/Mystic Quest sent to their own sections. It's simple, it's easy, and it makes everything much neater. I think it's worth doing.

Isn't it just as simple as sorting by filename or track title? If not, we may need to reconsider the idea. Someone run a test.

I dunno about anyone else's iPod, but mine automatically sorts all albums by track numbers and only defaults to sorting alphabetically by title if there are no track numbers (or all track numbers are equal). So no, no way that I know of to sort album http://www.ocremix.org by title if there are track numbers there.

[Liontamer]

Well, all you need to do to fix all that is change everything to "Final Fantasy 1" instead of plain "Final Fantasy" and all that goes away. FF1 gets sent to the front of the list, and Tactics/Mystic Quest sent to their own sections. It's simple, it's easy, and it makes everything much neater. I think it's worth doing.

The reason it's not for me is because if we did that for some games, in order to properly and uniformly do that, we'd have add 1s for all the games with sequels. It's a beyotch. If djp were down with that change, sure, but otherwise, I'd rather not.

[chanq]

I know I've already posted a thread about this, but yet again here's my suggestion for OCR album art:

I use it for all my OC ReMixes and it works perfectly in both iTunes and on my iPod. 400x400 pixels.

[anosou]

The reason it's not for me is because if we did that for some games, in order to properly and uniformly do that, we'd have add 1s for all the games with sequels. It's a beyotch. If djp were down with that change, sure, but otherwise, I'd rather not.

Most of all I think it would be quite awkward to "rename" some games so they would fit better on someone's iPod. If it's an issue anyone can redo the tags slightly on their own.

The thing I'm stressing here is that we shouldn't alter the games original name (in this case American release-name since OCR is an american site). In cases like FFIII later released as FFVI one SHOULD go with FFVI since that is more accurate re: the games original japanese name.

Pretty much.

Isn't it just as simple as sorting by filename or track title? If not, we may need to reconsider the idea. Someone run a test.

Yes. The only issue is that most MP3 players and apps sort albums by track number and not title (the latter would be stupid really). But then again I PREFER ocremixes sorted by Remix ID instead of game. It brings more awareness to every remix regardless of game and if someone wants Zelda they can easily 1. sort by title, 2. search for zelda or 3. set up a playlist.

I vote strongly FOR Remix ID as Track Number.

[SoulinEther]

I PREFER ocremixes sorted by Remix ID instead of game. It brings more awareness to every remix regardless of game and if someone wants Zelda they can easily 1. sort by title, 2. search for zelda or 3. set up a playlist.

Now that I think about it, I'm fairly certain I could set up a playlist with the tracks sorted alphabetically by title name - and it wouldn't be hard to do at all.

So... now I'm for this as well (remix id as track number)

[anosou]

Now that I think about it, I'm fairly certain I could set up a playlist with the tracks sorted alphabetically by title name - and it wouldn't be hard to do at all.

So... now I'm for this as well (remix id as track number)

Welcome to the light side young one

[The Pezman]

Ooooooh...

The ID3v2 Chapter Addendum was published in December 2005 but is not widely supported as yet. It allows users to jump easily to specific locations or chapters within an audio file and can provide a synchronized slide show of images and titles during playback. Typical applications include Enhanced podcasts and it can be used in ID3v2.3 or ID3v2.4 tags.

Nothing to use as a standard, but this could allow for some interesting audiovisual collages people could put together and perhaps host here as well.

[Rozovian]

What about name changes? Just remembered that DJ Crono changed his remixer name to Jovette Rivera, dunno if there are any others. Should the old remixes have the artist tag changed to match the remixer's current remixer name?

Oh and the name tag of one of Destiny's sd3 remixes begins with a space.

[Liontamer]

What about name changes? Just remembered that DJ Crono changed his remixer name to Jovette Rivera, dunno if there are any others. Should the old remixes have the artist tag changed to match the remixer's current remixer name?

Oh and the name tag of one of Destiny's sd3 remixes begins with a space.

Appreciated, but I've already gone through all those issues. I don't need any help with current ReMixer names and the program I'm using caught other issues like spaces. But I do appreciate the lookout.

If I miss something like that, feel free to gloat.

[Olarin]

With respect to track numbers being the remix ID number:

I've already been doing this manually in iTunes for my own collection, but I ran into a problem - I could only input a maximum of three digits for the track number. Is this a limitation of ID3 tags or merely iTunes being stupid? I worked around this by making remixes below 1000 disc number 1, and remixes above 1000 disc number 2. That's still not a perfect solution though as it leaves the question of what to do with mix #1000 (can't have a track number of 0, as far as I know).

With respect to album artist:

Isn't the usual practice in a case like this to just make it "Various Artists" or simply "Various"? (That's what I've been using)

[The Author]

I think I remember one of the biggest complaints people have had in the past is that, when put into iTunes, all the songs come up as separate albums based on the artist, despite all the songs being part of the same album. It results in a plethora of "http://www.ocremix.org" albums showing up. So I suggest that this problem be fixed.

In Itunes you have to change the songs to a "compilation" in the song properties... dunno if that can be done in the tags.

[Liontamer]

With respect to track numbers being the remix ID number:

I've already been doing this manually in iTunes for my own collection, but I ran into a problem - I could only input a maximum of three digits for the track number. Is this a limitation of ID3 tags or merely iTunes being stupid? I worked around this by making remixes below 1000 disc number 1, and remixes above 1000 disc number 2. That's still not a perfect solution though as it leaves the question of what to do with mix #1000 (can't have a track number of 0, as far as I know).

Bah, kick u iTunes. Can't anyone else verify that issue?

With respect to album artist:

Isn't the usual practice in a case like this to just make it "Various Artists" or simply "Various"? (That's what I've been using)

It'll be OverClocked ReMix. We ain't no generic "various artists".

In Itunes you have to change the songs to a "compilation" in the song properties... dunno if that can be done in the tags.

Using the Album artist/Band field should fix that.

[Doulifée]

Bah, kick u iTunes. Can't anyone else verify that issue?

work with 6 digits with a regular editor.

[The Author]

Although, maybe there should be an album number, like one for each year...

[Liontamer]

Although, maybe there should be an album number, like one for each year...

You'd then have to redo the track #s, which is pointless busywork.

work with 6 digits with a regular editor.

If anyone one else could verify that just to be doubly sure, that'd be great. OCR IDs are 5 digits, so I'd be happy if there wasn't an issue with that.

[Olarin]

Okay, just to test, I opened up a file in winamp, set its track number to a five digit number with a leading zero (01752), and brought it back into iTunes. It displayed correctly, although without the leading zero (1752) (but it didn't actually remove the leading zero, it just wouldn't display it). So it can read extra digits, it just won't let you make them. Incredibly stupid, but not an actual problem as far as this goes.

Incidentally, as you might guess from the ID #, I was testing with my own ReMix, and I happened to notice that the "Encoded By" field contains the email address from which I submitted the mix. Not a problem for me personally, but is this necessary/desirable? I certainly wasn't aware of this practice when I submitted, and it might be an unpleasant shock to those who prefer to keep their email address a closely guarded secret.

[Mini-Me]

I've just thought of a suggestion...it's kind of the opposite of adding information to unused fields, but this seems like a good chance to bring it up:

Some ReMixes have APE tags embedded in them in addition to ID3 tags, and the title, artist, album, etc. are generally written incorrectly in the APE tags. Unfortunately, certain players (e.g. Rhythmbox under Linux) strangely give priority to these tags at the expense of the ID3 tags. Until I finally realized what was going on, I kept trying to change the ID3 tags to no effect, and the same wrong title/artist/album etc. from the APE tags kept being displayed in Rhythmbox. To give a few examples (by no means an exhaustive list), http://www.ocremix.org/remix/OCR01139/, http://www.ocremix.org/remix/OCR01238/, and http://www.ocremix.org/remix/OCR00191/ exhibit this problem. Actually, the last one has some other kind of superfluous tag (AAC? Some old Quicktime tagging? It's Apple something.) rather than APE, but in any case, it causes the same problem. Whatever it is, it's especially hard to get rid of, too.

I'm not sure what MP3 tagging program you're using to do your editing, but certain ones (like Mp3tag in Windows) have options to strip out APE tags, etc....it would probably be helpful to some Linux users if you enabled those options. It won't fix every file (like in the Broken Mirror Reflections file), but it'll help.

[Liontamer]

Okay, just to test, I opened up a file in winamp, set its track number to a five digit number with a leading zero (01752), and brought it back into iTunes. It displayed correctly, although without the leading zero (1752) (but it didn't actually remove the leading zero, it just wouldn't display it). So it can read extra digits, it just won't let you make them. Incredibly stupid, but not an actual problem as far as this goes.

Well, that may mean doing track # is still a viable option. I'll have to look into it more, especially how it would sort with different digits (e.g. 00001, 00010, 00100, 01000). Thanks for checking that!

Incidentally, as you might guess from the ID #, I was testing with my own ReMix, and I happened to notice that the "Encoded By" field contains the email address from which I submitted the mix. Not a problem for me personally, but is this necessary/desirable? I certainly wasn't aware of this practice when I submitted, and it might be an unpleasant shock to those who prefer to keep their email address a closely guarded secret.

I generally add some sort of data in that field, but I never add contact/site/name in that field if I know an artist wants it hidden. If they provide that information in a sub letter, it's safe to say they're OK with the info out there, but I appreciate the sentiment.

I've just thought of a suggestion...it's kind of the opposite of adding information to unused fields, but this seems like a good chance to bring it up:

Some ReMixes have APE tags embedded in them in addition to ID3 tags, and the title, artist, album, etc. are generally written incorrectly in the APE tags. Unfortunately, certain players (e.g. Rhythmbox under Linux) strangely give priority to these tags at the expense of the ID3 tags. Until I finally realized what was going on, I kept trying to change the ID3 tags to no effect, and the same wrong title/artist/album etc. from the APE tags kept being displayed in Rhythmbox. To give a few examples (by no means an exhaustive list), http://www.ocremix.org/remix/OCR01139/, http://www.ocremix.org/remix/OCR01238/, and http://www.ocremix.org/remix/OCR00191/ exhibit this problem. Actually, the last one has some other kind of superfluous tag (AAC? Some old Quicktime tagging? It's Apple something.) rather than APE, but in any case, it causes the same problem. Whatever it is, it's especially hard to get rid of, too.

I'm not sure what MP3 tagging program you're using to do your editing, but certain ones (like Mp3tag in Windows) have options to strip out APE tags, etc....it would probably be helpful to some Linux users if you enabled those options. It won't fix every file (like in the Broken Mirror Reflections file), but it'll help.

Good catch! I removed APE tags from the few mixes with them, and I also caught several with ID3v1 tags as well and removed those. Winamp 5 can remove those types of tags. If you have any other info on 'Broken Mirror Reflection' and what may be the tagging case there, let me know.

[Mini-Me]

Good catch! I removed APE tags from the few mixes with them, and I also caught several with ID3v1 tags as well and removed those. Winamp 5 can remove those types of tags. If you have any other info on 'Broken Mirror Reflection' and what may be the tagging case there, let me know.

Alrighty, here's what I've been able to figure out so far:

Three of Alexander Prievert's ReMixes have these persistent "mystery tags:"

Castlevania II: Simon's Quest 'Bloody Tears Inspired' OC ReMix

Shadow of the Beast 'Journey' OC ReMix

Xenogears 'Broken Mirror Reflection' OC ReMix

It's possible some of his other files have them too, but if so, they haven't shown any symptoms. After using EasyTAG to strip out all "ordinary" tags, the files appear to be clean of all tags in every Linux-based tag editor I've tried. (EasyTAG, MusicBrainz Picard, Audio Tag Tool, Ex Falso, and Kid3...in other words, a bunch). The "mystery tags" are still there, but none of these programs have caught on to them...however, certain music players are somehow able to pick them up.*

Anyway, using a hex editor, I was quickly able to see that the tags are related to Apple, and after searching for some strings, it appears they're old Quicktime tags...as in, the kind of tags that were intended for Quicktime movies. Unfortunately, I have no idea what program to use to gracefully remove them. I mean, I can use the hex editor itself to roughly get rid of them in a quick and dirty way, and it "works," but figuring out where exactly they actually start/end is another matter altogether. I've found a few references, but I haven't had the time to really go through them (http://developer.apple.com/quicktime/qttutorial/qtatoms.html, http://wiki.multimedia.cx/index.php?title=QuickTime_container, http://www.sno.phy.queensu.ca/~phil/exiftool/TagNames/QuickTime.html, etc.).

In Windows, Quicktime Player will show the offending tags, as long as you wipe away the ID3 tags first, but I can't find a way to edit/delete them using the free version. Maybe the Pro version can do it, but then again, maybe not...so until we find some program that will edit/delete these tags, we're stuck either ignoring them or guesstimating with a hex editor.

*To recap and add a bit more information in case you're interested:

Rhythmbox is the default music player in Ubuntu, and it's based on the GStreamer pipeline. Apparently, it displays whichever tags are last read in by GStreamer...and in certain files, those tags are APE tags or even worse, Quicktime tags. Because of that, some of these superfluous tags appear to override any existing ID3 tags. Regardless of whether the ID3 tags are intact or if I've scrubbed them, Rhythmbox keeps displaying the Quicktime tags for the aforementioned Prievert mixes. (Do we have a "shoot myself in the head smiley here?) I'm not sure just how many other players this problem crops up in though.

Other audio players seem to exhibit a couple other behaviors:

Some are completely oblivious to the Quicktime tags and try to play them as audio data (which results in a short burst of static at the beginning of "Shadow of the Beast 'Journey'" for example). Others seem to realize that the tags exist, but they aren't able to read them, so they presumably just skip over them (resulting in no static). I've noticed that at least one player seems to notice and skip the Quicktime tags if no ID3 tags are present, but if there are any ID3 tags, it will try to play the Quicktime tags as audio data.

[Mini-Me]

UPDATE:

Actually, this isn't too bad. I've read a bit about the Quicktime file format here, and I've been able to remove the Quicktime tags very precisely. The first four bytes of each "atom" give the size in bytes (as a 32-bit big-endian unsigned integer), the next four give the name of the atom, and the remaining ones are the data.

After stripping out the ordinary tags, I noticed all three files were broken into four major atoms:

1. Size = variable, name = "moov", data is the tag data (some is human readable, some is not)
   
2. Size = 8, name = "free", no data (because the four bytes for the size and the atom name added up to
   
3. Size = 8, name = "wide", no data
   
4. Size = variable (but the size of all four of these elements adds up exactly to the total filesize in bytes, assuming all other tags have been deleted), name = mdat, data = everything left in the file.

The last atom, mdat, is broken into the following three sections:

1. A small atom: Size = 8, name = wide, data no data
   
2. Some kind of prefix to the actual mp3 data: Size = 0 (?), name = mdat
   
3. The actual mp3 data!

So, long story short: After scrubbing the file of all ordinary tags, open the file up with a hex editor and search for "mdat" - there should be two and only two references to this string in the whole file. Delete everything that comes before the final mdat reference (including the mdat string itself), and it seems you're left with the raw mp3 data! I did notice that the first two bytes of the mp3 data were (in hex) "FF FB" for all three files, and the next byte was A0 for two of them. I'm not sure what the significance is there, but...from where I'm standing, the important thing is that Quicktime tags are gone and the files play without any noticeable static at the beginning (or any notes cut off or anything like that).

Of course, some files have extraneous data at the very end as well (like back-loaded ID3 or APE tags or a bunch of random references to the LAME encoder), and getting rid of the Quicktime tags won't get rid of all that, but...the Prievert files don't seem to have that problem anyway.