Talk:ITunesDB
From iPodLinux
Sample code
Sample code in C++
If you need some working sample code, might check out http://ottodestruct.com/itunes/iPodDB.zip to see how I did it. This code works and is being used in the foo_pod plugin for the foobar 2000 audio player, but it's not really wholly suitable for use on the iPod itself. For one thing, it's a massive memory hog and somewhat slow. For another, it's not particularly easy to use or anything. I wrote it mainly to spur development of the plugin for foobar, and to demonstrate how the layout of the iTunesDB file works.
It's feature rich, but somewhat haphazardly organized. Still, it's useful to see how each section of the iTunesDB is parsed because of the object structure and so forth, so it makes for a good example code to use to develop your own code. It supports everything documented here, with parsing and writing capabilities. There's also about a zillion functions for doing things like inserting tracks, modifying playlists, converting OTGPlaylists into normal playlists, populating smart playlists from it's set of rules, etc. So if you need to know how to do something, I highly recommend it.
The one thing that it does very well is to emulate iTunes exactly. If you parse an iTunesDB made by iTunes into the structures and then write it out again, you'll get the exact same file, bit for bit. This took some doing. ;)
Note that this code may be out of date, but it should take only minor changes to get it working again.
-Otto
Sample code in Java
A similar project can be found at http://www.math.tu-berlin.de/~tbinder/jpot/. It is written in Java. Currently, it has roughly the same functionality as Otto's code. Check the web page for latest news.
-Tom
... also includes a python class to universally parse (and write!) the iTunesDB and related formats, and copy photos onto the ipod.
-- Henryk 05:54, 18 Aug 2005 (CDT)
Another Java project http://mediachest.sf.net
The information on the iTunesDB about the type 2 and 3 MHSDs is correct. I've seen iTunes do it. I'm uncertain as to why you're getting all songs in both of them, but I cannot reproduce that. All the files sent to me that were made by iTunes show the Type 3 MHSD only having the podcast list in it.
Might want to check your files again. Or try different iPod types for comparison purposes.. Or something. Clearly we may not know everything about this yet.
But in any case, it makes really no sense to duplicate all the playlists in both places. You must be mistaken.
-Otto
4G ipod, iTunes 4.9: all playlists show up in both MHSDs. The difference is exactly as WossName indicated above. It kind of makes sense: iPods that are unaware of MHSD type 3 simply show a playlist named 'Podcasts' because they read the type 2 only. iPods that know about the MHSD type 3 will read type 3 only.
-jcsjcs Tue Sep 20 00:21:08 JST 2005
Glad I found this. I was expecting one and only one library playlist, but when examining type 52 MHOD's found two of each. I logged the offset of every 52 MHOD of each sort type.
| Sort | Offset 1 | Offset 2 |
| 0x03 | 272180 | 393726 |
| 0x04 | 275588 | 397134 |
| 0x05 | 273884 | 395430 |
| 0x07 | 277292 | 398838 |
| 0x12 | 278996 | 400542 |
All offsets are number of bytes from start. iTunes 4.9/db ver 0x0D.
Cyresse 06:10, 20 Sep 2005 (CDT)
I would like to split mhyp's 'Library MHOD Count' into two 2-byte fields. Like wossname I found that when setting the four Bytes to 0x00010001, the playlist will not be displayed in the Playlist section. If I set two playlists to 0x00010001, no Podcast playlist will be shown, and neither playlists will be displayed under Playlists. Normal playlists (iTunes 4.9) have 0x00000001 set, the Podcast playlist has 0x00010001 set.
My iTunesDB does not have type 52 MHODs, by the way. Changing the lower 2 bytes to '0x0000' or '0x0002' did not have any effect. Also, my iPod (nano) did not appear to read the mhsd type 2 (last of the three mhsds).
I'd propose to call the second 2-byte field 'podcastflag'.
-jcsjcs Fri Sep 23 02:46:48 JST 2005
jcsjcs: Go for it. I may not like some of your other changes, but not enough to spend the time to remove them so far. :)
Anybody should feel free to make whatever changes they find necessary. We'll just argue about them later. I mean, the wiki has a history,it's not like we're losing anything here. :)
-Otto
Sample code in Objective-C
expod is a Mac OS X app that parses iTunesDB and allows copying files off the ipod. Source code included.
- Astroboi 19:33, 10 Oct 2006 (CDT)
ArtworkDB
Can anyone say with absolute certainty that the ArtworkDB image data is in RGB565? After playing around with this for a few hours, I believe it's not in this format, but in something else (still looking).
Yes it is. If it doesn't work for you, try swapping the bytes (so that it looks like GBRG 3553, where the first G are the lower three bits of green). For some miraculous reason the iPod seems to use both formats: I see the regular RGB565 in the Photo Database images for the normal display; and the weirdo-swapped format in the Photo Database images for thumbnails and in the ArtworkDB images.
-- Henryk 03:18, 2 Sep 2005 (CDT)
GBRG3553 would be the byte reversed version of RGB565. Think of it as two bytes: RRRRRGGG GGGBBBBB Byte reverse that and you get: GGGBBBBB RRRRRGGG
It's a little endian vs. big endian thing. They use both where it seems to suit their whims. :P
-Otto
Yay! It works! I just swapped the bytes, and then read them in as normal RGB565 and the images load! Such a simple fix. Thanks guys.
I've also found that it appears that the small artwork thumbnail shows up twice on the iPod. From what I've seen, it's more like a mhii object that contains 3 mhod's, instead of 2, in the order of small, big, small thumbnails. Not sure why they'd include it twice... it looks exactly the same.
So I gather "swapped bytes" in the RGB565 description means that data is stored little-endian in the file? I'd rather see little-endian in the description than "swapped bytes", at least little-endian is unambiguous.
A Little Light on unk19
In trying to figure out what Apple have done in iTunes 5 by now offering the user the ability to manual set the "bookmark-ability" of files, I found that the changes happen in unk19 in iTunesDB.
There are indeed 4 independent byte-sized flags.
Flag 1 always seems to be 0x02. Flag 2 sets "Skip When Shuffling" (0x01 is excluded, 0x00 is included) Flag 3 sets "Remember Playback Position", ie bookmarkability (0x01 is bookmarkable) Flag 4 - don't know.
Note that Protected AAC files (.m4b extension) and Audible files (.aa extension) DO NOT SET flags 2 or 3, and yet are always bookmarkable and are not included in the song shuffle.
To determine if a file is bookmarkable, therefore, check the file type first. If it's not an .m4b or .aa, then check flag 3 in this field in iTunesDB.
Well, don't just document this on the talk page... If you figure out changes, add them to the page itself. Go ahead and split unk19 up into 4 single byte fields and explain the meaning of each one, as best you can. That's kinda the point of making this whole thing on a wiki. :)
-Otto
Flag 4 can probably be most adequately described as 'Podcast flag'. When the flag is set then the artist is not shown on the "Now playing" page (e.g. it displays title - album instead of title - artist - album). Also: when Flag 4 is set to 1 and Flag 1 is set to 2 there will be an additional sub-page (or whatever these different views that you can switch with the center button are called) available on the "Now playing" page with some info. Title and runtime in the case of ordinary tracks. (And IIRC also some date with Podcasts, but I can't test that right now.)
-- Henryk 21:19, 22 Sep 2005 (CDT)
Nano and iPod's name
With previous iPods, iTunes would read the iPod's name from the DeviceInfo file located at /iPod_Control/iTunes/DeviceInfo. However, this file doesn't exist on nano's, at least not on mine. I've searched, but can't find where iTunes stores the name. Does anyone know where it stores the name?
When you edit the name, iTunes 5 updates the name of your iPod in the filesystem to this new name (works same way on Mac and PC). Usually it'll update the DeviceInfo file too, but that's not created. I wonder if somewhere on the iPod it stores the casing of the characters of the iPod's name, because those are remembered, even going from Mac to PC and back. But when you're on the iPod, About > Settings shows the name as full caps, i.e. FAT32. Hmmm...
I *think* (ie it needs to be checked) that the master playlist is given the same name as the iPod name by iTunes. Did you also check with an hexeditor whether the filesystem label has the correct case or whether it's all upper-case?
Yep, you're right: The name of the library playlist is used as the name of the iPod in iTunes (it's treated as some kind of fancy playlist, after all). Also: in the filesystem label the name is all uppercase (and of course truncated to 8+3).
-- Henryk 21:44, 22 Sep 2005 (CDT)
New file: iTunesPodcasts
It's a 16 byte long file -- I have two samples:
01ce f631 8982 a7d6 1fce f631 8982 a7d6: ...1.......1....
(2 podcast subscriptions with 5 podcasts in total)
3325 bd9a 06de 7ceb 8325 bd9a 06de 7ceb: 3%....|..%....|.
(2 podcast subscriptions with 2 podcasts in total -- completely different user)
Any ideas would they could stand for?
-jcsjcs Tue Sep 20 23:07:29 JST 2005
Is it always 16 bytes? What if you had three podcasts subscribed?
-Otto
01ce f631 8982 a7d6 1fce f631 8982 a7d6 ...1.......1....
57dc c9b1 2bf5 ca00 W...+...
(3 podcast subscriptions with many podcasts in total)
I found this in an mhits at offsets 112 and 168 (dbid and dbid2 of track)
dbid dbid2
07ce f631 8982 a7d6 iden
30ce f631 8982 a7d6 iden
32ce f631 8982 a7d6 31ce f631 8982 a7d6
33ce f631 8982 a7d6 iden
2adc c9b1 2bf5 ca00 iden
4edc c9b1 2bf5 ca00 iden
6adc c9b1 2bf5 ca00 iden
the following are dbid and dbid2 from non-podcast tracks
6edc c9b1 2bf5 ca00 4bdc c9b1 2bf5 ca00
6fdc c9b1 2bf5 ca00 65dc c9b1 2bf5 ca00
70dc c9b1 2bf5 ca00 68dc c9b1 2bf5 ca00
71dc c9b1 2bf5 ca00 69dc c9b1 2bf5 ca00
in contrast to what the wiki says, dbid2 is not always the same as dbid. Maybe a reference to a different ID in the Library database?
The interesting thing is that the DBIDs mentioned in the iTunesPodcasts file do not appear anywhere in the iTunesDB.
-jcsjcs Fri Sep 23 02:00:39 JST 2005
Yeah, I think they changed the meaning of the dbid2 field, or at least they changed the way that the DBID's are structured with the introduction of podcasts.
Perhaps it's a "parent id" now, or something to that effect. I dunno. Clearly it is an auto-incrementing ID of some type, it's the relationship we're not seeing.
-Otto
Offset indicator??
Why in the heck was this added? What's the point? To make it harder to update the wiki? Because now if the size of a field is determined to be incorrect, the offsets have to be adjusted for the whole table.
I don't like this change and I say it should be changed back, by having these offsets removed.
-Otto
I was wondering about that when it went in. I'm in two minds, it looks like it could be a useful reference but definitely looks like a pain to update. Not having used the tables extensively myself I'd be happy to defer to those that have.
Sorry if I offended anyone, but I thought it was a no-brainer and useful for everyone looking at the iTunesDB in hexdump format. I desperately needed this reference because of the way libgpod is reading the iTunesDB.
I disagree with the notion that the offset indicator makes updating the tables more difficult because all offset indicators would have to be adjusted if a single field had the wrong length. For example's sake let's say the 'unknown' field in the mhsd hunk turns out to be one field 4 bytes long, but 2 fields with 2 bytes each. This does not change the offsets of 'version', 'number of children', 'id' and the other 'unknown'. I don't think that we'll find that the unknown field is 6 bytes or 2 bytes total instead of 4 bytes. I can only imagine splits or mergers of neighboring fields. I've verified most offsets in libgpod, and they seem to be correct, judging from that the parsing of the iTunesDB is successful and the result is plausible.
I'll be more than happy to help out adjusting the offsets should this become necessary. Just drop me a note to jcsjcs at users sourceforge net.
What if they add a field in the middle of the thing? They've done it before, you know.
I just think adding this info is pointless, is all. It adds a bunch of what seems to be really unnecessary information. Yes, I know how libgpod (formerly internal chunks of gnupod) works, but really, using fixed offsets is a bad, bad way to code the thing anyway. Read it sequentially, like from a stream. Parse it like you would HTML. Heck, read the thing into a big struct with the right sizes and such if you want (which is functionally equivalent to offsets, but simpler to understand). Whatever.
But storing the thing in a block of memory and then referencing it by offsets just seems like really poor style, and I'd rather not encourage it. That's one reason I didn't use the gnupod iTunesDB code when I was trying to figure out this stuff. It was way too hard to understand because of this sort of thing.
-Otto
I am new to this but I happen to find the offset really useful - I don;t have to add upp all previous bytes to find a certain piece of info - iIcan seek to the offset and I'm good to go. I love it - my 2c -Paul
String MHODs
I received via email, does it make sense to those that understand iTunesDB?
The MHOD's type fields of type 1 and 3 are only 2 bytes and the following 2 bytes are the number of 0x00's which needs to be added to the string until this is true: str.len%4 equals 0 means in utf8 if you have a string like "bye" this need to be: "bye"+0x00 else the iPod wont be able to parse the file.
Thanks to http://openphotopod.sf.net
--leachbj
That might make sense for ArtworkDB MHODs, but it's definitely not true for iTunesDB MHODs. I've never, ever, seen a UTF8 string in an MHOD in the iTunesDB file. When I've attempted to put them there, it didn't work in usually rather spectacular ways.
Since I lack an iPod Photo myself, I've not tried to change any of the stuff in the ArtworkDB Sections. Those need their own MHOD entry added, I think, since the MHODs there clearly are not identical to the MHODs in the iTunesDB.
But what he's saying is simple, actually. You have to null pad the UTF8 string to a 4 byte block, then add the amount of padding you used (in bytes) as the top two bytes of the type field. Which only goes to show just how different these various MHODs are, really.
-Otto
I've never, ever, seen a UTF8 string in an MHOD in the iTunesDB file. When I've attempted to put them there, it didn't work in usually rather spectacular ways.
I still believe that the encoding is 'somehow' determined by one of the fields. I just did another experiment on the version 0xd iTunesDB on my color iPod and this time it seemed as though I have to put 0x2 in the field before the length field to get it decoded as UTF-8 (although then the first letter was cut off and there was some garbage at the end of the string).
Regarding the ArtworkDB/Photo Database: Yes, all objects must start on an offset that is a multiple of four. For the simpler objects that is trivial, because their header length is defined as some multiple of four, but strings need to be padded at the end. I did not notice before that the padding length is encoded in (what I believed to be) the type field, but now that leachbj said it: yep, that makes sense. It's probably not necessary (since the length of the padding is implicit by total length of the MHOD and length of the string) and I have successfully written Photo Databases without knowing about this detail, but a good thing to know.
-- Henryk 17:41, 21 Sep 2005 (CDT)
Henry: I added a new Data Object placeholder in the ArtworkDB area. Feel free to document what you know about the ArtworkDB MHODs there so we can all see it. Maybe we'll figure out something new from it and be able to combine these together.
- Otto
Regarding the format of strings. For what it's worth, the UTF-16 strings appear to be UTF-16le. How significant is that?
- Cyresse
Jup, they most probably are. I seem to have forgotten to mention that all the way long, but at least I documented it in the description of the ArtworkDB MHODs.
--Henryk 17:55, 3 Oct 2005 (CDT)
Do these UTF16 entries have a BOM in front of them?
--Otto
Has anyone figured out how to determine if a string is UTF-8 or UTF-16? I noticed that for all of the strings I've come across, any one string which is UTF-16 has it's high byte set to null. However, this does not necessarily mean that you could just check if a string contains \0, does it? What if a UTF-16 string actually uses it's high byte, will it look exactly like a UTF-8 string then, from a byte perspective?
Are ASCII strings always encoded as ASCII without extensions? I've assumed as much so far and just read everything as UTF-8, checking for \0 inside the strings and if so, re-decoding the bytes as UTF-16. I just figured today that that way probably won't work in ALL situations. Will it?
--Macke
OTGPlaylists
Since the last firmware update the 3rd Gen iPod can also create multiple OTGPlaylists. But i don't know whether the 1st and 2nd Gen iPod can do this.
--Der Papst 05:36, 10 Feb 2006 (CST)
Error in mhyp
On Feb 1 2006 aero added a 4-Byte unknown value (Library MHOD Count at offset 44) to the mhyp structure. This creates a number of errors. My thoughts:
- there are indeed 4 zeros at offset 36
- offset 40 is 2 bytes and could the the string MHOD count
- offset 42 is 2 bytes and is definitely the podcast flag
- offset 44 is 4 bytes and is definitely the sort order
There is no library MHOD count, and the string MHOD count is 2 bytes rather than 4.
Any thoughts? (Removed the library mhod count and modified along the info above on 20 Mar 2006)
--jcsjcs 14:40, 03 Mar 2006 (GMT)
No real thoughts except to say that he must be wrong to have actually added 4 bytes into the middle of the thing, as existing parsers already worked, therefore shifting the entire mhyp forward like that is definitely wrong. I'd say to revert it back with a note leaving info about this change there, until such time as somebody verifies it or corrects it completely. Leaving demonstrably wrong info there is a bit silly.
--Otto
Database Object IDs
The database object IDs of MHOD, MHIP, etc. seem incorrect to me. It's stated that the order of the fields are in little endian order, and this should apply to the object IDs as well. They are in the format of xxHM, where xx is a 2-letter abbreviation of the type. I'm not sure what HM stands for, but I'll guess it stands for Header Marker.
MHBD -> DBHM = Data Base Header Marker MHSD -> DSHM = Data Set Header Marker MHLT -> TLHM = Track List Header Marker MHIT -> TIHM = Track Item Header Marker MHLP -> PLHM = Playlist List Header Marker MHYP -> PYHM = PlaYlist Header Marker MHOD -> DOHM = Data Object Header Marker MHIP -> PIHM = Playlist Item Header Marker
Curly 00:38, 4 Apr 2006 (CDT)
You are correct, those are byte reversed as well, and your abbreviations are more or less correct. Where do you think I got those names from to begin with?
However, it's more convienent to leave them in byte reversed order in discussion about them for a couple of reasons: a) People looking at the file for the first time will see MHOD more easily and know exactly where to go to in these docs, and b) The reversed versions are more easily pronouncable. :)
Examples of b:
MHOD vs. DOHM MHIP vs. PIHM MHIT vs. TIHM
And so forth.
Also, I believe the "M" to be a leftover from the "ATOM" name instead of actually standing for something, as these are ATOMs in Apple's definition of the terminology.
iTunesDB sample
I'm looking for sample iTunesDB files. Anyone care to upload theirs? This is my iPod shuffle file.
I'm trying to find out a ballpark figure for the filesize of iTunesDB on the average iPod.
Anyone knows where to look for detecting the device type, e.g. 3rd gen iPod, etc ... What about firmware, where can we find out the firmware version?
To detect device type, look at Generations
abuAbu 10:49, 3 Jul 2006 (CDT)
Here is my iTunesDB from my iPod Photo, from iTunes v7.0.
Database starting identifier
Hello there
I'm actually developping a .NET IPod manipulation library (thanks to this wiki for the docs :). I'm starting with database writing debugging (testing on a 5G, dbversion 0x11) and I found a 8 bytes field that is not mentioned here. It can be found at (decimal) offset 36 of the MHBD header.
After some research, I think that this value is a sort of starting point for the items dbid generation, as it's value is just under my database's items dbid. Did someone already noticed this value, and / or has some additional information about it ?
Hi,
i tried the following several times with my 3G iPod and iTunes 6.0.5.20.
- restore the iPod
- connect to iTunes and sync 1 title
- inspect the id you mentioned and the dbids of the track item
I found out:
- the id inside the header has nothing to do with the dbid of the track!
- th dbid (offset 112) and the dbid2 (offset 168) are not identical!
Maybe this should be corrected if there is no veto from any other.
I'll have a closer look on my database as soon as my little database explorer program compiles again :). I don't think that the fact that the database header ID I found was like 764 476 476 1 and the smaller dbid being 764 476 476 2 was a coincidence, but I'll make sure I have the same results after some ITunes manipulations. Maybe that field is useless on 3G IPods ?
[UPDATE] : I just made an upgrade from version 1.1 to 1.1.2 and noticed several things :
- In v1.1, dbid and dbid2 are identical, but they differ after formatting the ipod with v1.1.2. The dbid seems to be randomly generated for each item, and dbid2 seems to follow a sequential order as observed previously. Maybe the brevious behavior was a bug ?
- After formatting the IPod with v1.1.2, the field I found in the database header and the items dbid was absolutely different, so thit field definitely not serves as a starting identifier. However, I noticed that it has exactly the same value before and after the formatting ! Therefore, this field may be related to an information such as the users's computer, serial number, or something like that (I have a value of 5C 0A 0B 8E 52 EA AB 3E, decimal 4515960691660360284).
Info on the ithmb files
This might merit a section on the main page, but I wanted to pass the idea through here first.
Anyway, I just maxed out my iPod with files to see how large the ithmb files get (5th gen. iPod). The maximum file size seems to be 524,320,000 bytes, or 6554 movies (assuming the 80,000 bytes per thumbnail).
After that, I deleted a song that I knew had a thumbnail stored in the F1029_1.ithmb file. The 80,000 bytes were removed from that file, and the F1029_2.ithmb was left untouched.
Finally, I readded the same song to place it at the end of the iPod's database. Checking the file sizes showed that the F1029_1.ithmb was chosen to add the final 80,000 bytes back into the file. This shows me that iTunes searches for the first ithmb file that has not yet reached the maximum file size - I originally suspected it would used the same ithmb file as the previous file in the database.
imikedaman 15:26, 9 Aug 2006 (EST)
New SPL action and field introduced with iTunes 7.0
There appears to be a new SPL action (and field) connected to the new "Video kind is Music Video" kind of splaylists. Normally, I would just document it but I'm having a hard time explaining it.
Here's a summary of what I've found regarding the new "Video kind" field :
- The field has a value of 0x3C (decimal: 60)
- The action (strangely) is always 0x400 (decimal: 1024), no matter whether the rule is negated or not (the from and to values vary for each of the 6 newly introduced types of playlists)
Following is a table containing the from and to fields for every kind of "Video kind" splaylist:
| Type of rule | From and to value | From and to date | From and to unit |
| Video kind is movie | 0x02 (same as media type) | 0 | 1 |
| Video kind is music video | 0x20 (same as media type) | 0 | 1 |
| Video kind is TV show | 0x40 (same as media type) | 0 | 1 |
| Video kind is not movie | 0x0E60 | 0 | 1 |
| Video kind is not music video | 0x0E42 | 0 | 1 |
| Video kind is not TV Show | 0x0E22 | 0 | 1 |
That's about as much as I know about this new splaylist. If anyone can paraphrase what I've found so that it's easier to understand, please do so and add it to the main page :)
Archivator 13:38, 16 Sep 2006 (CDT)
I forgot to state the obvious - the sum of the values in every is - is not pair is 0x0E62 (binary: 0000 1110 0110 0010; decimal: 3 682)..
Archivator 04:26, 17 Sep 2006 (CDT)
I've looked at this as well and came to the conclusion that the new action is a binary AND operation on the mediatype. The fact that there are some bits not yet accounted for hints at the planning on future extensions video media types.
jcsjcs Sat Mar 24 16:16:55 JST 2007
some fixes mainly with IDs
Hi,
there are a few corrections on the ids.
1.) they hardly seem randomly generated. i transfered 3 videos to an empty ipod and got some ids. then i restored the ipod and then again transfered the same 3 songs and the ids generated were the same as before so they can't be random.
2.) mhit::dbid isn't incremented by one for each song.
3.) mhit::dbid2 is sometimes incremented by 3/2 other times it is totally random.
4.) mhdb offset 70 seems to indicate language set on the iPod. in my case it is en. it is not padding as mentioned
also i am trying to transfer Videos to the ipod but ipod/itunes seems to reject the file i just added and reverts to the previous version of the db (?? it somehow goes back to the exact same file as before). only thing i am not doing is adding artwork and setting the artwork count and artwork size (which by the way is diffrent size for each file).
Spliting this Page Up
This page is one of the most referenced and useful pages on the wiki, great job on all the work done on it. It is, however, getting too large for a single page. It is causing PHP out of memory errors and needs to be split up. As to how, I'm not entirely sure. I do think it should all be under the ITunesDB page, i.e subsections are listed as iTunesDB/ArtworkDB and soforth. For an example, see the OLPC HIG. As a non-contributor to this page I feel reluctant to make the changes myself, but I also think it would be a good opportunity to do a general cleanup of the information at the same time. Any thoughts? --Coob 20:17, 3 Jan 2007 (CST)
- I agree. If we can came up with a naming convention we can have separate page for every "topic" in the discussion.
- I have some questions too.
- 1. Is there a news group related to iTunesDB?
- 2. My second question is in next topic below
- Architect
- Splitting it would be reasonable, however I'm not familiar enough with the wiki tricks to take on such a task. Does anybody have a good reference to making multi-page things like this on the wiki?
- Otto 12:36, 9 Mar 2007 (CST)
- Sure. It is quiet simple to do so. If you are looking at the iTunesDB page you only have add "/ITunesDB file" to the url like so: http://www.ipodlinux.org/ITunesDB/ITunesDB_file and you have created a new sub page. After that repet that with other content too.
- Now a new table of content has to be created with internal links. A link to the example sub-page would look like: [[ITunesDB/ITunesDB_file|descriptive text]] (rendered so: descriptive text).
- Okay, I've started splitting the page in the most obvious way. This will keep the sections separate but still allow them to be part of the whole. There may be better ways to do this, but I can perceive a definite speed increase with very little effort.
- Perhaps we should consider a better overall organization scheme... Ideas? Otto 13:38, 16 Apr 2007 (CDT)
- Today I encountered one problem of this splitting: If you choose the history tab on the main page, you don't see any more changes. You've got to open the subpage first. But on the main page, it is not at all obvious how to get there. You have to edit the page to see the names of the subpages. maybe there is a way to make this easier? Anybody with Wikimedia experience?
- Abu 01:22, 25 Apr 2007 (CDT)
- I've added some links to the subpage before a subpage starts. It's surely not the nicest way. One could make every headline clickable to link to the subpage as an alternative. But that is up to you. ;-) Der Papst
- I added a subpage infobox to address this. As we split pages more, I suggest editing this infobox to keep track of them.
- Otto 13:58, 26 Apr 2007 (CDT)
Adding mp3 to iPod
I am trying to figure out how location file names are generated on an iPod for mp3 files. I am adding some mp3 from my app, but when I use my iPod it doesn't play those files. I have used bunch of other iPod "capable" apps and they show my files, so in genral DB file is ok. Does anybody has it working? --Architect 17:07, 12 Feb 2007 (CST)
A new mhsd header discovered
Hi,
today one sent me an iTunesDB that had at verry first mhsd header an mhsd type 4(!) that was structured like follows:
<mhsd>
<mhla>
<mhia>
<mhod>
<mhod>
<mhia>
<mhod>
<mhod>
<mhia>
and so on...
This iTunesDB was found on an Shuffle 2G. Has anybody seen this before?
FFNB.
The names are suggestive of "album list" and "album item" or something similar. What's the content of the mhod's?
Otto 12:35, 9 Mar 2007 (CST)
Somebody sent me such a file, too. It was from a Shuffle as well. As far as I can tell, this is created by iTunes 7.1. The file does not contain any MHSD type 2 or 3 then. wrong, see below
Abu 12:42, 12 Mar 2007 (CDT)
This is new to me. The one i have has all types (including type 4).
I have discovered the following:
The structure is verry Simple:
There is an albumlist (mhla) that contains a bunch of mhia. In fact there is an mhia for every album/artist combination on the iPod. Every mhia has 2 mhods. One of type 200 that contains the album and one of type 201 containing the artist. Further every mhia has an ID that matches the offset 286 of every track item that is member of this artist/album.
Hope this helps a little...
Here's what I have so far about the mhias: offset 12: number of children (2 or 3 observed) 16: album ID 32bit 20: dbid 64bit 28: type. 2=album, 3=podcast group 32bit The rest is zero padding. The children are string MHODs of type 200 (album name), 201 (artist name), 202 (artist name again?), 203 (podcast RSS url) For type 2, i see a 200 and a 201 and sometimes a 202. For type 3, i see a type 200 and 203.
The album ID field seems to appear at offset 288 in the MHIT of all tracks in that album. (not 286)
Question is, why on earth do we need this? Do we need to generate this at all? I thought the type 100 mhods in playlist 0 had all this info in.
Will Fisher 06:10, 19 Mar 2007 (CDT)
I have to renounce my above given statement. That iTunesDB without the mhsd 2 or 3 was corrupt. A second one from the same shuffle contained all 4 mhsd's. Abu 10:43, 27 Apr 2007 (CDT)
Question about the new MHODs "for sorting"
I've seen that a couple of new MHODS "for sorting" were dicsovered recently (Artist, The; Album; Album-Title; Composer...). Am I correct in the understanding that the iPod uses these (when available) instead of the standard Artist, Album... MHODs to sort its display. Furthermore, when we write out the libgpod we should use these instead of the standard MHODs to create the MHOD-52 sorted lists?
jcsjcs Sat Mar 24 16:29:28 JST 2007
I believe you are correct, and yes, you should probably use those to write out your sorted 52's. On the other hand, you could probably sort by anything you liked, but if you want consistency with the way iTunes seems to do it, then yeah.
Otto 13:16, 16 Apr 2007 (CDT)
Random observations from a 3rd gen Nano
Using iTunes 7.4.1, I loaded the same music as I did with gnupod, and have been looking through the differences in the two iTunesDB's to try to figure out how to get something working under Linux for my new Nano.
Here's what I've found so far:
- mhit's from iTunes have 388 byte headers, gnupod ones are 244 bytes.
- mhbd header from iTunes has an extra 84 bytes of stuff (188 bytes vs 104)
- the bodies of mhod's for track info seem to be universally smaller from iTunes (again, for the same data)
- they both create a bunch of playlists (even though I didn't explicitly create any), but iTunes creates more. Three different mhsd/mhlp's vs one.
- bunch of extra mhod's at the beginning of two of iTunes' mhlp's. (i.e. mhod mhod mhod mhod mhod... mhip mhod mhip mhod mhip mhod)
I'll be investigating more.
-mackstann 9 Sep 2007
New database format for the iPod Classic
Anybody with such a nice new device, iPod Classic with 160GB (!) HD? Apparently they changed the database format again, because it does not work when using with gnupod...
-Abu 15:46, 11 Sep 2007 (CDT)
New database format - what i have found out so far (iPod Classic and new Nano)
Here is what i found out the last days. Thanks to all the people sending me their iTunesDB Files!!
MHBD header: 0x00 4 mhbd 0x04 4 <header size = 0xBC> (changed) 0x08 4 <filesize> 0x0C 4 <unknown = 1> 0x10 4 <version number = 0x19> (changed) 0x14 4 <child count = 0x05> (changed) 0x18 8 <itunes databaseid > 0x20 2 <unknown = 2 0x22 2 <unknown = 0x0263> (changed, 0x0000 before) 0x24 8 <ipod identification?> (changed) 0x2C 4 <zero padding> 0x30 2 <unknown = 1> 0x32 20 <unknown, changing completely from itdb to itdb> 0x46 2 <language, seen: de, en> 0x48 8 <library persistent id> 0x50 4 <unknown, seen: 1, 5> 0x54 4 <unknown, seen: 0x08, 0x0D, 0x1D, 0x4D, 0x8D > 0x58 20 <unknown some similarities between versions> 0x6C 4 <timezone offset in seconds. +2*60*60 -> 0x00001C20, -4*60*60 = 0xFFFFC7C0> (really?) 0x70 76 <zero padding 0x00000000>
The problem is that the mhbd seems to be
- ipod specific: you cant copy one itunesdb file to another ipod! -> no songs listed
(the 4 bytes at 0x24 seem to be ipod unique).
- checksum/fingerprint protected: if you change a byte -> no songs listed [note: i didnt check all bytes, just random positions in mhbd and mhit) SHA1 would explain those 20 bytes we see (160bit). I was unable to reproduce that data of those 20 bytes. Another candidate might be md5.
One thing that i am thinking about is that it would make sense to protect the mhbd header by a simple checksum like crc32.
some people are discussing this on irc: freenode / #gtkpod
-Simon0 14:49, 13 Sep 2007 (CDT)
== update of the mhbd header description == i did some more comparisms:
0x00 4 mhbd 0x04 4 <header size = 0xBC> (changed) 0x08 4 <filesize> 0x0C 4 <unknown = 1> 0x10 4 <version number = 0x19> (changed) 0x14 4 <child count = 0x05> (changed) 0x18 8 <itunes databaseid > 0x20 2 <unknown = 2> 0x22 2 <unknown = 0x0263> (changed, 0x0000 before) 0x24 8 <itunes id ?! seems to depend on itunes installation/pc ?!> (changed) 0x2C 4 <unknwon, seen 0 on new ipods, 2 on old shuffle > 0x30 2 <unknown = 1 > ***zero on old shuffle*** 0x32 20 <unknown, changing completely from itdb to itdb> ***zero on old shuffle*** 0x46 2 <language, seen: de, en> 0x48 8 <library persistent id> 0x50 4 <unknown, seen: 1, 5, 9> 0x54 4 <unknown, seen: 0x08, 0x0D, 0x1D, 0x4D, 0x8D, 0x9F > 0x58 20 <unknown some similarities between versions> ***zero on old shuffle*** 0x6C 4 <timezone offset in seconds. +2*60*60 -> 0x00001C20, -4*60*60 = 0xFFFFC7C0> (really?) 0x70 76 <zero padding 0x00000000>
note regarding filesize: seems like this is not checksum protected... Because if you change the filesize to a biiig number and fix the filesize to the correct value the "startup" takes longer: 4kbyte itdb: ~6seconds 16MB itdb (by editing fsize & adding padding): ~9s 160MB (by editing fsize & adding padding): ~46s (!!) (the time after unpluging the cable while it is showing ejecting) So it looks like the ipod parses/reads the whole file.
Another interesting thing: I plugged in my old shuffle and beside the fact that itunes wiped all my music on it (waaaah) it wrote an itunesDB File with interesting content: - same 0x24 block as my nano ! - the magic/unknown (fingerprint?) bytes are 0x00 !
0x30 is 0x0000 0x32-0x45 = 0x00 0x58-0x6B = 0x00
--> it makes no sense to write there values to my shuffle as it is too old to know about the new format (and the shuffle uses iTunesSD as input, right ? So iTunesDB is completely useless)
Depending on how you interpret the single bytes it is possible that the checksum field is 42,41 or 38 bytes. (SHA1, RIPEMD-160 fingerprint would be 20 bytes, md5sum 16 byte)
--Simon0 12:50, 14 Sep 2007 (CDT)
iTunesDB parsing is forwards compatible. This is the reason for the header sizes in the records: any fields added will increase the size, so old versions of the firmware will still be able to find what they're looking for. The extra (new) data in the records just gets ignored. That's why it writes the new stuff even to the shuffle, it knows that the shuffle will ignore it.
Seems like making some sample test databases using iTunes would help to figure out the checksum. Say an iTunesDB with 1 song and another one with a different song.
--Otto 04:16, 15 Sep 2007 (CDT)
Hash generating code can be found here: http://main.wtbw.co.uk/hash58.zip
--Otto 09:05, 17 Sep 2007 (CDT)
Purpose of MHOD53 discovered: Letter jump table for fast scrolling
I just added descriptions of what the MHOD53 is good for and what it looks like. I extracted that information from an iTunes 7.3 database. Could someone please verify this is true for other DB versions, too, and since which DB version this header exists? Another thing to check would be what happens if you move them around, i.e. don't place them immediately after the corresponding MHOD52. I guess they are associated by their sort order fields, but that's just a guess - better check this. It would also be nice to know what happens with other umlauts or even stranger characters which are not expressible in ASCII, but I guess this is just UTF-16LE with 2 padding null bytes. What happens, if multiple umlauts are derived from the same character as for example in French? How are these ordered? What happens, if you flip around the entries in this index, do they get reversed when scrolling, too? -- MS 00:53, 26 Sep 2007 (CDT)
Max ITunedDB entries ?
re-directed from iPodLinux Forums
There are discussions that the maximum no. of songs that the DB can handle are 49,000. Similarly what is the max number of albums, playlists, artwork entries, podcasts etc. Anyone has these numbers? I wanted to find out the maximum theoretical filesize of itunesdb with all entries maxed out. Thanks in advance. --Karbon 03:19, 26 Feb 2008 (CST)
0x1C = iTunes 7.6.1 Databases have a new field in MHIPs
0x1C = iTunes 7.6.1 Databases have a new 8-byte field after the first 8 bytes of padding. I have not yet found out what the value means, but it seems to be incremented by 3 for every MHIP. Someone should do further investigation there... --MS 06:19, 27 Apr 2008 (CDT)
