|
|
View previous topic :: View next topic |
Author |
Message |
thepoop555
Joined: 16 Sep 2007 Posts: 316 Location: Austin, TX
|
Posted: Wed Oct 27, 2010 2:32 am Post subject: Rock Band 3 Customs (Xbox 360) |
|
|
Hey everyone, Rock Band 3 released today. And I'm soooo glad. But then opening the box I remembered about the ScoreHero community and the Rock Band 1 and 2 customs that we all started. So I'm here to get that same spark about Rock Band 3!
This thread is about the files on disc and off, and any way to get user created into the game is accepted!
ABSOLUTELY NO TALK OF DECRYPTING/RIPPING AUDIO FILES FROM THE GAME! NONE! ANY POSTS OF SUCH WILL AUTOMATICALLY BE DELETED!
Okay, as of today (10/26/2010), the release day of the game, the files on-disc are the same (more, but they are the same format). For anyone who's wondering what the format is:
Code: | Xbox 360
$SystemUpdate
gen
AvatarAwards
charnames.zbm
default.xex
nxeart |
Inside the "gen" folder is the typical HMX game layout:
Code: | main_xbox.hdr
main_xbox_0.ark
main_xbox_1.ark
etc... |
The .hdr file is a file table of all the files that are contained inside the .ark files. The ark files hold EVERY SINGLE file that is in the game (models, charts, etc...). These files hold up to about 2 GB in files each, but sometimes they will split the files up unevenly.
The number of .ark files in Rock Band 3 is 10 (0-9).
The encryption of the .dtb and .hdr files is the same. The .hdr version in Rock Band 3 is 6. They have changed the format of the .hdr file from previous versions. We're still uncovering these new changes, but so far the research is very premature.
Post any information on this thread, and remember:
ABSOLUTELY NO TALK OF DECRYPTING/RIPPING AUDIO FILES FROM THE GAME! NONE! ANY POSTS OF SUCH WILL AUTOMATICALLY BE DELETED! _________________
SUPPORT GH: AVENGED SEVENFOLD
Arandomguy1 on YouTube
TrevorTheGreat wrote: | Oh my god I've never seen a bunch of bitching children like this before.
OMG I got perma-rockered! /world
So fucking what. Play your plastic guitar and get over it. |
Last edited by thepoop555 on Sun Oct 31, 2010 5:36 pm; edited 1 time in total |
|
Back to top |
|
|
ajanata
Joined: 07 Jul 2007 Posts: 1167 Location: South Bay Area, CA
|
Posted: Thu Oct 28, 2010 8:58 pm Post subject: Re: Rock Band 3 Customs (Xbox 360) |
|
|
thepoop555 wrote: | The encryption of the .dtb and .hdr files is the same. The .hdr version in Rock Band 3 is 6. |
I take it that you've been able to get the .dtb files out and look at the data in them? I've been trying to do so myself but that kind of analysis isn't my forte. Any tips you could share? (I need some of this data for my chart generator; Kawigi needs it for his stats too.) _________________
|
|
Back to top |
|
|
thepoop555
Joined: 16 Sep 2007 Posts: 316 Location: Austin, TX
|
Posted: Fri Oct 29, 2010 12:45 am Post subject: |
|
|
The encryption of the .hdr files have always been the same as .dtb files. So I assume that they are the same. I'm still working on getting a working ark explorer running because ArkTool doesn't work with this Rock Band. But if you have the latest dtb decryptor, you could get the file name, ark number, and file offset, etc., on your own. Tedious, but possible. _________________
SUPPORT GH: AVENGED SEVENFOLD
Arandomguy1 on YouTube
TrevorTheGreat wrote: | Oh my god I've never seen a bunch of bitching children like this before.
OMG I got perma-rockered! /world
So fucking what. Play your plastic guitar and get over it. |
|
|
Back to top |
|
|
ComicBookGuru
Joined: 16 Dec 2007 Posts: 1678
|
|
Back to top |
|
|
thesentence
Joined: 21 May 2008 Posts: 1389
|
Posted: Fri Oct 29, 2010 8:03 pm Post subject: |
|
|
I can confirm this. I've had DLC customs working for a couple weeks now. _________________
|
|
Back to top |
|
|
ajanata
Joined: 07 Jul 2007 Posts: 1167 Location: South Bay Area, CA
|
Posted: Fri Oct 29, 2010 10:53 pm Post subject: |
|
|
thepoop555 wrote: | The encryption of the .hdr files have always been the same as .dtb files. So I assume that they are the same. I'm still working on getting a working ark explorer running because ArkTool doesn't work with this Rock Band. But if you have the latest dtb decryptor, you could get the file name, ark number, and file offset, etc., on your own. Tedious, but possible. |
Cool, thanks. I'll check that out tomorrow. _________________
|
|
Back to top |
|
|
LocalH
Joined: 30 Oct 2006 Posts: 1400 Location: MiloHax
|
Posted: Sun Oct 31, 2010 2:19 pm Post subject: |
|
|
I'm trying to wrap my head around this to extract some dtb files manually, and either HMX severely changed the hdr format since v4, or the information I have is incomplete (mostly concerning the HDR header itself at this point although I haven't been working on this long). In the string table, pathnames are separate from filenames, right? So I have to determine where in the file table the dtbs are referenced (by string index number, not offset), then use that data as an offset into the entire ark set (treating it as one big file, right?)? The steps I'm thinking are right are as follows:
1) Count through string table to find the 0-based index number for the desired filenames
2) Begin going through the file table to get the ARK offset (to make sure I understand this, it seems each file table entry takes 24 bytes, 8 for ARK offset, 4 each for pathname index, filename index, file length, and a 00000000 delimiter) and name this offset X, grab the length and name it Y
3) Use hex editor to copy out bytes from offset X to offset X+Y-1 into a separate file
Anyone know if the actual DTB format is the same (32-bit key followed by payload?) I would hope so, anyway.
Quick question: Just to make sure I'm reading part of this HDR correctly - does it sound right for there to be $3E3C (15422) string entries, counting paths and filenames? Edit: Ok, I was definitely reading it wrong, but it doesn't really matter to me ATM as I'm going to be traversing this manually, but I took the string table out of the HDR and replaced $00 with $0D0A (CR/LF) - Hex Workshop did this $20F5 times (once for each string).
For anyone who might find the string list handy for digging out files yourself, I threw it up on Pastebin. The line numbers should come in really handy for locating the proper entry in the file table (just remember to subtract one from the line number listed on Pastebin, since it's 0-based). _________________
|
|
Back to top |
|
|
thepoop555
Joined: 16 Sep 2007 Posts: 316 Location: Austin, TX
|
Posted: Sun Oct 31, 2010 5:34 pm Post subject: |
|
|
LocalH wrote: | I'm trying to wrap my head around this to extract some dtb files manually, and either HMX severely changed the hdr format since v4, or the information I have is incomplete (mostly concerning the HDR header itself at this point although I haven't been working on this long). In the string table, pathnames are separate from filenames, right? So I have to determine where in the file table the dtbs are referenced (by string index number, not offset), then use that data as an offset into the entire ark set (treating it as one big file, right?)? The steps I'm thinking are right are as follows:
1) Count through string table to find the 0-based index number for the desired filenames
2) Begin going through the file table to get the ARK offset (to make sure I understand this, it seems each file table entry takes 24 bytes, 8 for ARK offset, 4 each for pathname index, filename index, file length, and a 00000000 delimiter) and name this offset X, grab the length and name it Y
3) Use hex editor to copy out bytes from offset X to offset X+Y-1 into a separate file
Anyone know if the actual DTB format is the same (32-bit key followed by payload?) I would hope so, anyway.
Quick question: Just to make sure I'm reading part of this HDR correctly - does it sound right for there to be $3E3C (15422) string entries, counting paths and filenames? Edit: Ok, I was definitely reading it wrong, but it doesn't really matter to me ATM as I'm going to be traversing this manually, but I took the string table out of the HDR and replaced $00 with $0D0A (CR/LF) - Hex Workshop did this $20F5 times (once for each string).
For anyone who might find the string list handy for digging out files yourself, I threw it up on Pastebin. The line numbers should come in really handy for locating the proper entry in the file table (just remember to subtract one from the line number listed on Pastebin, since it's 0-based). |
That method is the tedious method, but works. And what the problem is is for me is figuring out the new hdr file format. They changed that quite a bit.
The header now contains all the paths to each ark file on the disc. So it looks like there is a dtb node header in the beginning now. 0x0A. I looked it up and that signals a tree node, but it doesn't follow the typical pattern...
Also bytes 0x8 - 0x17 make no sense to me. I'm thinking maybe like a hash of some sort? Idk, hashes are not my strong suit...
And DLC customs are angering me, they don't show up. But then again I downloaded something from Rock Band 2, and that didn't show up. The only thing that shows up on Rock Band 3 as DLC is the Avenged Sevenfold Track Pack...
Anyway that's my progress so far with the on-disc stuff... _________________
SUPPORT GH: AVENGED SEVENFOLD
Arandomguy1 on YouTube
TrevorTheGreat wrote: | Oh my god I've never seen a bunch of bitching children like this before.
OMG I got perma-rockered! /world
So fucking what. Play your plastic guitar and get over it. |
|
|
Back to top |
|
|
LocalH
Joined: 30 Oct 2006 Posts: 1400 Location: MiloHax
|
Posted: Sun Oct 31, 2010 5:50 pm Post subject: |
|
|
thepoop555 wrote: | The header now contains all the paths to each ark file on the disc. So it looks like there is a dtb node header in the beginning now. 0x0A. I looked it up and that signals a tree node, but it doesn't follow the typical pattern... |
I noticed the ARK paths, however I didn't make it out to be part of a DTB file (but TBH, after tools became available to convert between DTB and DTA, I gave up on the "node" based tools, so I'm not as knowledgeable on the DTB binary format). Keep in mind that, at least under previous versions of HDR, there's supposed to be a count of the number of ARK parts somewhere, and there are 10 ARK parts (which of course is 0A). I did notice that there was some odd stuff between 0x132 and 0x164, is that what you're referring to? If so, then I can just ignore it for this, as I am the entire string offset table (which seems so far out of whack from what older HDR v3 and v4 documentation states that I'm not even touching it, luckily I don't need it to do this manually).
Quote: | Also bytes 0x8 - 0x17 make no sense to me. I'm thinking maybe like a hash of some sort? Idk, hashes are not my strong suit... |
I noticed the same thing, and they almost did seem either like a hash or checksum. Not my strong point either (I've spent several hours right now just to get to this point, it's almost like the old days of GH2 hacking again). _________________
|
|
Back to top |
|
|
thepoop555
Joined: 16 Sep 2007 Posts: 316 Location: Austin, TX
|
Posted: Sun Oct 31, 2010 8:51 pm Post subject: |
|
|
Ok, well after the so called "hash," there's the number of arks, a copy of that number, and then the sizes of the arks in order follow. So 0x20 to 0x48 are the sizes of the arks in order. Then it repeats the number of arks once more, and then a string table of the ark paths follow; the length of the file path and then the string itself.
EDIT: Found the size of the string table. A 4 byte integer at the offset 0x15E. I'm now going to start looking for the size of the data table at the bottom. Hopefully should be similar to the previous ark versions. _________________
SUPPORT GH: AVENGED SEVENFOLD
Arandomguy1 on YouTube
TrevorTheGreat wrote: | Oh my god I've never seen a bunch of bitching children like this before.
OMG I got perma-rockered! /world
So fucking what. Play your plastic guitar and get over it. |
|
|
Back to top |
|
|
LocalH
Joined: 30 Oct 2006 Posts: 1400 Location: MiloHax
|
Posted: Mon Nov 01, 2010 12:26 am Post subject: |
|
|
I don't know if you have this or not, but it might help a little bit (valid for HDR v4, as seen in RB1):
Code: | /// ARK header format:
/// OPTIONAL: 4 bytes - Encryption key. If present, the rest of the file is encrypted with the routine
/// found in dtb_xor_360. We assume if this value is a version we know about, it's not present.
/// 4 bytes: Version. Expected to be 3 or 4; these are the only versions we understand.
/// 4 bytes: Count of parts (ARK files). Eg, if this is 2, then there should be a main_0.ark and main_1.ark.
///
/// 4 bytes: Count of part length table. Should always be the count of parts, repeated.
/// --- Beginning of part length table; 1 record per count ---
/// Version 3: 4 bytes: Size of each part file (in bytes).
/// Version 4: 8 bytes: Size of each part (in bytes).
/// --- End part length table
///
/// 4 bytes: Size of string table (in bytes). *Not* a count.
/// --- Beginning of string table; number of records unspecified ---
/// VARIABLE: String table (list of null-terminated strings). Size is given in four bytes previous.
/// --- End string table ---
///
/// 4 bytes: Count of strings in string table
/// --- Beginning of string offset table; 1 record per count ---
/// 4 bytes: Offset into the string table to find start of string. This offset is from the
/// beginning of the string table, so 0 means the beginning of the string table.
/// --- End string offset table ---
///
/// 4 bytes: Count of file table
/// --- Beginning of file table; 1 record per count ---
/// Version 3: 4 bytes: Offset entry begins at (in all ARK parts, put together)
/// Version 4: 8 bytes: Offset entry begins at (in all ARK parts, put together)
/// 4 bytes: String 0-based index (from string table) for file name; 3 means "4th string"
/// 4 bytes: String 0-based index (from string table) for path name; 3 means "4th string"
/// 4 bytes: Length of entry
/// 4 bytes: Always 0
/// --- End file table ---
|
Problem is, starting at 0x3f2c9 (0x165+0x3f163), the data there doesn't match what the documentation I have states for the "string offset table". Unless the strings are just all over the place and not really in any order...now that I look at it more, all these values do seem to be less than the string table length...damn obfuscation...
Also, I'm unsure about the data from 0x4ebb9 to 0x4ecac inclusive...everything after that looks exactly like the HDR v4 file offset table, and even maybe the last 48 bytes of the aforementioned range look like they could be a couple of entries).
Edit 2: Wow, they seem to have really just obfuscated this, there seem to be lots of null (0x00000000) string offsets in the table, they jump around the place, and there's some odd entries at the top of the file offset table. The file count seems to be at 0x4ebb9, as when multiplying it by 24, it comes out to be the same as the length of the chunk from 0x4ebbd to EOF. The string count seems to be at 0x3f2c5, as when you multiply it by 4 and start counting from 0x3f2c9, it comes out with the last byte at 0x4ebb8, but there definitely weren't 15,000 entries in that string table, but lots of null pointers in the string offset table. One step closer to cracking this shit! _________________
|
|
Back to top |
|
|
LocalH
Joined: 30 Oct 2006 Posts: 1400 Location: MiloHax
|
Posted: Mon Nov 01, 2010 2:12 am Post subject: |
|
|
New post to trigger it to people who already read my other post.
I think I've more or less cracked the HDR format and boy, is it obfuscated. There's still some stuff I don't understand but for the most part, I think this might be enough to traverse the file table, extract the ARK offsets and lengths for each file, index into the string offset table to retrieve the file and path name, create directories as necessary, and then write out each file in order. The first 8 entries in the file table seem to be nulled out (they reference 0-size files), so they can be skipped.
Code: | HDR breakdown
-------------
Offset End (incl) Data
-------- -------- --------
00000000 00000003 HDR version ($00000006)
00000004 00000007 Unknown (supposed to be # of ARK parts, but there are 10 and this says $00000001)*
00000008 00000017 Unknown (possibly a hash of some sort, 43B2CBDE 347991F6 23860C6C 48BFF67F)
00000018 0000001B ARK part count ($0000000A = 10)
0000001C 0000001F ARK part length table count (supposed to be, and is, same as part count, $0000000A)
00000020 00000047 ARK part length table, 10 x 4 bytes
00000048 0000004B Unknown, seems to mirror part count and part length table count ($0000000A)
0000004C 0000004F Filename length ($00000013), goes with the following ARK part filename
00000050 00000062 "gen/main_xbox_0.ark"
00000063 00000131 Same as previous two entries but for ARK parts 1-9
00000132 00000135 Unknown (contains $0000000A)
00000136 00000139 Unknown (contains $00000002)
0000013A 0000013D Unknown (contains $00000003)
0000013E 00000141 Unknown (contains $00000002)
00000142 00000145 Unknown (contains $00000000)
00000146 00000149 Unknown (contains $FFFFFFFF)
0000014A 0000014D Unknown (contains $00000001)
0000014E 00000151 Unknown (contains $00000004)
00000152 00000155 Unknown (contains $FFFFFFFF)
00000156 00000159 Unknown (contains $FFFFFFFF)
0000015A 0000015D Unknown (contains $FFFFFFFF)
0000015E 00000161 String table length, in bytes, including string count ($0003F163)
00000162 0003F2C4 String table
00000162 00000163 Unknown (contains $2E00 but seems to be part of string table)
0003F2C5 0003F2C8 String count, including nulls ($00003E3C)
0003F2C9 0004EBB8 String offset table
0004EBB9 0004EBBC File count ($00001EBA)
0004EBBD 0007CD2C File offset table (and EOF)
*possibly a subversion of the file format, anyone got TBRB or GDRB to check this value and see if it's 0?
|
I have no way to test this, as my programming skills are limited to 68k ASM (as you can see by my use of $ rather than 0x in my notes). I did the math, however, and everything clicks perfectly. String offsets are 4 bytes, 0x3e3c x 4 = 0xf8f0 bytes calculated for the table, 0x4ebb8 - 0x3f2c9 + 1 = 0xf8f0 bytes actually used. File table entries are 24 bytes each, 0x1eba x 0x18 = 0x2e170 bytes calculated for file table, 0x7cd2c - 0x4ebbd + 1 = 0x2e170 bytes actually used. _________________
Last edited by LocalH on Mon Nov 01, 2010 6:01 pm; edited 1 time in total |
|
Back to top |
|
|
thepoop555
Joined: 16 Sep 2007 Posts: 316 Location: Austin, TX
|
Posted: Mon Nov 01, 2010 5:11 am Post subject: |
|
|
Ok so lemme get this straight. There are a bunch of null file offsets in the 3rd (file info) table? Like just 24 null bytes? And then the number of strings in the string offset table includes those? Or not? Because unless I'm screwing up big time, there are 15932 (0x3C3E) entries in the string offset table? And there are acutally only 7866 (0xBA1E) entries in the file offset table, but that has null entries in it? Or am I completely confused? This is obviously a little different the the v3 and v4 hdr files... _________________
SUPPORT GH: AVENGED SEVENFOLD
Arandomguy1 on YouTube
TrevorTheGreat wrote: | Oh my god I've never seen a bunch of bitching children like this before.
OMG I got perma-rockered! /world
So fucking what. Play your plastic guitar and get over it. |
|
|
Back to top |
|
|
LocalH
Joined: 30 Oct 2006 Posts: 1400 Location: MiloHax
|
Posted: Mon Nov 01, 2010 10:57 am Post subject: |
|
|
No, there are a bunch of null string offsets in the string offset table (the middle section). File table doesn't look like it has any nulls out of place (except for the aforementioned first 8 entries).
Maybe a few screenshots might help, don't feel like formatting a hex dump in a code tag. If it weren't copyright infringement I'd just post the extracted tables in raw binary form =P
Note about the screenshots - I set 00 values to display as red (and as hash marks on the ASCII side) and turned on "flip bytes in editor" in Hex Workshop, but in the version I'm using that feature's a bit buggy, so you'll see values on the left hand side be colored red that aren't 00, due to the reordering. AFAIK, all these values are supposed to be big-endian, and that's the only way they make sense to me.
Pardon me if the images are too wide, I cropped it down quite a bit from my desktop size =P
There's the first 0x300 of the string offset table. Notice how many 0x00000000 entries there are, and if you dig between 0x3f2c9 and 0x4ebb8 you shouldn't see any values larger than 0x3f162, the value you found at 0x15e (in fact, I would expect the largest value to be 0x3f14d, the start of the last string).
First 0x300 of file table, notice how neat it looks at 24 bytes per row. Also note that I have seen offsets in the file table use the upper 32 bits to store a 1, since the total ARK size is larger than 4GB. 5 fields, 24 bytes per entry. _________________
|
|
Back to top |
|
|
MahoppianGoon
Joined: 18 Mar 2008 Posts: 37
|
Posted: Mon Nov 01, 2010 5:17 pm Post subject: |
|
|
Well, that was fun! :P Ok, now I'm stuck. I got all '7858' files, 'offsets/sizes', but probably missed a few, but how do you get the filenames for each offset? The filenames at the top of the hdr file are not in the same order as the offsets at the bottom, unless I'm not reading it right. It seems each file-entry at the bottom is 24-bytes long. The first 8 is the offset, (plus 0x322), I don't know why you have to add that. The next 8 is unknown? Maybe the ones that tell you which filename that offset goes too? and the last 8 is the filesize. Heres an example:
8790D2C600000000 (C6D29087+322=C6D293A9)
021900006B370000 (???)
0412040000000000 (41204, normal, no reversing)
and that offset number is the offset if ALL the ark files was combined.
So guess which file that is ^? You'll NEVER guess in a million years! lol
Yep. the 'songs.dtb' file. (it decrypts fine using the dtbcrypt 3.0)
So who wants it? lol I'll give you a hint, its in the 'main_xbox_7.ark' file
Ok, fine, heres the exact location:
offset: 0x018FD4FF
size : 0x41204
first4: 594B0000
last 4: 0A0CAFE4
Have fun! |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Copyright © 2006-2024 ScoreHero, LLC
|
Powered by phpBB
|