FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
Rock Band 3 Customs (Xbox 360)
Goto page 1, 2, 3, 4  Next
 
Post new topic   Reply to topic    ScoreHero Forum Index -> Software
View previous topic :: View next topic  
Author Message
thepoop555  





Joined: 16 Sep 2007
Posts: 316
Location: Austin, TX

PostPosted: Wed Oct 27, 2010 2:32 am    Post subject: Rock Band 3 Customs (Xbox 360) Reply with quote

Hey everyone, Rock Band 3 released today. And I'm soooo glad. But then opening the box I remembered about the ScoreHero community and the Rock Band 1 and 2 customs that we all started. So I'm here to get that same spark about Rock Band 3!

This thread is about the files on disc and off, and any way to get user created into the game is accepted!

ABSOLUTELY NO TALK OF DECRYPTING/RIPPING AUDIO FILES FROM THE GAME! NONE! ANY POSTS OF SUCH WILL AUTOMATICALLY BE DELETED!

Okay, as of today (10/26/2010), the release day of the game, the files on-disc are the same (more, but they are the same format). For anyone who's wondering what the format is:

Code:
Xbox 360

$SystemUpdate
gen
AvatarAwards
charnames.zbm
default.xex
nxeart


Inside the "gen" folder is the typical HMX game layout:

Code:
main_xbox.hdr
main_xbox_0.ark
main_xbox_1.ark
etc...


The .hdr file is a file table of all the files that are contained inside the .ark files. The ark files hold EVERY SINGLE file that is in the game (models, charts, etc...). These files hold up to about 2 GB in files each, but sometimes they will split the files up unevenly.

The number of .ark files in Rock Band 3 is 10 (0-9).

The encryption of the .dtb and .hdr files is the same. The .hdr version in Rock Band 3 is 6. They have changed the format of the .hdr file from previous versions. We're still uncovering these new changes, but so far the research is very premature.

Post any information on this thread, and remember:
ABSOLUTELY NO TALK OF DECRYPTING/RIPPING AUDIO FILES FROM THE GAME! NONE! ANY POSTS OF SUCH WILL AUTOMATICALLY BE DELETED!
_________________
SUPPORT GH: AVENGED SEVENFOLD

Arandomguy1 on YouTube
TrevorTheGreat wrote:
Oh my god I've never seen a bunch of bitching children like this before.

OMG I got perma-rockered! /world

So fucking what. Play your plastic guitar and get over it.


Last edited by thepoop555 on Sun Oct 31, 2010 5:36 pm; edited 1 time in total
Back to top
View user's profile Send private message XBL Gamertag: guns4fun2872
ajanata  





Joined: 07 Jul 2007
Posts: 1167
Location: South Bay Area, CA

PostPosted: Thu Oct 28, 2010 8:58 pm    Post subject: Re: Rock Band 3 Customs (Xbox 360) Reply with quote

thepoop555 wrote:
The encryption of the .dtb and .hdr files is the same. The .hdr version in Rock Band 3 is 6.

I take it that you've been able to get the .dtb files out and look at the data in them? I've been trying to do so myself but that kind of analysis isn't my forte. Any tips you could share? (I need some of this data for my chart generator; Kawigi needs it for his stats too.)
_________________
Back to top
View user's profile Wiki User Page Send private message Visit poster's website MSN Messenger XBL Gamertag: ajanata
thepoop555  





Joined: 16 Sep 2007
Posts: 316
Location: Austin, TX

PostPosted: Fri Oct 29, 2010 12:45 am    Post subject: Reply with quote

The encryption of the .hdr files have always been the same as .dtb files. So I assume that they are the same. I'm still working on getting a working ark explorer running because ArkTool doesn't work with this Rock Band. But if you have the latest dtb decryptor, you could get the file name, ark number, and file offset, etc., on your own. Tedious, but possible.
_________________
SUPPORT GH: AVENGED SEVENFOLD

Arandomguy1 on YouTube
TrevorTheGreat wrote:
Oh my god I've never seen a bunch of bitching children like this before.

OMG I got perma-rockered! /world

So fucking what. Play your plastic guitar and get over it.
Back to top
View user's profile Send private message XBL Gamertag: guns4fun2872
ComicBookGuru  





Joined: 16 Dec 2007
Posts: 1678

PostPosted: Fri Oct 29, 2010 6:40 pm    Post subject: Reply with quote

MahoppianGoon managed to get DLC customs working a while ago, and AerialX made one for Wii I think. Fast, eh?
_________________
Yep, I learned through Rocksmith!
Back to top
View user's profile Send private message
thesentence  





Joined: 21 May 2008
Posts: 1389

PostPosted: Fri Oct 29, 2010 8:03 pm    Post subject: Reply with quote

I can confirm this. I've had DLC customs working for a couple weeks now.
_________________
Back to top
View user's profile Send private message
ajanata  





Joined: 07 Jul 2007
Posts: 1167
Location: South Bay Area, CA

PostPosted: Fri Oct 29, 2010 10:53 pm    Post subject: Reply with quote

thepoop555 wrote:
The encryption of the .hdr files have always been the same as .dtb files. So I assume that they are the same. I'm still working on getting a working ark explorer running because ArkTool doesn't work with this Rock Band. But if you have the latest dtb decryptor, you could get the file name, ark number, and file offset, etc., on your own. Tedious, but possible.

Cool, thanks. I'll check that out tomorrow.
_________________
Back to top
View user's profile Wiki User Page Send private message Visit poster's website MSN Messenger XBL Gamertag: ajanata
LocalH  





Joined: 30 Oct 2006
Posts: 1400
Location: MiloHax

PostPosted: Sun Oct 31, 2010 2:19 pm    Post subject: Reply with quote

I'm trying to wrap my head around this to extract some dtb files manually, and either HMX severely changed the hdr format since v4, or the information I have is incomplete (mostly concerning the HDR header itself at this point although I haven't been working on this long). In the string table, pathnames are separate from filenames, right? So I have to determine where in the file table the dtbs are referenced (by string index number, not offset), then use that data as an offset into the entire ark set (treating it as one big file, right?)? The steps I'm thinking are right are as follows:

1) Count through string table to find the 0-based index number for the desired filenames
2) Begin going through the file table to get the ARK offset (to make sure I understand this, it seems each file table entry takes 24 bytes, 8 for ARK offset, 4 each for pathname index, filename index, file length, and a 00000000 delimiter) and name this offset X, grab the length and name it Y
3) Use hex editor to copy out bytes from offset X to offset X+Y-1 into a separate file

Anyone know if the actual DTB format is the same (32-bit key followed by payload?) I would hope so, anyway.

Quick question: Just to make sure I'm reading part of this HDR correctly - does it sound right for there to be $3E3C (15422) string entries, counting paths and filenames? Edit: Ok, I was definitely reading it wrong, but it doesn't really matter to me ATM as I'm going to be traversing this manually, but I took the string table out of the HDR and replaced $00 with $0D0A (CR/LF) - Hex Workshop did this $20F5 times (once for each string).

For anyone who might find the string list handy for digging out files yourself, I threw it up on Pastebin. The line numbers should come in really handy for locating the proper entry in the file table (just remember to subtract one from the line number listed on Pastebin, since it's 0-based).
_________________
MiloHax 2.0 Discord server - Everything GH/RB (RIP MiloHax 1.0 01/30/2017 - 04/04/2022)
O211: An Unofficial Scorehero Alumni Discord


Alakaiser sez: POST BECAUSE YOU HAVE SOMETHING TO SAY, NOT BECAUSE YOU HAVE TO SAY SOMETHING.
Back to top
View user's profile Send private message
thepoop555  





Joined: 16 Sep 2007
Posts: 316
Location: Austin, TX

PostPosted: Sun Oct 31, 2010 5:34 pm    Post subject: Reply with quote

LocalH wrote:
I'm trying to wrap my head around this to extract some dtb files manually, and either HMX severely changed the hdr format since v4, or the information I have is incomplete (mostly concerning the HDR header itself at this point although I haven't been working on this long). In the string table, pathnames are separate from filenames, right? So I have to determine where in the file table the dtbs are referenced (by string index number, not offset), then use that data as an offset into the entire ark set (treating it as one big file, right?)? The steps I'm thinking are right are as follows:

1) Count through string table to find the 0-based index number for the desired filenames
2) Begin going through the file table to get the ARK offset (to make sure I understand this, it seems each file table entry takes 24 bytes, 8 for ARK offset, 4 each for pathname index, filename index, file length, and a 00000000 delimiter) and name this offset X, grab the length and name it Y
3) Use hex editor to copy out bytes from offset X to offset X+Y-1 into a separate file

Anyone know if the actual DTB format is the same (32-bit key followed by payload?) I would hope so, anyway.

Quick question: Just to make sure I'm reading part of this HDR correctly - does it sound right for there to be $3E3C (15422) string entries, counting paths and filenames? Edit: Ok, I was definitely reading it wrong, but it doesn't really matter to me ATM as I'm going to be traversing this manually, but I took the string table out of the HDR and replaced $00 with $0D0A (CR/LF) - Hex Workshop did this $20F5 times (once for each string).

For anyone who might find the string list handy for digging out files yourself, I threw it up on Pastebin. The line numbers should come in really handy for locating the proper entry in the file table (just remember to subtract one from the line number listed on Pastebin, since it's 0-based).


That method is the tedious method, but works. And what the problem is is for me is figuring out the new hdr file format. They changed that quite a bit.

The header now contains all the paths to each ark file on the disc. So it looks like there is a dtb node header in the beginning now. 0x0A. I looked it up and that signals a tree node, but it doesn't follow the typical pattern...

Also bytes 0x8 - 0x17 make no sense to me. I'm thinking maybe like a hash of some sort? Idk, hashes are not my strong suit...

And DLC customs are angering me, they don't show up. But then again I downloaded something from Rock Band 2, and that didn't show up. The only thing that shows up on Rock Band 3 as DLC is the Avenged Sevenfold Track Pack...

Anyway that's my progress so far with the on-disc stuff...
_________________
SUPPORT GH: AVENGED SEVENFOLD

Arandomguy1 on YouTube
TrevorTheGreat wrote:
Oh my god I've never seen a bunch of bitching children like this before.

OMG I got perma-rockered! /world

So fucking what. Play your plastic guitar and get over it.
Back to top
View user's profile Send private message XBL Gamertag: guns4fun2872
LocalH  





Joined: 30 Oct 2006
Posts: 1400
Location: MiloHax

PostPosted: Sun Oct 31, 2010 5:50 pm    Post subject: Reply with quote

thepoop555 wrote:
The header now contains all the paths to each ark file on the disc. So it looks like there is a dtb node header in the beginning now. 0x0A. I looked it up and that signals a tree node, but it doesn't follow the typical pattern...

I noticed the ARK paths, however I didn't make it out to be part of a DTB file (but TBH, after tools became available to convert between DTB and DTA, I gave up on the "node" based tools, so I'm not as knowledgeable on the DTB binary format). Keep in mind that, at least under previous versions of HDR, there's supposed to be a count of the number of ARK parts somewhere, and there are 10 ARK parts (which of course is 0A). I did notice that there was some odd stuff between 0x132 and 0x164, is that what you're referring to? If so, then I can just ignore it for this, as I am the entire string offset table (which seems so far out of whack from what older HDR v3 and v4 documentation states that I'm not even touching it, luckily I don't need it to do this manually).

Quote:
Also bytes 0x8 - 0x17 make no sense to me. I'm thinking maybe like a hash of some sort? Idk, hashes are not my strong suit...

I noticed the same thing, and they almost did seem either like a hash or checksum. Not my strong point either (I've spent several hours right now just to get to this point, it's almost like the old days of GH2 hacking again).
_________________
MiloHax 2.0 Discord server - Everything GH/RB (RIP MiloHax 1.0 01/30/2017 - 04/04/2022)
O211: An Unofficial Scorehero Alumni Discord


Alakaiser sez: POST BECAUSE YOU HAVE SOMETHING TO SAY, NOT BECAUSE YOU HAVE TO SAY SOMETHING.
Back to top
View user's profile Send private message
thepoop555  





Joined: 16 Sep 2007
Posts: 316
Location: Austin, TX

PostPosted: Sun Oct 31, 2010 8:51 pm    Post subject: Reply with quote

Ok, well after the so called "hash," there's the number of arks, a copy of that number, and then the sizes of the arks in order follow. So 0x20 to 0x48 are the sizes of the arks in order. Then it repeats the number of arks once more, and then a string table of the ark paths follow; the length of the file path and then the string itself.

EDIT: Found the size of the string table. A 4 byte integer at the offset 0x15E. I'm now going to start looking for the size of the data table at the bottom. Hopefully should be similar to the previous ark versions.
_________________
SUPPORT GH: AVENGED SEVENFOLD

Arandomguy1 on YouTube
TrevorTheGreat wrote:
Oh my god I've never seen a bunch of bitching children like this before.

OMG I got perma-rockered! /world

So fucking what. Play your plastic guitar and get over it.
Back to top
View user's profile Send private message XBL Gamertag: guns4fun2872
LocalH  





Joined: 30 Oct 2006
Posts: 1400
Location: MiloHax

PostPosted: Mon Nov 01, 2010 12:26 am    Post subject: Reply with quote

I don't know if you have this or not, but it might help a little bit (valid for HDR v4, as seen in RB1):

Code:
    /// ARK header format:
    /// OPTIONAL: 4 bytes - Encryption key.  If present, the rest of the file is encrypted with the routine
    ///        found in dtb_xor_360.  We assume if this value is a version we know about, it's not present.
    /// 4 bytes: Version.  Expected to be 3 or 4; these are the only versions we understand.
    /// 4 bytes: Count of parts (ARK files).  Eg, if this is 2, then there should be a main_0.ark and main_1.ark.
    ///
    /// 4 bytes: Count of part length table.  Should always be the count of parts, repeated.
    /// --- Beginning of part length table; 1 record per count ---
    ///        Version 3: 4 bytes: Size of each part file (in bytes).
    ///        Version 4: 8 bytes: Size of each part (in bytes).
    /// --- End part length table
    ///
    /// 4 bytes: Size of string table (in bytes).  *Not* a count.
    /// --- Beginning of string table; number of records unspecified ---
    ///        VARIABLE: String table (list of null-terminated strings).  Size is given in four bytes previous.
    /// --- End string table ---
    ///
    /// 4 bytes: Count of strings in string table
    /// --- Beginning of string offset table; 1 record per count ---
    ///        4 bytes: Offset into the string table to find start of string.  This offset is from the
    ///            beginning of the string table, so 0 means the beginning of the string table.
    /// --- End string offset table ---
    ///
    /// 4 bytes: Count of file table
    /// --- Beginning of file table; 1 record per count ---
    ///        Version 3: 4 bytes: Offset entry begins at (in all ARK parts, put together)
    ///        Version 4: 8 bytes: Offset entry begins at (in all ARK parts, put together)
    ///        4 bytes: String 0-based index (from string table) for file name; 3 means "4th string"
    ///        4 bytes: String 0-based index (from string table) for path name; 3 means "4th string"
    ///        4 bytes: Length of entry
    ///        4 bytes: Always 0
    /// --- End file table ---

Problem is, starting at 0x3f2c9 (0x165+0x3f163), the data there doesn't match what the documentation I have states for the "string offset table". Unless the strings are just all over the place and not really in any order...now that I look at it more, all these values do seem to be less than the string table length...damn obfuscation...

Also, I'm unsure about the data from 0x4ebb9 to 0x4ecac inclusive...everything after that looks exactly like the HDR v4 file offset table, and even maybe the last 48 bytes of the aforementioned range look like they could be a couple of entries).

Edit 2: Wow, they seem to have really just obfuscated this, there seem to be lots of null (0x00000000) string offsets in the table, they jump around the place, and there's some odd entries at the top of the file offset table. The file count seems to be at 0x4ebb9, as when multiplying it by 24, it comes out to be the same as the length of the chunk from 0x4ebbd to EOF. The string count seems to be at 0x3f2c5, as when you multiply it by 4 and start counting from 0x3f2c9, it comes out with the last byte at 0x4ebb8, but there definitely weren't 15,000 entries in that string table, but lots of null pointers in the string offset table. One step closer to cracking this shit!
_________________
MiloHax 2.0 Discord server - Everything GH/RB (RIP MiloHax 1.0 01/30/2017 - 04/04/2022)
O211: An Unofficial Scorehero Alumni Discord


Alakaiser sez: POST BECAUSE YOU HAVE SOMETHING TO SAY, NOT BECAUSE YOU HAVE TO SAY SOMETHING.
Back to top
View user's profile Send private message
LocalH  





Joined: 30 Oct 2006
Posts: 1400
Location: MiloHax

PostPosted: Mon Nov 01, 2010 2:12 am    Post subject: Reply with quote

New post to trigger it to people who already read my other post.

I think I've more or less cracked the HDR format and boy, is it obfuscated. There's still some stuff I don't understand but for the most part, I think this might be enough to traverse the file table, extract the ARK offsets and lengths for each file, index into the string offset table to retrieve the file and path name, create directories as necessary, and then write out each file in order. The first 8 entries in the file table seem to be nulled out (they reference 0-size files), so they can be skipped.

Code:
HDR breakdown
-------------

Offset      End (incl)  Data
--------    --------    --------
00000000    00000003    HDR version ($00000006)
00000004    00000007    Unknown (supposed to be # of ARK parts, but there are 10 and this says $00000001)*
00000008    00000017    Unknown (possibly a hash of some sort, 43B2CBDE 347991F6 23860C6C 48BFF67F)
00000018    0000001B    ARK part count ($0000000A = 10)
0000001C    0000001F    ARK part length table count (supposed to be, and is, same as part count, $0000000A)
00000020    00000047    ARK part length table, 10 x 4 bytes
00000048    0000004B    Unknown, seems to mirror part count and part length table count ($0000000A)
0000004C    0000004F    Filename length ($00000013), goes with the following ARK part filename
00000050    00000062    "gen/main_xbox_0.ark"
00000063    00000131    Same as previous two entries but for ARK parts 1-9
00000132    00000135    Unknown (contains $0000000A)
00000136    00000139    Unknown (contains $00000002)
0000013A    0000013D    Unknown (contains $00000003)
0000013E    00000141    Unknown (contains $00000002)
00000142    00000145    Unknown (contains $00000000)
00000146    00000149    Unknown (contains $FFFFFFFF)
0000014A    0000014D    Unknown (contains $00000001)
0000014E    00000151    Unknown (contains $00000004)
00000152    00000155    Unknown (contains $FFFFFFFF)
00000156    00000159    Unknown (contains $FFFFFFFF)
0000015A    0000015D    Unknown (contains $FFFFFFFF)
0000015E    00000161    String table length, in bytes, including string count ($0003F163)
00000162    0003F2C4    String table
00000162    00000163    Unknown (contains $2E00 but seems to be part of string table)
0003F2C5    0003F2C8    String count, including nulls ($00003E3C)
0003F2C9    0004EBB8    String offset table
0004EBB9    0004EBBC    File count ($00001EBA)
0004EBBD    0007CD2C    File offset table (and EOF)

*possibly a subversion of the file format, anyone got TBRB or GDRB to check this value and see if it's 0?


I have no way to test this, as my programming skills are limited to 68k ASM (as you can see by my use of $ rather than 0x in my notes). I did the math, however, and everything clicks perfectly. String offsets are 4 bytes, 0x3e3c x 4 = 0xf8f0 bytes calculated for the table, 0x4ebb8 - 0x3f2c9 + 1 = 0xf8f0 bytes actually used. File table entries are 24 bytes each, 0x1eba x 0x18 = 0x2e170 bytes calculated for file table, 0x7cd2c - 0x4ebbd + 1 = 0x2e170 bytes actually used.
_________________
MiloHax 2.0 Discord server - Everything GH/RB (RIP MiloHax 1.0 01/30/2017 - 04/04/2022)
O211: An Unofficial Scorehero Alumni Discord


Alakaiser sez: POST BECAUSE YOU HAVE SOMETHING TO SAY, NOT BECAUSE YOU HAVE TO SAY SOMETHING.


Last edited by LocalH on Mon Nov 01, 2010 6:01 pm; edited 1 time in total
Back to top
View user's profile Send private message
thepoop555  





Joined: 16 Sep 2007
Posts: 316
Location: Austin, TX

PostPosted: Mon Nov 01, 2010 5:11 am    Post subject: Reply with quote

Ok so lemme get this straight. There are a bunch of null file offsets in the 3rd (file info) table? Like just 24 null bytes? And then the number of strings in the string offset table includes those? Or not? Because unless I'm screwing up big time, there are 15932 (0x3C3E) entries in the string offset table? And there are acutally only 7866 (0xBA1E) entries in the file offset table, but that has null entries in it? Or am I completely confused? This is obviously a little different the the v3 and v4 hdr files...
_________________
SUPPORT GH: AVENGED SEVENFOLD

Arandomguy1 on YouTube
TrevorTheGreat wrote:
Oh my god I've never seen a bunch of bitching children like this before.

OMG I got perma-rockered! /world

So fucking what. Play your plastic guitar and get over it.
Back to top
View user's profile Send private message XBL Gamertag: guns4fun2872
LocalH  





Joined: 30 Oct 2006
Posts: 1400
Location: MiloHax

PostPosted: Mon Nov 01, 2010 10:57 am    Post subject: Reply with quote

No, there are a bunch of null string offsets in the string offset table (the middle section). File table doesn't look like it has any nulls out of place (except for the aforementioned first 8 entries).

Maybe a few screenshots might help, don't feel like formatting a hex dump in a code tag. If it weren't copyright infringement I'd just post the extracted tables in raw binary form =P

Note about the screenshots - I set 00 values to display as red (and as hash marks on the ASCII side) and turned on "flip bytes in editor" in Hex Workshop, but in the version I'm using that feature's a bit buggy, so you'll see values on the left hand side be colored red that aren't 00, due to the reordering. AFAIK, all these values are supposed to be big-endian, and that's the only way they make sense to me.

Pardon me if the images are too wide, I cropped it down quite a bit from my desktop size =P


There's the first 0x300 of the string offset table. Notice how many 0x00000000 entries there are, and if you dig between 0x3f2c9 and 0x4ebb8 you shouldn't see any values larger than 0x3f162, the value you found at 0x15e (in fact, I would expect the largest value to be 0x3f14d, the start of the last string).


First 0x300 of file table, notice how neat it looks at 24 bytes per row. Also note that I have seen offsets in the file table use the upper 32 bits to store a 1, since the total ARK size is larger than 4GB. 5 fields, 24 bytes per entry.
_________________
MiloHax 2.0 Discord server - Everything GH/RB (RIP MiloHax 1.0 01/30/2017 - 04/04/2022)
O211: An Unofficial Scorehero Alumni Discord


Alakaiser sez: POST BECAUSE YOU HAVE SOMETHING TO SAY, NOT BECAUSE YOU HAVE TO SAY SOMETHING.
Back to top
View user's profile Send private message
MahoppianGoon  





Joined: 18 Mar 2008
Posts: 37

PostPosted: Mon Nov 01, 2010 5:17 pm    Post subject: Reply with quote

Well, that was fun! :P Ok, now I'm stuck. I got all '7858' files, 'offsets/sizes', but probably missed a few, but how do you get the filenames for each offset? The filenames at the top of the hdr file are not in the same order as the offsets at the bottom, unless I'm not reading it right. It seems each file-entry at the bottom is 24-bytes long. The first 8 is the offset, (plus 0x322), I don't know why you have to add that. The next 8 is unknown? Maybe the ones that tell you which filename that offset goes too? and the last 8 is the filesize. Heres an example:

8790D2C600000000 (C6D29087+322=C6D293A9)
021900006B370000 (???)
0412040000000000 (41204, normal, no reversing)

and that offset number is the offset if ALL the ark files was combined.
So guess which file that is ^? You'll NEVER guess in a million years! lol
Yep. the 'songs.dtb' file. (it decrypts fine using the dtbcrypt 3.0)
So who wants it? lol I'll give you a hint, its in the 'main_xbox_7.ark' file
Ok, fine, heres the exact location:

offset: 0x018FD4FF
size : 0x41204
first4: 594B0000
last 4: 0A0CAFE4

Have fun!
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    ScoreHero Forum Index -> Software All times are GMT
Goto page 1, 2, 3, 4  Next
Page 1 of 4

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum





Copyright © 2006-2024 ScoreHero, LLC
Terms of Use | Privacy Policy


Powered by phpBB