Metroid Prime hacking research

MrSinistar

Edit history:

MrSinistar: 2014-09-06 04:23:43 am

2014-09-06 04:11:11 am

Similar 4 bytes appear right above in row 0x50. Samus Death Scream / 4A66553F9D4BC355.csmp

Code:

Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00000000  43 53 4D 50 00 00 00 01 49 4E 46 4F 00 00 00 0C  CSMP....INFO....
00000010  01 00 00 00 00 00 00 00 42 C8 00 00 50 41 44 20  ........BÈ..PAD 
00000020  00 00 00 14 FF FF FF FF FF FF FF FF FF FF FF FF  ....ÿÿÿÿÿÿÿÿÿÿÿÿ
00000030  FF FF FF FF FF FF FF FF 44 41 54 41 00 00 C8 04  ÿÿÿÿÿÿÿÿDATA..È.
00000040  00 01 5D 3E 00 01 8F 24 00 00 7D 00 00 00 00 00  ..]>...$..}.....
00000050  00 00 00 00 00 00 00 00 00 00 00 00 02 5F FD A2  ............._ý¢
00000060  0B 77 FA B0 07 98 FE 17 0E BA F8 7B 05 FE FD A4  .wú°.˜þ..ºø{.þý¤
00000070  0C 18 FB 32 08 9F FE 6E 0F 29 F8 64 00 00 00 00  ..û2.Ÿþn.)ød....
00000080  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000090  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000000A0  00 0C 4C 00 E1 F0 3D 20 00 C5 D0 4B 30 B7 A0 68  ..L.áð= .ÅÐK0· h

EDIT:

I think I found the loop flag. 0x11 looks like is 1 for every loop sound and 0 for every sound that doesn't have loop points. I loaded several sounds that have loops like the charge beam and one-off sounds like the power beam fire and the byte changes just as I predicted.

Similar 4 bytes appear right above in row 0x50.    Samus Death Scream / 4A66553F9D4BC355.csmp

[code]Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00000000  43 53 4D 50 00 00 00 01 49 4E 46 4F 00 00 00 0C  CSMP....INFO....
00000010  01 00 00 00 00 00 00 00 42 C8 00 00 50 41 44 20  ........BÈ..PAD 
00000020  00 00 00 14 FF FF FF FF FF FF FF FF FF FF FF FF  ....ÿÿÿÿÿÿÿÿÿÿÿÿ
00000030  FF FF FF FF FF FF FF FF 44 41 54 41 00 00 C8 04  ÿÿÿÿÿÿÿÿDATA..È.
00000040  00 01 5D 3E 00 01 8F 24 00 00 7D 00 00 00 00 00  ..]>...$..}.....
00000050  00 00 00 00 00 00 00 00 00 00 00 00 02 5F FD A2  ............._ý¢
00000060  0B 77 FA B0 07 98 FE 17 0E BA F8 7B 05 FE FD A4  .wú°.˜þ..ºø{.þý¤
00000070  0C 18 FB 32 08 9F FE 6E 0F 29 F8 64 00 00 00 00  ..û2.Ÿþn.)ød....
00000080  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000090  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000000A0  00 0C 4C 00 E1 F0 3D 20 00 C5 D0 4B 30 B7 A0 68  ..L.áð= .ÅÐK0· h

[/code]

EDIT:

I think I found the loop flag.  0x11 looks like is 1 for every loop sound and 0 for every sound that doesn't have loop points.  I loaded several sounds that have loops like the charge beam and one-off sounds like the power beam fire and the byte changes just as I predicted.

Antidote

2014-09-06 04:26:02 am

Nice find, thats in the INFO chunk right?

MrSinistar

2014-09-06 04:26:47 am

Quote from Antidote:

Nice find, thats in the INFO chunk right?

Thanks! Yup, that's right.

MrSinistar

Edit history:

MrSinistar: 2014-09-06 05:06:17 am

MrSinistar: 2014-09-06 05:06:15 am

MrSinistar: 2014-09-06 05:06:14 am

MrSinistar: 2014-09-06 05:05:48 am

2014-09-06 05:04:15 am

I think I found the actual loop point data.

Byte 0x4D (found in the DATA chunk) turns on to 01, indicating the loop flag; just like 0x11 in the INFO chunk. Then the first four bytes of row 0x50 is the starting loop point and after that, the next four bytes are the ending loop point. The hex values should be converted to decimal to give you the sample number that the loop point falls on.

Again, we'll use the Scan Visor download sound as an example:

Code:

Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00000000  43 53 4D 50 00 00 00 01 49 4E 46 4F 00 00 00 0C  CSMP....INFO....
00000010  01 01 00 00 00 00 00 00 42 C8 00 00 50 41 44 20  ........BÈ..PAD 
00000020  00 00 00 14 FF FF FF FF FF FF FF FF FF FF FF FF  ....ÿÿÿÿÿÿÿÿÿÿÿÿ
00000030  FF FF FF FF FF FF FF FF 44 41 54 41 00 00 73 24  ÿÿÿÿÿÿÿÿDATA..s$
00000040  00 00 C8 C0 00 00 E5 6E 00 00 7D 00 00 01 00 00  ..ÈÀ..ån..}.....
00000050  00 00 00 00 00 00 C8 BF 00 00 00 00 01 3C 04 9F  ......È¿.....<.Ÿ

0xC8BF (aka sample number 51391) should be our ending loop point. On sounds that have no loops, byte 0x4D is set to 00 and the first 8 bytes of row 0x50 are all zeros.

Okay, my brain is fried, I'm going to bed, haha.

I think I found the actual loop point data.

Byte 0x4D (found in the DATA chunk) turns on to 01, indicating the loop flag; just like 0x11 in the INFO chunk.  Then the first four bytes of row 0x50 is the starting loop point and after that, the next four bytes are the ending loop point.  The hex values should be converted to decimal to give you the sample number that the loop point falls on.

Again, we'll use the Scan Visor download sound as an example:

[code]Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00000000  43 53 4D 50 00 00 00 01 49 4E 46 4F 00 00 00 0C  CSMP....INFO....
00000010  01 01 00 00 00 00 00 00 42 C8 00 00 50 41 44 20  ........BÈ..PAD 
00000020  00 00 00 14 FF FF FF FF FF FF FF FF FF FF FF FF  ....ÿÿÿÿÿÿÿÿÿÿÿÿ
00000030  FF FF FF FF FF FF FF FF 44 41 54 41 00 00 73 24  ÿÿÿÿÿÿÿÿDATA..s$
00000040  00 00 C8 C0 00 00 E5 6E 00 00 7D 00 00 01 00 00  ..ÈÀ..ån..}.....
00000050  00 00 00 00 00 00 C8 BF 00 00 00 00 01 3C 04 9F  ......È¿.....<.Ÿ
[/code]

0xC8BF (aka sample number 51391) should be our ending loop point.   On sounds that have no loops, byte 0x4D is set to 00 and the first 8 bytes of row 0x50 are all zeros.

Okay, my brain is fried, I'm going to bed, haha.

Antidote

Edit history:

Antidote: 2014-09-06 05:33:04 am

2014-09-06 05:27:33 am

I'm thinking that float at the end of the INFO chunk is the volume, the most common value is 100f
I'm also thinking that that 7D value is the frequency, because at offset 0x48 it's 32,000.

Antidote

Edit history:

Antidote: 2014-09-06 06:10:00 am

2014-09-06 06:08:57 am

Retro, thank you for being utterly lazy. I now have the ENTIRE DATA chunk header doc'd, It's just the standard GCN adpcm data dressed up:
http://hcs64.com/files/DSPADPCM.us.pdf

Code:

//--------------------------------------
//--- 010 Editor v5.0.2 Binary Template
//
// File:
// Author:
// Revision:
// Purpose:
//--------------------------------------

BigEndian();
enum ChunkType
{
    INFO = 0x494E464F,
    PAD  = 0x50414420,
    DATA = 0x44415441
};

struct CSMPChunk
{
    ChunkType type;
    uint chunkLength;
    union
    {
        if (type != PAD)
        {
            struct
            {
                switch(type)
                {
                    case INFO:
                        ubyte unk1;
                        ubyte isLooped;
                        uint unk2;
                        ushort unk3;
                        float volume;
                    break;
                    case DATA:
                        uint sampleCount;
                        uint numAdpcmNibbles;
                        uint sampleRate;
                        ushort loopFlag;
                        ushort format;
                        uint loopStart;
                        uint loopEnd;
                        uint padding1;
                        ushort coef[16];
                        ushort gain;
                        ushort predictorScale;
                        ushort yn1;
                        ushort yn2;
                        ushort loopPredictorScale;
                        ushort lyn1;
                        ushort lyn2;
                        ushort padding2[11];
                        break;
                }
            } info;
        }
        char rawData[(type != DATA) ? chunkLength : chunkLength - 4];
    } data;
};

struct
{
    uint magic <format = hex>;
    uint version;
    CSMPChunk chunks[3] <optimize = false>;
}file;

Retro, thank you for being utterly lazy. I now have the ENTIRE DATA chunk header doc'd, It's just the standard GCN adpcm data dressed up:
http://hcs64.com/files/DSPADPCM.us.pdf
[code]
//--------------------------------------
//--- 010 Editor v5.0.2 Binary Template
//
// File:
// Author:
// Revision:
// Purpose:
//--------------------------------------

BigEndian();
enum ChunkType
{
    INFO = 0x494E464F,
    PAD  = 0x50414420,
    DATA = 0x44415441
};

struct CSMPChunk
{
    ChunkType type;
    uint chunkLength;
    union
    {
        if (type != PAD)
        {
            struct
            {
                switch(type)
                {
                    case INFO:
                        ubyte unk1;
                        ubyte isLooped;
                        uint unk2;
                        ushort unk3;
                        float volume;
                    break;
                    case DATA:
                        uint sampleCount;
                        uint numAdpcmNibbles;
                        uint sampleRate;
                        ushort loopFlag;
                        ushort format;
                        uint loopStart;
                        uint loopEnd;
                        uint padding1;
                        ushort coef[16];
                        ushort gain;
                        ushort predictorScale;
                        ushort yn1;
                        ushort yn2;
                        ushort loopPredictorScale;
                        ushort lyn1;
                        ushort lyn2;
                        ushort padding2[11];
                        break;
                }
            } info;
        }
        char rawData[(type != DATA) ? chunkLength : chunkLength - 4];
    } data;
};

struct
{
    uint magic <format = hex>;
    uint version;
    CSMPChunk chunks[3] <optimize = false>;
}file;
[/code]

Antidote

Edit history:

Antidote: 2014-09-06 07:07:25 am

2014-09-06 06:36:16 am

Output from DSPADPCM:

Code:

Header size: 96 bytes

Sample     : 'test.dsp'
Length     : 29340 samples
Num nibbles: 33532 ADPCM nibbles
Sample Rate: 32000 Hz
Loop Flag  : LOOPED

Start Addr : 0x000001B5 + ARAM_offset (ADPCM nibble mode)
End Addr   : 0x0000729A + ARAM_offset (ADPCM nibble mode)
Curr Addr  : 0x00000000 + ARAM_offset (ADPCM nibble mode)

a1[0]: 0x0F91 a2[0]: 0xF82F 
a1[1]: 0x0FEC a2[1]: 0xF80C 
a1[2]: 0x0DFC a2[2]: 0xF9BF 
a1[3]: 0x0000 a2[3]: 0x0000 
a1[4]: 0x0FB8 a2[4]: 0xF82B 
a1[5]: 0x0000 a2[5]: 0x0000 
a1[6]: 0x0E4F a2[6]: 0xF99C 
a1[7]: 0x0000 a2[7]: 0x0000 

Gain      : 0x0000
Pred/Scale: 0x0060
y[n-1]    : 0x0000
y[n-2]    : 0x0000

Loop Pred/Scale: 0x0012
Loop y[n-1]    : 0xFE94
Loop y[n-2]    : 0xFAEA

Unfortunately it produced an empty wav, but I think that has to do with the fact that I'm using wine.

EDIT:
GOT IT!!!
https://dl.dropboxusercontent.com/u/21757902/test.wav

MrSinistar

2014-09-06 10:20:36 am

OH MY GOD, YES!!!!! YES, YES, YES!!! THANK YOU SO MUCH!

How did you make the DSP file to put in DSPADPCM?

Antidote

Edit history:

Antidote: 2014-09-06 11:51:12 am

2014-09-06 11:49:27 am

I just took the adpcm data and put in a separate file, dsp is a bit misleading as I was going to give it a DSP header but failed to realize that DSP just is dual channel adpcm, while these are mono.
Here is another audio clip
https://dl.dropboxusercontent.com/u/21757902/sam_death_scream.wav

MrSinistar

2014-09-06 12:54:15 pm

This is so freaking awesome...I've been waiting to rip the Metroid Prime sound effects ever since I hear 'em back since I was 13 years old. This is just absolutely phenomenal. Thank you so much, Antidote and Paraxade.

Here's the Scan Download sound effect:

https://www.mediafire.com/?sjogjx95s5yh0zf

Antidote

2014-09-06 12:59:55 pm

No problem, but that sounds a tad corrupted D:

MrSinistar

Edit history:

MrSinistar: 2014-09-06 01:18:01 pm

2014-09-06 01:16:11 pm

Hmmm, plays fine on my computer and in FL Studio.

Here's "Into Hud", one of my favorite sound effects in the game:

http://www.mediafire.com/?dd57j8jyb36cak6

Since Mediafire doesn't play 32khz sounds that well, here's the Scan Download visor, upscaled to 44.1khz.

https://www.mediafire.com/?qtockkdhgs559he

Antidote

2014-09-06 01:17:38 pm

That's much better, also the intro hud is actually the first file you linked.

MrSinistar

2014-09-06 01:31:17 pm

FL Studio doesn't want to play the AIFF files, which sucks since they're the only format that DSPADPCM supports that has loop points. Antidote, can you confirm that the loop points actually get exported with DSPADPCM?

Antidote

2014-09-06 01:32:44 pm

If you use -f they should be, send me an aiff and I'll check, (i'm having to use foobar2k to play the dsp files)

MrSinistar

Edit history:

MrSinistar: 2014-09-06 01:36:00 pm

2014-09-06 01:34:47 pm

Here you go, Charge Beam loop: https://www.mediafire.com/?09i6tctg03l0e3s

EDIT: VLC can play the file, but I can't tell if there's loop points or not.

Antidote

2014-09-06 01:43:34 pm

Yep, they're being exported:

Code:

4D 41 52 4B 00 00 00 23 00 02 00 00 00 00 EB EE
08 62 65 67 20 6C 6F 6F 70 00 00 01 00 01 E8 BF
30 08 65 6E 64 20 6C 6F 6F 70 00

MrSinistar

Edit history:

MrSinistar: 2014-09-06 01:56:57 pm

MrSinistar: 2014-09-06 01:56:38 pm

MrSinistar: 2014-09-06 01:49:40 pm

2014-09-06 01:48:46 pm

Something weird's going on here. Audacity crashes everytime I try to convert AIFF to WAV, and it imports the AIFF with having 8000000000Hz. 0_o

foobar doesn't load the file correctly, but VLC does and with the correct sample rate...the hell?

Antidote

2014-09-06 02:00:16 pm

I noticed that too o.o

MrSinistar

2014-09-06 02:06:55 pm

Is there a way we can batch delete the first four rows of CSMP header information so that we have just the DSP data? It would make file conversion a lot quicker.

Antidote

Edit history:

Antidote: 2014-09-06 02:12:39 pm

2014-09-06 02:11:40 pm

I'm actually working on figuring out how to decode the adpcm data myself, but what you could do is write a batch file that deletes the first 0x40 bytes (not sure how you can do that).
If you like python you could do it that way as well, that way it's cross platform from the outset.

EDIT:
Also:
https://dl.dropboxusercontent.com/u/21757902/dkcr_bop.wav

MrSinistar

Edit history:

MrSinistar: 2014-09-06 02:16:39 pm

2014-09-06 02:16:13 pm

I can't program to save my life. I tried a long time ago and I can't wrap my head around it. I'm much better at analyzing, lol.

And hey, Donkey Kong!

MrSinistar

2014-09-06 02:28:20 pm

More sounds for your enjoyment ;)

https://www.mediafire.com/?9dj0q784ykr8wie

Aruki

2014-09-06 02:37:09 pm

Oh wow, sounds actually being ripped. That's super cool; awesome work, both of you.

MrSinistar

Edit history:

MrSinistar: 2014-09-06 02:52:53 pm

2014-09-06 02:52:07 pm

Thanks! I could really use is a CSMP to DSP batch converter, which would be really easy since we just need to remove the first 64 bytes of the CSMP header and change the file extension to DSP for conversion in DSPADPCM. That way I can cue up all the sounds and go through trial and error on finding what sounds are which, since there's no NAME chunk like the prototype CSMPs (which I'm finding out to be extremely helpful). If someone can write that for me, I would be forever grateful.

I really wanna get some Metroid sounds but I realize this is going to be like trying to find needle in a haystack. I figure batch conversion will make the process easier to go through.