Metroid Prime hacking research

2014-06-14 05:29:48 pm

paraxade is there any chance we could get a look at the roof of mp1's training chamber? I tried for hours but never managed to do the SW in there without floaty jump (which is why there's no video for it on the site), so I always wondered where the lowest invisible wall was in that room

T3

2014-06-14 10:52:04 pm

you can actually get oob by just doing a ubj which gives plenty of height: http://www.youtube.com/watch?v=z8IgXlBwaqA

with that being said, i'd also be interested in seeing the collision for the roof.

Aruki

Edit history:

Aruki: 2014-06-15 01:38:47 am

Aruki: 2014-06-15 01:38:18 am

Aruki: 2014-06-15 01:37:53 am

Aruki: 2014-06-15 01:13:52 am

2014-06-15 01:13:30 am

Training Chamber pictures:

1. The collision actually seems to match the terrain very very closely, with only minor differences. This shot is taken angled from under the hole towards the far side of the room. So pretty much whatever looks like the lowest point actually is. The spot T3 uses in his video looks about good.
2. The pipes over the door
3. This is what the ledge T3 does his UBJ from looks like.
4. It's a lot bigger and flatter than it looks...
5. There's a similar ledge on the left side of the room, but for some reason it seems to be smaller, so I guess the right side is the way to go.

Also, ever wondered why wallcrawling Phendrana is such a pain? Well, here you go! WHY DO THEY DO THIS eaearjerlhera;heaheah

1. Ice Ruins West/Courtyard Access
2. Frost Cave Access/Frost Cave
3. Ruined Courtyard/Quarantine Access

On an unrelated note, xinoea in Miles's Twitch chat made us an awesome Phazon Suit pickup texture, so I went and got DDS->TXTR conversion working and modded a couple Gravity Suit files... and now we have a custom Phazon Suit pickup for the randomizer!

Training Chamber pictures:

1. [url=http://i.imgur.com/QfzcYNa.png]The collision actually seems to match the terrain very very closely, with only minor differences. This shot is taken angled from under the hole towards the far side of the room. So pretty much whatever looks like the lowest point actually is. The spot T3 uses in his video looks about good.[/url]
2. [url=http://i.imgur.com/NKB16Yg.png]The pipes over the door[/url]
3. [url=http://i.imgur.com/I5XQ30O.png]This is what the ledge T3 does his UBJ from looks like.[/url]
4. [url=http://i.imgur.com/QGjPf6P.jpg]It's a lot bigger and flatter than it looks...[/url]
5. [url=http://i.imgur.com/04R16Qr.png]There's a similar ledge on the left side of the room, but for some reason it seems to be smaller, so I guess the right side is the way to go.[/url]

Also, ever wondered why wallcrawling Phendrana is such a pain? Well, here you go! WHY DO THEY DO THIS eaearjerlhera;heaheah

1. [url=http://i.imgur.com/B1la6LF.png]Ice Ruins West/Courtyard Access[/url]
2. [url=http://i.imgur.com/iCGy2RL.png]Frost Cave Access/Frost Cave[/url]
3. [url=http://i.imgur.com/Dy7BI0V.png]Ruined Courtyard/Quarantine Access[/url]

On an unrelated note, xinoea in Miles's Twitch chat made us an awesome Phazon Suit pickup texture, so I went and got DDS->TXTR conversion working and modded a couple Gravity Suit files... [url=https://pbs.twimg.com/media/BqHEKmZIYAA8dwZ.png:large]and now we have a custom Phazon Suit pickup for the randomizer![/url]

TheMG2

2014-06-15 01:40:57 am

The phazon suit is off-center.

Aruki

2014-06-15 02:00:48 am

Well obviously. It's just a test, that's not actually in a randomizer game.

TheMG2

2014-06-15 02:08:27 am

I am extremely concerned with things being off-center.

Aruki

2014-06-15 02:08:58 am

Your post is off-center.

It should be center-aligned, like this.

DJGrenola

2014-06-15 07:10:49 am

I'd never seen that UBJ before. not sure why I didn't try doing that. or maybe I did and just sucked at it. can't remember. it was a long time ago. anyway, nice.

parax: thanks. the shape of that little ledge certainly explains why all my attempts to ghetto jump off it didn't work. :P

another room I'd like to see is chapel of the elders, after spending ages in there mucking about getting a video of the SW for the site. there's some strange collision in that room.

Aruki

2014-06-15 07:26:45 am

What parts of CotE exactly?

DJGrenola

2014-06-15 07:28:21 am

I guess the areas above the cubicles in the corners of the room? there are some strange invisible walls and floors

DJGrenola

2014-06-15 07:28:36 am

dont' worry too much about it, I'm just curious

Aruki

2014-06-15 08:43:27 am

CotE:

#1 - So, first of all, this is what the back-right cubicle looks like from the outside. (Ignore the random polygon sticking out of the wall, that's a bug.) There's a big invisible wall there to prevent you from getting up on the outside.

#2 - This is the hole in the back from below. It matches up with the terrain pretty closely.

#3 - Here's the hole again, from above. The fact that you can even stand here at all seems to be a stroke of luck - there's no ledge here at all. What's happening is that the slope's collision geometry doesn't actually stretch all the way to the wall, so when you jump there, you're actually standing on the top edge of the slope, not on any actual ledge.

#4 - This is on top of the cubicle. Despite the fact that they blocked off the front with a wall, and the only way to get up here is through that little hole, the entire floor up here is perfectly solid. I'm not sure why they bothered putting collision up here, but apparently they did. There's a backfacing wall here - that's the same wall from screenshot #1.

#5 - Same location with backface culling disabled so you can see the wall more clearly.

#6 - This is the top of the wall. Once again, there's no ledge here - when you bomb jump up here you're actually standing on the top edge of the wall. From there it's a simple matter of jumping out.

CotE:

[url=http://i.imgur.com/L5t3xlK.png]#1[/url] - So, first of all, this is what the back-right cubicle looks like from the outside. (Ignore the random polygon sticking out of the wall, that's a bug.) There's a big invisible wall there to prevent you from getting up on the outside.

[url=http://i.imgur.com/L4V90gW.png]#2[/url] - This is the hole in the back from below. It matches up with the terrain pretty closely.

[url=http://i.imgur.com/VnS7pwC.png]#3[/url] - Here's the hole again, from above. The fact that you can even stand here at all seems to be a stroke of luck - there's no ledge here at all. What's happening is that the slope's collision geometry doesn't actually stretch all the way to the wall, so when you jump there, you're actually standing on the top edge of the slope, not on any actual ledge.

[url=http://i.imgur.com/R4dJqnW.png]#4[/url] - This is on top of the cubicle. Despite the fact that they blocked off the front with a wall, and the only way to get up here is through that little hole, the entire floor up here is perfectly solid. I'm not sure why they bothered putting collision up here, but apparently they did. There's a backfacing wall here - that's the same wall from screenshot #1.

[url=http://i.imgur.com/vE8TniO.png]#5[/url] - Same location with backface culling disabled so you can see the wall more clearly.

[url=http://i.imgur.com/IjF8tas.png]#6[/url] - This is the top of the wall. Once again, there's no ledge here - when you bomb jump up here you're actually standing on the top edge of the wall. From there it's a simple matter of jumping out.

DJGrenola

Edit history:

DJGrenola: 2014-06-15 09:04:27 am

2014-06-15 09:02:56 am

Quote:

I'm not sure why they bothered putting collision up here

yeah that was more or less my thought once I finally managed to get on top of the cubicle for the first time.

intriguing about the slope. always wondered why you could only stand properly when you were tightly in the corner. also interesting to see the top of that invisible wall. it seemed in-game like there was a whole invisible floor up there and I never knew why they put that up there, but it looks like I was just imagining that.

I went through a lot of painful trial and error making those SW videos, lol.

very interesting, thanks.

Antidote

Edit history:

Antidote: 2014-06-15 05:25:08 pm

Antidote: 2014-06-15 05:24:43 pm

Antidote: 2014-06-15 05:22:36 pm

2014-06-15 05:09:36 pm

I'm going to take a look in that room in Magmoor caverns just past the elevator from TOW, the one with that ledge you can stand on before heading to the plasma beam.

Interesting:

plasma_sb There doesn't seem to a legitimate ledge to stand on :O
Lawl bugs

Aruki

2014-06-19 07:47:00 pm

Quote from BioSpark:

cel-shaded prime hack please.

A bunch of people kept asking for this so I went ahead and did it, and streamed a randomizer playthrough with it. It was kinda half and half for the most part, some stuff looks kinda neat and others not so much. It worked nicely in areas with a lot of contrasting colors. Not so nice in places like Phendrana.

Artifact Temple and the Ridley fight were really cool in particular, so I highlighted that: http://www.twitch.tv/parax0/c/4497068

TheMG2

2014-06-19 09:23:36 pm

Why didn't you highlight the horrifying samus face?

Aruki

Edit history:

Aruki: 2014-06-30 08:46:05 am

Aruki: 2014-06-29 08:43:36 pm

Aruki: 2014-06-29 07:50:34 pm

2014-06-29 07:46:48 pm

Little update... been adding object support to my tool (which I'm now calling Prime World Editor)! With help from Miles.

There's tons left to do to make this remotely satisfactory for either viewing or editing, but it's finally looking like an editor! (Even though it isn't one yet.) I'm about to have a lot going on IRL but hopefully soon I'll be able to get back to doing a lot more work on this.

Since object properties are different on every type of object, each one needs to be processed differently. I ended up coming up with an XML template system to handle that; it reads an XML file that specifies the properties to read, and what type of variable each property is. It works very well, and it's going to make it very easy to add support for more object types. It's also gonna give me a lot of room to expand on this in the future to customize the display of each type of property further, so you could do things like say what type of widget to use on a given property, or group properties into categories (like weapon vulnerabilities).

Soon I want to add things like an improved object selection system, the ability to view triggers (should be extremely useful), and some actual editing and resaving functionality.

Also, I'm stuck on the PAS4 thing Antidote mentioned as well. <_< It's a section in the ANCS format that has a variable length and a variety of different variable types with no discernible pattern or any metadata. Since there's no way to skip past it either, that means it's basically preventing parsing the rest of the file.

The ANCS file specifies what set of assets is used by most objects (the dedicated CMDL field you can see in that screenshot is only used when there's no ANCS file). This isn't really problematic for most objects, because the CMDL ID comes before the PAS4 section. The problem is that some ANCS files have multiple nodes, so objects have a value (incorrectly labeled "Model flag" in the screenshot) that says what node to use. Thanks to PAS4, I can't parse any nodes after the first one, so I can't pull out the correct model for any object that doesn't use the first node.

It's not really a huge deal - doesn't affect most objects and even then only affects how it looks in the editor - but I guess it'll be more important if we get to the point of wanting to make more custom stuff.

Geez this post ended up being way longer than I thought it would be. Sorry for the wall of text.

Little update... been adding object support to my tool (which I'm now calling Prime World Editor)! With help from Miles.

[img]https://pbs.twimg.com/media/BrVW-hiCYAAlcQ0.png:large[/img]
[img]http://i.imgur.com/xpVh5q4.jpg[/img]

There's tons left to do to make this remotely satisfactory for either viewing or editing, but it's finally looking like an editor! (Even though it isn't one yet.) I'm about to have a lot going on IRL but hopefully soon I'll be able to get back to doing a lot more work on this.

Since object properties are different on every type of object, each one needs to be processed differently. I ended up coming up with an XML template system to handle that; it reads an XML file that specifies the properties to read, and what type of variable each property is. It works very well, and it's going to make it very easy to add support for more object types. It's also gonna give me a lot of room to expand on this in the future to customize the display of each type of property further, so you could do things like say what type of widget to use on a given property, or group properties into categories (like weapon vulnerabilities).

Soon I want to add things like an improved object selection system, the ability to view triggers (should be extremely useful), and some actual editing and resaving functionality.

Also, I'm stuck on the PAS4 thing Antidote mentioned as well. <_< It's a section in the ANCS format that has a variable length and a variety of different variable types with no discernible pattern or any metadata. Since there's no way to skip past it either, that means it's basically preventing parsing the rest of the file.

The ANCS file specifies what set of assets is used by most objects (the dedicated CMDL field you can see in that screenshot is only used when there's no ANCS file). This isn't really problematic for most objects, because the CMDL ID comes before the PAS4 section. The problem is that some ANCS files have multiple nodes, so objects have a value (incorrectly labeled "Model flag" in the screenshot) that says what node to use. Thanks to PAS4, I can't parse any nodes after the first one, so I can't pull out the correct model for any object that doesn't use the first node.

It's not really a huge deal - doesn't affect most objects and even then only affects how it looks in the editor - but I guess it'll be more important if we get to the point of wanting to make more custom stuff.

Geez this post ended up being way longer than I thought it would be. Sorry for the wall of text.

Baby Sheegoth

2014-07-03 01:10:25 pm

Since when is there boxes in Frigate?

TheMG2

2014-07-03 01:12:31 pm

Quote from Baby Sheegoth:

Since when is there boxes in Frigate?

Unused room layer I think.

Aruki

Edit history:

Aruki: 2014-07-03 03:47:56 pm

2014-07-03 03:47:12 pm

yeah, those are on layer 4 IIRC, which is never enabled normally.

fun fact, that room also has a PlayerActor object that uses an outdated ANCS file for Samus that uses an early Phazon Suit model instead of the correct final one. So if you've seen pictures of that greenish Phazon Suit around, that's why the model ended up on the disc in the first place. (This is the same reason the Ripper model is on the disc - it's still used in the Glider's ANCS file, there just aren't any Gliders that use that node normally.)

SkippyJr

Edit history:

SkippyJr: 2014-07-03 10:51:18 pm

2014-07-03 10:47:04 pm

Had a bit of luck manually figuring out stuff from the dumped symbol data from the demo version. There are CodeWarrior MAP files that look like this but don't match default.dol. For example zlib's inflate() is at an offset of -0xdea8 in the default.dol relative to the location in the MAP. To find the name of a function in the MAP (or find the code in the dol) I first made a table with function addresses and sizes and look for matching large functions in the dol & MAP near the estimated offsets. When that's found then move in towards the function of interest.

One early highlight is that the type of data in some of the unknown file formats is given. The function AddPaksAndFactories() registers the 4 byte file type with a handler. The following are those pairings (without "Factory" at the end of the function name). If you want to play along at home, in 0-00 it is located at 0x80006064; look for the lis-lis-addi-addi-addi-bl pattern.

Code:

STRG StringTable
CMDL Model
TXTR Texture
CSKR SkinRules
ANIM AnimSource
CINF CharLayoutInfo
ANCS AnimCharacterSet
CSTB CollisionResponseData
SWHC ParticleSwooshData
PART Particle
ELSC ParticleElectricData
WPSC ProjectileWeaponData
FRME GuiKeyFrame
KFAM GuiKeyFrameAnim
FONT RasterFont
SCAN ScannableObjectInfo
EVNT AnimPOIData
AFSM AiFiniteStateMachine
AGSC AudioGroupSetLocData
DCLN CollidableOBBTreeGroup
DPSC DecalData
ATBL AudioTranslationTable
PATH PathFindArea
MAPW MapWorld
MAPA MapArea
MAPU MapUniverse
CSNG MidiData
DGRP DepencyGroup

I see PAS stuff (but no PAS4 references) in the symbol data, and it looks like it builds some kind of database. I'll follow the ANCS handler codepath in AddPacksAndFactories() to see what it does when it reaches a PAS4 section. I won't have meaning of the data, just how to parse it.

Had a bit of luck manually figuring out stuff from the dumped symbol data from the demo version.  There are CodeWarrior MAP files that look like [url=https://github.com/jkmiller11/FSAE_Main_Controller/blob/master/main_controller_rappid/FLASH/main_controller_rappid.MAP]this[/url] but don't match default.dol.  For example zlib's inflate() is at an offset of -0xdea8 in the default.dol relative to the location in the MAP.  To find the name of a function in the MAP (or find the code in the dol) I first made a table with function addresses and sizes and look for matching large functions in the dol & MAP near the estimated offsets.  When that's found then move in towards the function of interest.

One early highlight is that the type of data in some of the unknown file formats is given.  The function AddPaksAndFactories() registers the 4 byte file type with a handler.  The following are those pairings (without "Factory" at the end of the function name).  If you want to play along at home, in 0-00 it is located at 0x80006064; look for the lis-lis-addi-addi-addi-bl pattern.

[code]
STRG StringTable
CMDL Model
TXTR Texture
CSKR SkinRules
ANIM AnimSource
CINF CharLayoutInfo
ANCS AnimCharacterSet
CSTB CollisionResponseData
SWHC ParticleSwooshData
PART Particle
ELSC ParticleElectricData
WPSC ProjectileWeaponData
FRME GuiKeyFrame
KFAM GuiKeyFrameAnim
FONT RasterFont
SCAN ScannableObjectInfo
EVNT AnimPOIData
AFSM AiFiniteStateMachine
AGSC AudioGroupSetLocData
DCLN CollidableOBBTreeGroup
DPSC DecalData
ATBL AudioTranslationTable
PATH PathFindArea
MAPW MapWorld
MAPA MapArea
MAPU MapUniverse
CSNG MidiData
DGRP DepencyGroup
[/code]

I see PAS stuff (but no PAS4 references) in the symbol data, and it looks like it builds some kind of database.   I'll follow the ANCS handler codepath in AddPacksAndFactories() to see what it does when it reaches a PAS4 section.  I won't have meaning of the data, just how to parse it.

Baby Sheegoth

2014-07-03 11:58:52 pm

Quote from Paraxade:

yeah, those are on layer 4 IIRC, which is never enabled normally.

fun fact, that room also has a PlayerActor object that uses an outdated ANCS file for Samus that uses an early Phazon Suit model instead of the correct final one. So if you've seen pictures of that greenish Phazon Suit around, that's why the model ended up on the disc in the first place. (This is the same reason the Ripper model is on the disc - it's still used in the Glider's ANCS file, there just aren't any Gliders that use that node normally.)

Pictures of both of these?

Aruki

2014-07-04 08:52:26 am

I think there's a picture on TCRF. I don't have access to my computer until next week so I can't take any pictures myself. :/

Skippy, really cool stuff. That might turn out to be useful later... Good luck figuring out PAS4. If you can figure out how to parse it then hopefully it won't be too hard to figure out what it actually does from there. I tried changing some values on the Beetle's ANCS a week or two ago, and everything I changed either made it behave very strangely or crashed the game.

DJGrenola

Edit history:

DJGrenola: 2014-07-04 01:40:48 pm

2014-07-04 10:25:31 am

<3 skippy

there really should be a public repository for symbols in the executables that people have figured out.

I say repository but a text file would do honestly

DJGrenola

2014-07-07 10:29:15 pm

attachment:

GM8E01-4.png