Metroid Prime hacking research

Edit history:

Aruki: 2015-01-24 09:19:37 pm

Aruki: 2015-01-24 09:17:00 pm

2015-01-24 09:15:05 pm

Ok, I'm not too super proud of these scripts these days, but this is probably sufficiently clean to release. Lots of improvements over the CMDL script even. Here you go. I'll prolly stick this in the first post soon.

It works correctly on some of the biggest rooms so it should be fine but lemme know if you run into any problems.

edit: lol and the second I release it I try to import something and I get an error. Of course. Lessee what's wrong with it...

edit 2: Again. Why the hell are there materials that only have a lightmap and nothing else

Aruki

2015-01-24 09:20:44 pm

Ok, there we go, fixed.

attachment:

mrea.rar

skysurf3000

2015-01-25 08:13:03 am

Quote from Antidote:

I do have one gripe though, It's kinda hard to tell how many states are actually there, some files have more than one of a specific state, case in point: Metroid Prime's AFSM which has 5 copies of "// retreat wait" states.

A decent solution is to stick it in the first referenced state, then store the index for the referenced states in each trigger after that.

I'm not really sure if I understand what you are saying: you want the vertices to be fused together?

A way you can do this "by hand" is to use the grouping tool in yEd: you can easily select vertices with the same name in the "structure view panel". Then go to Grouping -> Group. I'll see if i can modify convertAFSM so that those groups are created automatically.

However I noticed that when doing so the number of edges going in or out of a group quickly becomes too big. I have a few ideas on how to deal with that though but I'll have to experiment a bit.

Also: how do you find which file corresponds to which enemy?

[quote="Antidote"]
I do have one gripe though, It's kinda hard to tell how many states are actually there, some files have more than one of a specific state, case in point: Metroid Prime's AFSM which has 5 copies of "// retreat wait" states.

A decent solution is to stick it in the first referenced state, then store the index for the referenced states in each trigger after that.[/quote]

I'm not really sure if I understand what you are saying: you want the vertices to be fused together?

A way you can do this "by hand" is to use the grouping tool in yEd: you can easily select vertices with the same name in the "structure view panel". Then go to Grouping -> Group. I'll see if i can modify convertAFSM so that those groups are created automatically.

However I noticed that when doing so the number of edges going in or out of a group quickly becomes too big. I have a few ideas on how to deal with that though but I'll have to experiment a bit.

Also: how do you find which file corresponds to which enemy?

Aruki

Edit history:

Aruki: 2015-01-25 08:23:41 am

2015-01-25 08:15:21 am

They're different states, regardless of if they have the same text... they most definitely shouldn't be fused together.

The only real way to tell which file is for which enemy is to find an instance of that enemy in an MREA file and check which AFSM file it's set to use. There's no links aside from that.

edit: By the way skysurf, does your tool handle triggers that have multiple conditions attached to them?

skysurf3000

2015-01-25 08:52:01 am

Quote from Paraxade:

edit: By the way skysurf, does your tool handle triggers that have multiple conditions attached to them?

Yes it does. The label of the edge will then be the name of the first condition, but in the "details" field, all the conditions will be listed with their parameters.

Antidote

2015-01-25 03:10:18 pm

Skysurf, any AFSM can be used with any enemy, I've messed around with this, and let me tell you, it's entertaining to see a skree try to behave like a pirate.

Aruki

2015-01-25 03:15:23 pm

Although it really depends which AI functions are reimplemented on different enemy types. I think CPatterned has base implementations for a lot of them though, and a lot of enemies inherit from CPatterned.

Antidote

2015-01-25 03:20:52 pm

All of them do as far as i can tell actually.

finiteBytes

2015-01-26 01:13:43 am

The script seems to be working great, Paraxade. Fantastic work! Now to see if I can dig out those collisions...

Antidote

2015-01-26 02:26:15 am

I have patched the bug in pakextract and have made the source available again:
https://dl.dropboxusercontent.com/u/21757902/downloads/pakextract.zip

I have also provided a 64bit version for linux, which may or may not work for you, I will have a windows binary available for you soon as well, again I apologize for any inconvenience the MREA corruption bug may have caused.

Antidote

Edit history:

Antidote: 2015-01-26 03:47:18 am

2015-01-26 03:46:57 am

if you downloaded pakextract within the past hour, redownload, I made a change and forgot to debug it to make sure it worked properly >.>

skysurf3000

Edit history:

skysurf3000: 2015-01-26 02:33:36 pm

2015-01-26 11:28:48 am

A small update to convertAFSM
- arrows now have a pointy end by default :)
- states with the same name are grouped together. You can display a group as a single vertex or show it's full content by clicking on the [+] in the corner :)
This should make big AFSM files easier to read.
- If a transition has multiple conditions, the label of the edge will end with "(...)"

Quote:

It's a linux app. Produces a .graphml from a valid .AFSM file.
Once you open the .graphml with yEd there are probably 2 things you will want to do:
- go to Tools -> Fit Node to Label
- go to Layout, and select your favorite one. I found that the best one is often the "Flowchart" one, but any of the "Orthogonal" one is worth a try.

About the output:
Each edge corresponds to one Trigger.
Edges are named according to the first condition of the Trigger. If there are additional conditions, the label of the edge ends with "(...)"
Each edge has a "Details" attribute, which can be seen in the right hand side panel, where all the conditions of the Trigger are listed, according to the format:
Condition_Name [float_parameter]
States with the same name are grouped.

Once again, any comment / suggestion / bug report is welcome :)

-- EDIT --
There seems to be a bug somewhere, there are a couple of .AFSM that i can't open.

A small update to convertAFSM
 - arrows now have a pointy end by default :)
 - states with the same name are grouped together. You can display a group as a single vertex or show it's full content by clicking on the [+] in the corner :)
This should make big AFSM files easier to read.
 - If a transition has multiple conditions, the label of the edge will end with "(...)"

[quote]It's a linux app. Produces a .graphml from a valid .AFSM file. 
Once you open the .graphml with yEd there are probably 2 things you will want to do:
- go to Tools -> Fit Node to Label
- go to Layout, and select your favorite one. I found that the best one is often the "Flowchart" one, but any of the "Orthogonal" one is worth a try.

About the output:
Each edge corresponds to one Trigger.
Edges are named according to the first condition of the Trigger. If there are additional conditions, the label of the edge ends with "(...)"
Each edge has a "Details" attribute, which can be seen in the right hand side panel, where all the conditions of the Trigger are listed, according to the format:
[i]Condition_Name[/i] [[i]float_parameter[/i]]
States with the same name are grouped. [/quote]

Once again, any comment / suggestion / bug report is welcome :)

-- EDIT --
There seems to be a bug somewhere, there are a couple of .AFSM that i can't open.

skysurf3000

Edit history:

skysurf3000: 2015-01-26 03:05:42 pm

2015-01-26 03:05:18 pm

It's fixed

Also, question: at the end of an AFSM file, there is a various number of FF bits. Do they have a meaning? Or can i just not care and write my customs AFSM without them?

attachment:

convertAFSM

Antidote

Edit history:

Antidote: 2015-01-26 03:11:55 pm

Antidote: 2015-01-26 03:11:13 pm

2015-01-26 03:10:16 pm

No, that's just padding. But when you export you should probably go ahead and pad to the nearest 32bytes

Code:

offset = (offset + 31) & ~31

Aruki

2015-01-26 03:13:40 pm

or you can pad until (currentoffset % 32 == 0). Slower than Antidote's code but a bit easier to understand imo.

Antidote

2015-01-26 03:18:15 pm

skysurf, turn off smooth bends please, poor kittens being slaughtered by them ;.;

finiteBytes

2015-01-26 06:58:42 pm

Been digging around but I'm a little lost. Is there any way that I could use the MREA maxscript to convert the level collisions as well?

Aruki

2015-01-26 07:01:28 pm

No, the MREA maxscript only imports terrain. Collision is completely separate. You'd need a different script.

skysurf3000

2015-01-27 03:37:25 am

Quote from Antidote:

No, that's just padding. But when you export you should probably go ahead and pad to the nearest 32bytes

Ok, thanks!

Quote from Antidote:

skysurf, turn off smooth bends please, poor kittens being slaughtered by them ;.;

Haha I'll save the kittens I promise! I'm very bad at anything that has to do with style choices by the way, so if you have other ideas that could save kitties do not hesitate ^^

MrSinistar

Edit history:

MrSinistar: 2015-01-27 05:00:34 am

2015-01-27 05:00:17 am

Antidote and I figured out a lot about SCAN. Here's a fast scan on Parasite Queen, instead of the regular slow Boss fight scan.

Also here's the images loading at different times, by changing the floats in the file.

Aruki

Edit history:

Aruki: 2015-01-28 08:56:37 pm

Aruki: 2015-01-28 01:45:10 am

Aruki: 2015-01-28 01:44:01 am

2015-01-27 04:39:31 pm

If anyone's interested, I've dumped Miles's script object templates for PWE onto the wiki. For now it's just in my sandbox until someone makes proper pages for them, but if you want to know about the structure of one of the objects, you might be able to find what you're looking for. Keep in mind it's incomplete; there are a few missing objects and a LOT of unknowns. Feel free to research some object properties if you want, but just keep in mind it's going to be a thousand times easier to experiment with object properties once there's an actual editor (and PWE's being built specifically with that kind of experimentation in mind). http://www.metroid2002.com/retromodding/wiki/User:Parax0/Sandbox

EDIT: Experimented with writing a preliminary first dependency parser for PakTool. After spending a day or two writing stuff to parse every file format that can have dependencies I finally got the thing to run and actually complete a full parse on the Metroid1 MLVL. End result is a 200 MB pak, compressed, that contains some 26000 files. The parser misses some 900 files, though; all the ANIMs and EVNTs are missing because we don't know the ending of ANCS yet (but also because I forgot I had to parse that part), and that alone accounts for 700 of the missing files. All the DCLNs are missing because the script templates are missing most of them. Finally, a bunch of particles and textures are missing, because the particle formats are a BITCH. Tried to parse all of them but obviously I'm not doing the job well enough.

Good start and a nice first try, but we at the very least need to crack the ending of ANCS and get more complete documentation on the object structures before this'll really become feasible to do.

edit 2: Turns out I was dumb. The only object that uses DCLN is Platform, which Miles did actually mark on the template, and I just missed it when I was writing the object parsing code. So that's fixed.

If anyone's interested, I've dumped Miles's script object templates for PWE onto the wiki. For now it's just in my sandbox until someone makes proper pages for them, but if you want to know about the structure of one of the objects, you might be able to find what you're looking for. Keep in mind it's incomplete; there are a few missing objects and a LOT of unknowns. Feel free to research some object properties if you want, but just keep in mind it's going to be a thousand times easier to experiment with object properties once there's an actual editor (and PWE's being built specifically with that kind of experimentation in mind). http://www.metroid2002.com/retromodding/wiki/User:Parax0/Sandbox

EDIT: Experimented with writing a preliminary first dependency parser for PakTool. After spending a day or two writing stuff to parse every file format that can have dependencies I finally got the thing to run and actually complete a full parse on the Metroid1 MLVL. End result is a 200 MB pak, compressed, that contains some 26000 files. The parser misses some 900 files, though; all the ANIMs and EVNTs are missing because we don't know the ending of ANCS yet (but also because I forgot I had to parse that part), and that alone accounts for 700 of the missing files. All the DCLNs are missing because the script templates are missing most of them. Finally, a bunch of particles and textures are missing, because the particle formats are a BITCH. Tried to parse all of them but obviously I'm not doing the job well enough.

Good start and a nice first try, but we at the very least need to crack the ending of ANCS and get more complete documentation on the object structures before this'll really become feasible to do.

edit 2: Turns out I was dumb. The only object that uses DCLN is Platform, which Miles did actually mark on the template, and I just missed it when I was writing the object parsing code. So that's fixed.

Lord Fredrik

2015-01-28 08:38:27 pm

The RetroModding Wiki is very interesting. I will be checking back on the Tropical Freeze stuff (naturally) from time to time. ;)

Aruki

2015-01-29 03:36:57 pm

I really love how this rewrite is coming out.

BioSpark

2015-01-29 04:32:06 pm

very nice. just curious, how are you writing the prime world editor? are you using qt?

Aruki

2015-01-29 04:35:45 pm

Yeah, using C++/Qt/OpenGL.