Metroid Prime hacking research

2014-08-28 09:27:51 pm

Compression wasn't added for simplicity sake (re: laziness) but you're correct.

2014-08-28 09:51:07 pm

I'm confused how implementing decompression to every CMDL/TXTR/etc reading function is simpler than just decompressing them from the pak in the first place and being done with it, but oh well :P

Antidote

2014-08-28 09:52:47 pm

Like I said, laziness xD, there really was no reason not to add it, other than having to write the decompression code. My pak viewer automatically decompresses the data now.

Antidote

2014-08-28 11:05:15 pm

Quote from MrSinistar:

Phenomenal job, Paraxade! I'm absolutely loving the progress so far, keep up the great work!

Just curious, are there binaries available of Antidote's tools? I tried compiling them myself but I couldn't get it to work in QT.

Didn't see this, I don't have any available at this time, but I highly suggest simply using Paraxade's tools anyway, mine are rendered obsolete by his. Also they're not really completely mine, they were originally written by thakis, and modified by revel8n, interdpth, and myself.

Aruki

Edit history:

Aruki: 2014-08-28 11:12:14 pm

2014-08-28 11:11:48 pm

I don't think I've released any tools that overlap yours aside from PakTool. MrSinistar is asking about stuff like RetroView I believe?

Antidote

2014-08-28 11:18:15 pm

At this point, the only thing that you don't have is a texture dumping util, I have a new version available, but I don't have a windows machine available to build a windows version.

Aruki

Edit history:

Aruki: 2014-08-28 11:45:58 pm

Aruki: 2014-08-28 11:20:52 pm

2014-08-28 11:20:37 pm

Yeah but they aren't released :P I dunno when I would release a version of PWE, possibly after I have some basic object editing supported?

Also PWE actually has texture conversion functions, so I do have that (plus some other utilities I've split the TXTR functionality off into for batch conversion)

Antidote

2014-08-28 11:48:12 pm

Don't worry about object editing for now, you want to get feedback on the UI first then worry about adding something relatively complicated like object editing, (since that can be rather volatile given certain details about the engine)

Aruki

2014-08-28 11:49:04 pm

Um... I'm not sure what there is to give feedback on if there's no functionality whatsoever.

Antidote

2014-08-28 11:54:51 pm

Even if there isn't any functionality, the layout is pretty important.

Aruki

Edit history:

Aruki: 2014-08-29 01:54:18 am

Aruki: 2014-08-29 01:51:21 am

Aruki: 2014-08-29 12:54:20 am

2014-08-29 12:50:36 am

I get your point. I think I'd like to focus on functionality for the time being, though, with the understanding that it means I'm gonna have to overhaul a lot of things to improve the UI later. I wanna get a good handle on what the game is capable of and what sort of editing functionality is needed/desired before I try to design a more comprehensive UI around it.

I'm also pretty sure I want to change it later so it opens/edits MLVLs rather than directly editing MREAs, and that would probably require a UI overhaul as well, so oh well. (I look forward to that because then I'll be able to load the skybox and make it look REALLY cool. :P)

Antidote

2014-08-29 02:09:25 am

We should look at how the engine scales the Skybox, because it's obviously based on camera position rather than global (just use Dolphin's free camera to see what I mean)

Aruki

2014-08-29 02:22:29 am

I don't think it does scale it, it just moves it with the camera and draws everything else on top of it.

Antidote

2014-08-29 02:26:40 am

Not entirely sure about that, that would lead to some weird artifacting (plus going behind talon IV hides the frigate iirc) you may be right though

Aruki

2014-08-29 03:00:57 am

Decided to try setting up the Chozo Ruins skybox the way I described just for the hell of it

Aruki

Edit history:

Aruki: 2014-08-30 12:35:33 am

2014-08-30 12:28:38 am

So alpha is mostly working now. I think the only significant things left to handle on the materials are some minor depth sorting issues that only really pop up in places that have a lot of transparent stuff (like Research Lab Aether), and some problems on textures that use punchthrough alpha. (edit: and material animations, I guess. I'll prolly try supporting those once I've taken a look at how they work) I wanna post a lot of pictures of stuff looking awesome soon, but for now here's another picture of the unused morph ball, this time with the materials rendered correctly and the glass.

Aruki

Edit history:

Aruki: 2015-01-07 01:28:58 am

Aruki: 2014-12-14 07:15:34 pm

Aruki: 2014-12-14 07:15:31 pm

Aruki: 2014-12-14 07:15:30 pm

Aruki: 2014-10-28 04:22:38 am

Aruki: 2014-08-31 05:20:23 am

2014-08-31 05:06:55 am

I'm getting the idea you guys aren't as excited about all this as I am :P But whatever, for those interested, here's a big Imgur album with pictures of a bunch of models from Primes 1 and 2!

I also said I'd post in detail what I've worked out about the material format. There's a LOT, so I'll spoiler tag it for those of you who aren't interested in the technical details:

Quote:

struct prime_material {
u32 flags;
u32 texture_count;
std::vector<u32> texture_IDs;
u32 vertex_attribute_flags; // GX_SetVtxDesc()
u32 unknown1;
u32 konst_count;
u32 konst_color[4]; // GX_SetTevKColor()
u16 blend_dst_factor; // GX_SetBlendMode()
u16 blend_src_factor; // GX_SetBlendMode()
u32 unknown2;
u32 color_channel_flag_count;
std::vector<u32> color_channel_flags;

u32 tev_stage_count; // GX_SetNumTevStages()
struct tev_stage {
u32 color_in_flags; // GX_SetTevColorIn()
u32 alpha_in_flags; // GX_SetTevAlphaIn()
u32 color_op_flags; // GX_SetTevColorOp()
u32 alpha_op_flags; // GX_SetTevAlphaOp()
u8 padding;
u8 konst_alpha_in; // GX_SetTevKAlphaSel()
u8 konst_color_in; // GX_SetTevKColorSel()
u8 rasterized_in; // GX_SetTevOrder()
};
std::vector<tev_stage> tev_stages;
std::vector<u32> tev_tex_in; // GX_SetTevOrder()

u32 texgen_count; // GX_SetNumTexGens()
std::vector<u32> texgen_flags; // GX_SetTexCoordGen2()

u32 material_animations_size;
u32 animation_count;
struct animation {
u32 type; // see this post for more info
std::vector<float> parameters; // type 2/4/5 - four parameters, type 3 - two parameters, others - no parameters
};
std::vector<animation> anim_table;
};

Shoutout to libOGC's gx.h for providing useful information on how the Gamecube's 3D graphics pipeline, GX, works: http://gc.devkitpro.org/gx_8h.html

Metroid Prime's graphics rendering system will probably look familiar to anyone who already understand how GX works, but for anyone who doesn't (like me before I researched it), here's the gist of it. Graphics rendering is done through a series of rendering stages called Texture EnVironment, or TEV stages for short. Prior to a TEV stage, per-vertex lighting calculations and texture coordinate generation (texgen) is performed. The resulting vertex color is passed to TEV as a rasterized color value. Then a couple parameters are passed that specify TEV order - namely, a set of values that describe which texture, UV coordinate, and rasterized color is available to each respective TEV stage.

How TEV stages actually work: Each TEV stage takes in four color (RGB) inputs, and four alpha inputs, from one of eight sources. Then these four input colors are combined into one output color, and the output is saved into one of four registers, which can subsequently be used as input in the next TEV stages. More info on this is below. The final stage must always save its output into the "previous TEV stage" register; that register is used as the final pixel color that gets displayed onscreen.

Here is an explanation of how each of the values works:

*** flags:
These are flags that affect general things on the material related to what data is present and how the material should be rendered.

Bit 1 (0x1): Unused, always set
Bit 2 (0x2): Unused, always set
Bit 3 (0x4): Unused, always unset
Bit 4 (0x8): Enable Konstant values
Bit 5 (0x10): Mark transparent material
Bit 6 (0x20): Enable punchthrough alpha
Bit 7 (0x40): Enable Samus's reflection
Bit 8 (0x80): Enable depth writing
Bit 9 (0x100): ?
Bit 10 (0x200): Don't draw (?)
Bit 11 (0x400): Enable an extra value (unknown2)
Bit 12 (0x800): ?
Bit 13 (0x1000): ?
Bits 14+15 (0x6000): Toggle which UV array UV0 reads from; used on MREA
Bit 16 (0x8000): ?
Bit 17 (0x10000): ?
Bit 18 (0x20000): ?
Bit 19 (0x40000): ?
Bit 20 (0x80000): ?
Bits 21+: Unused

*** vertex_attribute_flags:
These flags are in pairs, and set what attribute vertices using this material have. I believe each flag in the pair actually does something different, but they're always set together.

There is one strange thing I want to point out. GX supports up to eight UV coordinates per vertex, and some other values seem to be working with a max of eight UV coordinates, but the game won't seem to allow you to assign more than seven to one vertex. I'm not 100% sure what's going on here and it's not impossible I made a mistake somewhere (although I don't think I did), so keep that in mind.

Flag 1 (0x3): Position
Flag 2 (0xC): Normal
Flag 3 (0x30): Color 0
Flag 4 (0xC0): Color 1
Flag 5 (0x300): UV 0
Flag 6 (0xC00): UV 1
Flag 7 (0x3000): UV 2
Flag 8 (0xC000): UV 3
Flag 9 (0x30000): UV 4
Flag 10 (0xC0000): UV 5
Flag 11 (0x300000): UV 6
Flags 12+: Unused (Something here is used in Echoes; haven't determined what it is)

*** konst_color:
This value is present if bit 4 on material_flags is set. They allow you to set Konstant values, which can subsequently be used in TEV stages. The maximum number of Konst values you can set on one material is 4. The values themselves are simply 32-bit RGBA values.

*** blend_dst_factor / blend_src_factor:
These set the blending mode used. Most opaque materials will have these set to 0/1, a ton of transparent materials set them to 1/1 for additive blending, and some set it to 5/4 for source alpha/inverse source alpha, for alpha blending. The full list of possible values corresponds to those listed here: http://libogc.devkitpro.org/group__blendfactor.html

*** unknown2:
I have no idea what this does, but it's only present if bit 11 on material_flags is set.

*** color_channel_flags:
Needs research; don't know how these work. The count value indicates how many bytes are present from here until the TEV stages start, but the game only actually reads the first value, so that first one is the only one that matters. Why? Good question; ask Retro.

*** color_in_flags:
These flags contain data for the four colors used as inputs in the corresponding TEV stage. Each input color is allocated five bits, although only four of those five bits is actually used. Each color input is set to one of sixteen possible values.

Values listed are per-input, in binary:

00000: Previous stage RGB
00001: Previous stage AAA
00010: Color 0 RGB
00011: Color 0 AAA
00100: Color 1 RGB
00101: Color 1 AAA
00110: Color 2 RGB
00111: Color 2 AAA
01000: Texture RGB
01001: Texture AAA
01010: Rasterized RGB
01011: Rasterized AAA
01100: One
01101: Half
01110: Konstant RGB
01111: Zero

*** alpha_in_flags:
These flags contain data for the four alpha values used as inputs in the corresponding TEV stage. Each input value is allocated five bits, although only three of these five bits is actually used. Each alpha input is set to one of eight possible values.

Values listed are per-input, in binary:

00000: Previous stage alpha
00001: Color 0 alpha
00010: Color 1 alpha
00011: Color 2 alpha
00100: Texture alpha
00101: Rasterized alpha
00110: Konstant alpha
00111: Zero

*** color_op_flags:
These flags specify how the operation that combines the four color inputs should be performed to create one output color. The values correspond to the parameters passed to the GX function GX_SetTevColorOp(): http://libogc.devkitpro.org/gx_8h.html#a067b875ff1e3e6760d0d7d7838c6c4c2

This is the combiner function:

tevrigid = (d (tevop) ((1.0-c)*a + c*b) + tevbias) * tevscale;

Note that most materials in the game simply enable clamping, set a register, and leave the rest of the values at their defaults of 0. You can get pretty accurate rendering without bothering with most of this stuff (I actually haven't bothered doing it yet).

Bits 1-4: tevop - there are ten accepted values, which correspond to the ones listed here: http://libogc.devkitpro.org/group__tevop.html
Bits 5-6: tevbias - 00 does nothing, 01 adds 0.5, and 10 subtracts 0.5.
Bits 7-8: tevscale - 00 multiplies by 1, 01 multiplies by 2, 10 multiplies by 4, and 11 multiplies by 0.5.
Bit 9: clamp - Clamp value between 0 and 1
Bits 10-11: tevrigid - Set the register that the output color is saved into. 00 = Previous TEV Stage; 01 = Color 0; 10 = Color 1; 11 = Color 2
Bits 12+: Unused

*** alpha_op_flags:
These flags specify how the operation that combines the four alpha inputs should be performed to create one output alpha value. It is exactly the same as color_op_flags, except they operate on alpha instead of color, so refer to the explanation of color_op_flags above for details. The values correspond to the parameters passed to the GX function SetTevAlphaOp(): http://libogc.devkitpro.org/gx_8h.html#a5781722fdc450cb717e6dcc1f67810c7

*** padding:
Does nothing.

*** konst_alpha_in:
Determines which Konstant alpha value is used for the current TEV stage. There are 24 possible settings: http://libogc.devkitpro.org/group__tevkalphasel.html

*** konst_color_in:
Determines which Konstant color value is used for the current TEV stage. There are 28 possible settings: http://libogc.devkitpro.org/group__tevkcolorsel.html

*** rasterized_in:
Determines which rasterized color is used for the current TEV stage. There are 10 possible settings: http://libogc.devkitpro.org/group__channelid.html

*** tev_tex_in:
This value simply sets which texture and UV coordinate is used in each TEV stage. Byte 3 refers to the texture, while byte 4 refers to the UV. Bytes 1 and 2 are unused. A value of 0xFF in either value indicates that the stage does not use a texture/UV.

*** texgen_flags
These flags determine how texture coordinates are generated (texgen). Note that any number of UV coordinates can be generated from any vertex attribute; you aren't limited to how many UV indices are present on the vertex. These are the values that are subsequently used in the TEV stages. Texgen is done by multiplying the raw input coordinate by a texture matrix; after that it is (optionally) normalized, then multiplied by a post-transform matrix.

Bits 1-4: These do... something. They affect how/if the UV's W (or R/Z) component is calculated, but they can also set the texgen to use the vertex position as input, which doesn't make much sense to me because that's supposed to be handled by the next set of bits. I couldn't really figure out specifically what these do. Most materials leave this value at 0, so it's not super important.
Bits 5-9: These determine which vertex attribute is used as the raw input. 0 and 1 correspond to position and normal, 2 and 3 correspond to probably vertex color (untested), and 4-11 correspond to UV coordinates. Anything above that defaults to UV0.
Bits 10-14: These appear unused, although I can't 100% confirm that.
Bit 15: Toggles whether or not the coordinate is normalized after being multiplied by the texture matrix.
Bits 16-21: Modify the index into the post-transform matrix used for the final calculations.
Bits 22+: Unused

*********

That about sums up what I know of the file format. Finally, I want to note the differences between the material format in Prime 1 and 2. Prime 2 has a couple more values than Prime 1 does; there are two notable differences that need to be taken into account when converting between the two. Note that as I haven't yet begun researching Echoes's file formats, I don't know exactly what these values in Echoes actually do yet.

1. The leftmost byte in the vertex attribute flag is actually used in Echoes. When this value is set, there'll be an extra index present on the vertex, prior to the position index. This extra index needs to be removed from the mesh data for compatibility with Prime 1. Since Prime 1 doesn't use this byte, leaving the flag itself as it is probably won't cause any problems, but I would recommend fixing it anyway.
2. There are two extra flags present immediately following the vertex attribute flags. While I don't know what they do, I do know that you can simply remove them for compatibility with Prime 1.

I'm getting the idea you guys aren't as excited about all this as I am :P But whatever, for those interested, [url=http://imgur.com/a/7n0YE]here's a big Imgur album with pictures of a bunch of models from Primes 1 and 2![/url]

I also said I'd post in detail what I've worked out about the material format. There's a LOT, so I'll spoiler tag it for those of you who aren't interested in the technical details:

[spoiler][quote]struct prime_material {
    u32 flags;
    u32 texture_count;
    std::vector<u32> texture_IDs;
    u32 vertex_attribute_flags; // [url=http://libogc.devkitpro.org/gx_8h.html#af41b45011ae731ae5697b26b2bf97e2f]GX_SetVtxDesc()[/url]
    u32 unknown1;
    u32 konst_count;
    u32 konst_color[4];  // [url=http://libogc.devkitpro.org/gx_8h.html#a4d35fcfc635753a469c2d77325cd9305]GX_SetTevKColor()[/url]
    u16 blend_dst_factor; // [url=http://libogc.devkitpro.org/gx_8h.html#a1d9c43b161f3c5a30b9fd8ea182c8eb6]GX_SetBlendMode()[/url]
    u16 blend_src_factor; // [url=http://libogc.devkitpro.org/gx_8h.html#a1d9c43b161f3c5a30b9fd8ea182c8eb6]GX_SetBlendMode()[/url]
    u32 unknown2;
    u32 color_channel_flag_count;
    std::vector<u32> color_channel_flags;

u32 tev_stage_count; // [url=http://libogc.devkitpro.org/gx_8h.html#a408612e8fdbd9e0ff0fd78edce13e386]GX_SetNumTevStages()[/url]
    struct tev_stage {
        u32 color_in_flags; // [url=http://libogc.devkitpro.org/gx_8h.html#a54d21b7ad8f5edfcc4d055d21f1a6870]GX_SetTevColorIn()[/url]
        u32 alpha_in_flags; // [url=http://libogc.devkitpro.org/gx_8h.html#a4a9970519c90a72b0a5b161d3252bb76]GX_SetTevAlphaIn()[/url]
        u32 color_op_flags; // [url=http://libogc.devkitpro.org/gx_8h.html#a067b875ff1e3e6760d0d7d7838c6c4c2]GX_SetTevColorOp()[/url]
        u32 alpha_op_flags; // [url=http://libogc.devkitpro.org/gx_8h.html#a5781722fdc450cb717e6dcc1f67810c7]GX_SetTevAlphaOp()[/url]
        u8 padding;
        u8 konst_alpha_in; // [url=http://libogc.devkitpro.org/gx_8h.html#a214f1fb5cd2b770fa197b774e07f8fbb]GX_SetTevKAlphaSel()[/url]
        u8 konst_color_in; // [url=http://libogc.devkitpro.org/gx_8h.html#a5197b7367619940d411a3d33ab572e99]GX_SetTevKColorSel()[/url]
        u8 rasterized_in; // [url=http://libogc.devkitpro.org/gx_8h.html#ae64799e52298de39efc74bf989fc57f5]GX_SetTevOrder()[/url]
    };
    std::vector<tev_stage> tev_stages;
    std::vector<u32> tev_tex_in; // [url=http://libogc.devkitpro.org/gx_8h.html#ae64799e52298de39efc74bf989fc57f5]GX_SetTevOrder()[/url]

u32 texgen_count; // [url=http://libogc.devkitpro.org/gx_8h.html#a55a79a1688d3a6957ee0c37d6323d159]GX_SetNumTexGens()[/url]
    std::vector<u32> texgen_flags; // [url=http://libogc.devkitpro.org/gx_8h.html#a55a426a3ff796db584302bddd829f002]GX_SetTexCoordGen2()[/url]

u32 material_animations_size;
    u32 animation_count;
    struct animation {
      u32 type; // see [url=https://m2k2.taigaforum.com/post/metroid_prime_hacking_research_732.html]this post[/url] for more info
      std::vector<float> parameters; // type 2/4/5 - four parameters, type 3 - two parameters, others - no parameters
    };
    std::vector<animation> anim_table;
};

Shoutout to libOGC's gx.h for providing useful information on how the Gamecube's 3D graphics pipeline, GX, works: http://gc.devkitpro.org/gx_8h.html

Metroid Prime's graphics rendering system will probably look familiar to anyone who already understand how GX works, but for anyone who doesn't (like me before I researched it), here's the gist of it. Graphics rendering is done through a series of rendering stages called Texture EnVironment, or TEV stages for short. Prior to a TEV stage, per-vertex lighting calculations and texture coordinate generation (texgen) is performed. The resulting vertex color is passed to TEV as a rasterized color value. Then a couple parameters are passed that specify TEV order - namely, a set of values that describe which texture, UV coordinate, and rasterized color is available to each respective TEV stage.

How TEV stages actually work: Each TEV stage takes in four color (RGB) inputs, and four alpha inputs, from one of eight sources. Then these four input colors are combined into one output color, and the output is saved into one of four registers, which can subsequently be used as input in the next TEV stages. More info on this is below. The final stage must always save its output into the "previous TEV stage" register; that register is used as the final pixel color that gets displayed onscreen.

Here is an explanation of how each of the values works:

*** flags:
These are flags that affect general things on the material related to what data is present and how the material should be rendered.

Bit 1 (0x1): Unused, always set
Bit 2 (0x2): Unused, always set
Bit 3 (0x4): Unused, always unset
Bit 4 (0x8): Enable Konstant values
Bit 5 (0x10): Mark transparent material
Bit 6 (0x20): Enable punchthrough alpha
Bit 7 (0x40): Enable Samus's reflection
Bit 8 (0x80): Enable depth writing
Bit 9 (0x100): ?
Bit 10 (0x200): Don't draw (?)
Bit 11 (0x400): Enable an extra value (unknown2)
Bit 12 (0x800): ?
Bit 13 (0x1000): ?
Bits 14+15 (0x6000): Toggle which UV array UV0 reads from; used on MREA
Bit 16 (0x8000): ?
Bit 17 (0x10000): ?
Bit 18 (0x20000): ?
Bit 19 (0x40000): ?
Bit 20 (0x80000): ?
Bits 21+: Unused

*** vertex_attribute_flags:
These flags are in pairs, and set what attribute vertices using this material have. I believe each flag in the pair actually does something different, but they're always set together.

There is one strange thing I want to point out. GX supports up to eight UV coordinates per vertex, and some other values seem to be working with a max of eight UV coordinates, but the game won't seem to allow you to assign more than seven to one vertex. I'm not 100% sure what's going on here and it's not impossible I made a mistake somewhere (although I don't think I did), so keep that in mind.

Flag 1  (0x3):      Position
Flag 2  (0xC):      Normal
Flag 3  (0x30):     Color 0
Flag 4  (0xC0):     Color 1
Flag 5  (0x300):    UV 0
Flag 6  (0xC00):    UV 1
Flag 7  (0x3000):   UV 2
Flag 8  (0xC000):   UV 3
Flag 9  (0x30000):  UV 4
Flag 10 (0xC0000):  UV 5
Flag 11 (0x300000): UV 6
Flags 12+: Unused (Something here is used in Echoes; haven't determined what it is)

*** konst_color:
This value is present if bit 4 on material_flags is set. They allow you to set Konstant values, which can subsequently be used in TEV stages. The maximum number of Konst values you can set on one material is 4. The values themselves are simply 32-bit RGBA values.

*** blend_dst_factor / blend_src_factor:
These set the blending mode used. Most opaque materials will have these set to 0/1, a ton of transparent materials set them to 1/1 for additive blending, and some set it to 5/4 for source alpha/inverse source alpha, for alpha blending. The full list of possible values corresponds to those listed here: http://libogc.devkitpro.org/group__blendfactor.html

*** unknown2:
I have no idea what this does, but it's only present if bit 11 on material_flags is set.

*** color_channel_flags:
Needs research; don't know how these work. The count value indicates how many bytes are present from here until the TEV stages start, but the game only actually reads the first value, so that first one is the only one that matters. Why? Good question; ask Retro.

*** color_in_flags:
These flags contain data for the four colors used as inputs in the corresponding TEV stage. Each input color is allocated five bits, although only four of those five bits is actually used. Each color input is set to one of sixteen possible values.

Values listed are per-input, in binary:

00000: Previous stage RGB
00001: Previous stage AAA
00010: Color 0 RGB
00011: Color 0 AAA
00100: Color 1 RGB
00101: Color 1 AAA
00110: Color 2 RGB
00111: Color 2 AAA
01000: Texture RGB
01001: Texture AAA
01010: Rasterized RGB
01011: Rasterized AAA
01100: One
01101: Half
01110: Konstant RGB
01111: Zero

*** alpha_in_flags:
These flags contain data for the four alpha values used as inputs in the corresponding TEV stage. Each input value is allocated five bits, although only three of these five bits is actually used. Each alpha input is set to one of eight possible values.

Values listed are per-input, in binary:

00000: Previous stage alpha
00001: Color 0 alpha
00010: Color 1 alpha
00011: Color 2 alpha
00100: Texture alpha
00101: Rasterized alpha
00110: Konstant alpha
00111: Zero

*** color_op_flags:
These flags specify how the operation that combines the four color inputs should be performed to create one output color. The values correspond to the parameters passed to the GX function GX_SetTevColorOp(): http://libogc.devkitpro.org/gx_8h.html#a067b875ff1e3e6760d0d7d7838c6c4c2

This is the combiner function:

tevrigid = (d (tevop) ((1.0-c)*a + c*b) + tevbias) * tevscale;

Note that most materials in the game simply enable clamping, set a register, and leave the rest of the values at their defaults of 0. You can get pretty accurate rendering without bothering with most of this stuff (I actually haven't bothered doing it yet).

Bits 1-4: tevop - there are ten accepted values, which correspond to the ones listed here: http://libogc.devkitpro.org/group__tevop.html
Bits 5-6: tevbias - 00 does nothing, 01 adds 0.5, and 10 subtracts 0.5.
Bits 7-8: tevscale - 00 multiplies by 1, 01 multiplies by 2, 10 multiplies by 4, and 11 multiplies by 0.5.
Bit 9: clamp - Clamp value between 0 and 1
Bits 10-11: tevrigid - Set the register that the output color is saved into. 00 = Previous TEV Stage; 01 = Color 0; 10 = Color 1; 11 = Color 2
Bits 12+: Unused

*** alpha_op_flags:
These flags specify how the operation that combines the four alpha inputs should be performed to create one output alpha value. It is exactly the same as color_op_flags, except they operate on alpha instead of color, so refer to the explanation of color_op_flags above for details. The values correspond to the parameters passed to the GX function SetTevAlphaOp(): http://libogc.devkitpro.org/gx_8h.html#a5781722fdc450cb717e6dcc1f67810c7

*** padding:
Does nothing.

*** konst_alpha_in:
Determines which Konstant alpha value is used for the current TEV stage. There are 24 possible settings: http://libogc.devkitpro.org/group__tevkalphasel.html

*** konst_color_in:
Determines which Konstant color value is used for the current TEV stage. There are 28 possible settings: http://libogc.devkitpro.org/group__tevkcolorsel.html

*** rasterized_in:
Determines which rasterized color is used for the current TEV stage. There are 10 possible settings: http://libogc.devkitpro.org/group__channelid.html

*** tev_tex_in:
This value simply sets which texture and UV coordinate is used in each TEV stage. Byte 3 refers to the texture, while byte 4 refers to the UV. Bytes 1 and 2 are unused. A value of 0xFF in either value indicates that the stage does not use a texture/UV.

*** texgen_flags
These flags determine how texture coordinates are generated (texgen). Note that any number of UV coordinates can be generated from any vertex attribute; you aren't limited to how many UV indices are present on the vertex. These are the values that are subsequently used in the TEV stages. Texgen is done by multiplying the raw input coordinate by a texture matrix; after that it is (optionally) normalized, then multiplied by a post-transform matrix.

Bits 1-4: These do... something. They affect how/if the UV's W (or R/Z) component is calculated, but they can also set the texgen to use the vertex position as input, which doesn't make much sense to me because that's supposed to be handled by the next set of bits. I couldn't really figure out specifically what these do. Most materials leave this value at 0, so it's not super important.
Bits 5-9: These determine which vertex attribute is used as the raw input. 0 and 1 correspond to position and normal, 2 and 3 correspond to probably vertex color (untested), and 4-11 correspond to UV coordinates. Anything above that defaults to UV0.
Bits 10-14: These appear unused, although I can't 100% confirm that.
Bit 15: Toggles whether or not the coordinate is normalized after being multiplied by the texture matrix.
Bits 16-21: Modify the index into the post-transform matrix used for the final calculations.
Bits 22+: Unused

*********

That about sums up what I know of the file format. Finally, I want to note the differences between the material format in Prime 1 and 2. Prime 2 has a couple more values than Prime 1 does; there are two notable differences that need to be taken into account when converting between the two. Note that as I haven't yet begun researching Echoes's file formats, I don't know exactly what these values in Echoes actually do yet.

1. The leftmost byte in the vertex attribute flag is actually used in Echoes. When this value is set, there'll be an extra index present on the vertex, prior to the position index. This extra index needs to be removed from the mesh data for compatibility with Prime 1. Since Prime 1 doesn't use this byte, leaving the flag itself as it is probably won't cause any problems, but I would recommend fixing it anyway.
2. There are two extra flags present immediately following the vertex attribute flags. While I don't know what they do, I do know that you can simply remove them for compatibility with Prime 1.[/quote][/spoiler]

Prime Hunter

2014-08-31 08:41:32 am

Quote from Paraxade:

...for those interested, here's a big Imgur album with pictures of a bunch of models from Primes 1 and 2!

Some pretty cool stuff in there. Being able to see some of the models, such as the various Pirates, when they aren't trying to kill me makes me realize how skilled Retro's artists were even back then. Emperor Ing especially based on your comment. It couldn't have been an easy task to pull off that level of detail within a single texture's resolution for a game that's 10 years old. Although then you see how poor ZZS is compared to the rest and have to wonder how long they spent on her.

And from the bits I can figure out in your description of the process, it seems like Gamecube graphics follow a pretty complex system from start to finish. Quite a bit different from what I remember going through in my college days at the very least, although Prime and Echoes predate that by a few years.

MrSinistar

2014-08-31 12:51:20 pm

Quote from Paraxade:

I'm getting the idea you guys aren't as excited about all this as I am :P

No dude, this is freaking fantastic stuff!! Thanks so much for all your hard work :) I can't wait to poke around the files myself.

Thanks so much for uploading the Imgur gallery too! The texture and model work is absolutely incredible...so much detail; it still holds up very, very well to this day.

Antidote

Edit history:

Antidote: 2014-08-31 02:06:15 pm

2014-08-31 01:47:41 pm

You know I'm pretty excited about your progress.
ZZS From what I understand was added at the last possible second.

Caveberry

2014-09-01 12:17:29 am

getting there...

Wow, I really dig this sort of stuff. I would love to see how the impact crater is modeled.

We could play a game where you post a picture of a room's collision model and we guess its' name. Fun for everyone!

But seriously, I can't wait to see what comes of this. What's the endgame for this? Are we going to ever see stuff like challenge rooms with timers that put all our speedrunning tricks to the test?

Aruki

Edit history:

Aruki: 2014-09-01 12:34:42 am

Aruki: 2014-09-01 12:34:30 am

2014-09-01 12:22:02 am

Quote from Caveberry:

Wow, I really dig this sort of stuff. I would love to see how the impact crater is modeled.

Anything you wanna see in particular? I can try to grab a picture for you.

Quote:

But seriously, I can't wait to see what comes of this. What's the endgame for this? Are we going to ever see stuff like challenge rooms with timers that put all our speedrunning tricks to the test?

haha, that's actually possible. I'd be down for making that happen if someone wants to put together some level design and basic art assets (mostly textures). (Won't be til custom room are a thing though.)

Antidote

2014-09-01 12:56:21 am

Quote from Caveberry:

We could play a game where you post a picture of a room's collision model and we guess its' name. Fun for everyone!

That would be easier than you'd think, the collision models are fairly detailed.

Aruki

2014-09-01 12:57:09 am

I think people on this site know better than anyone how detailed the collision models are xP

Caveberry

Edit history:

Caveberry: 2014-09-01 01:08:32 am

2014-09-01 12:58:55 am

getting there...

Oh fuck yeah, I'm pulling out the graph paper now.

I might need some basic measurements though. (i.e. height of a jump/SJ/BJ/UBJ, max distance of a dash jump, etc.)

What unit of measure is the game modeled around anway?

I'm actually getting some really cool ideas as I'm typing this up...

EDIT: You both are probably right about the collision maps being easy to guess. After all, one the things that made me fall in love with this game is the fact that you can utilize tiny pieces of geometry to your advantage. There are almost no arbitrary invisible walls. That being said, parax posted some collision maps a few pages back, and I could barely make heads or tales of them. I think it has something to do with the camera being at a position that it couldn't possibly be, in game.