Melee dat format...

[Achilles1515]

Does it also disappear when the last byte is set to something like 22 or 1c. I havent really experimented with 00.
Also yeah the mesh attribute array points to it. Id guess that about half of them are simply one color of gray though so those could be replaced with the color node. Thought I know playing with the alpha value of the color there makes the object transparency change accordingly which sounds like what you want to try.

I posted more in my post above btw. Hopefully I can get some more testing done tomorrow.

 

[DRGN]

Have you guys seen my recent notes in the section "Adjusting Model Part Transparency" here?

The documentation shown there assumes two bytes of unknown flags at 0x4 in the material structure, followed by two more bytes of unknown data. In DTW, these are shown as "Unknown Flags", and "Unknown 1". I take it these are probably all flags, and I should probably combine them to 4 bytes of "Unknown Flags". Do you know how we can identify all of these as individual bitwise booleans, rather than perhaps some other integers?

I think the most important thing in those notes is the connection between the unknown flags in joint/bone structures and the unknown flags in the child material structure's unknown flags. If they weren't set "properly" in respect to each other, the objects would never render, regardless of any changes to either set of flags on their own.

Idk. I'm still trying to work out color structs and transparency flags. I also don't think you can enable specular lighting and color nodes at the same time. The metal cavern import is confusing me, because anytime I put a (40000000) in the material struct flag, the object completely disappears. Do you know why this occurs? Because I think you have to have that enabled to use the transparency value in the color struct.

Have you tried 60000000?

I also found that if you have many nested child node or object structures which have these unknown flags, transparency support would not propagate up through the tree unless ALL of the structures allowed for transparency.

Of course, my experiments were with character files, but they have similar structuring, at least at certain levels in the hierarchy, so I would guess the same associations between these two sets of unknown flags (joint & material) would be there for stages.

 

[Achilles1515]

Have you guys seen my recent notes in the section "Adjusting Model Part Transparency" here?

The documentation shown there assumes two bytes of unknown flags at 0x4 in the material structure, followed by two more bytes of unknown data. In DTW, these are shown as "Unknown Flags", and "Unknown 1". I take it these are probably all flags, and I should probably combine them to 4 bytes of "Unknown Flags". Do you know how we can identify all of these as individual bitwise booleans, rather than perhaps some other integers?

I think the most important thing in those notes is the connection between the unknown flags in joint/bone structures and the unknown flags in the child material structure's unknown flags. If they weren't set "properly" in respect to each other, the objects would never render, regardless of any changes to either set of flags on their own.



Have you tried 60000000?

I also found that if you have many nested child node or object structures which have these unknown flags, transparency support would not propagate up through the tree unless ALL of the structures allowed for transparency.

Of course, my experiments were with character files, but they have similar structuring, at least at certain levels in the hierarchy, so I would guess the same associations between these two sets of unknown flags (joint & material) would be there for stages.

Yeah I tried that flag and looked at your guide in regards to the bone flag but couldn't get anything working. I've never heard of your comment about all the object structs needing the transparency flag. That is really interesting and never something I thought of.

 

[zankyou]

Wait my bad. If I had to guess your issue it would be that I ordered the objects wrong. I didnt realize it mattered when I originally did that stage. You could try swapping the last bone with the first bone order wise. That might fix it.

 

[zankyou]

How you like them crystals!
(still testing some things)

View attachment 109837

EDIT:
Nvm. Even f**king better.

View attachment 109838

Until right now I didnt even realize the crystal wasnt imported properly. I wonder what I did wrong to cause this.
What exactly did you do to get the shading to work. Im thinking that if I combine it with the blend struct something really amazing can come of it. And I move to change the name of the unk struct in the text struct to blend struct since thats what it is.

 

[Achilles1515]

Until right now I didnt even realize the crystal wasnt imported properly. I wonder what I did wrong to cause this.
What exactly did you do to get the shading to work. Im thinking that if I combine it with the blend struct something really amazing can come of it. And I move to change the name of the unk struct in the text struct to blend struct since thats what it is.

Confirmed on one of your previous imports that you can combine transparency (color struct) with specular lighting. Sweet! That's what the crystals need.

While doing so, I made an interesting discovery, if that's what you want to call it.
Let's go back to the Hyrule Castle import.

Remember how that stage had "broken" color nodes on the tent and a few side walls?

Well, Edwguard Flows made the tents/roofing have specular lighting.

If I enable the flags for specular lighting on the combo tent, it turns out like this:

SO MUCH BETTER! And I only added the lighting effect to the main green roof material, not the light green at the top, or the fencing, or the little jut out green piece before the lip on the entrance ways - so it can look even better.

So my "discovery" might be that all your "broken" color nodes in your imports, might just be that those materials had specular lighting enabled in the brawl file, meaning the color struct doesn't matter and could have been bad in the first place. I'm sure this is why some of your stages look too dark as well.

To enable specular lighting:
Bone struct flags contains (00010080)
Material struct flags contains (0000000C)
*I just updated my post from a few posts up for the specular lighting requirements to match these

For the above Hyrule example, the bone struct flags = 00150080 and material flag = 0000000C

 

[zankyou]

So brawl uses shaders which have the final say in what the object looks like. Im pretty sure this is why the stages are so dark since melee doesnt have that. In general all default stages uses the color 808080 b3b3b3 or something like that for diffuse and ambient but dont look as dark.
Brawl also has light channels which I assume is the equivalent to melees color nodes. Though for brawl both are active and in melee you have to pick one.
Im playing with those flags now.

 

[DRGN]

Confirmed on one of your previous imports that you can combine transparency (color struct) with specular lighting. Sweet! That's what the crystals need.

While doing so, I made an interesting discovery, if that's what you want to call it.
Let's go back to the Hyrule Castle import.

Remember how that stage had "broken" color nodes on the tent and a few side walls?

View attachment 109927

Well, Edwguard Flows made the tents/roofing have specular lighting.

View attachment 109928

If I enable the flags for specular lighting on the combo tent, it turns out like this:

View attachment 109929

SO MUCH BETTER! And I only added the lighting effect to the main green roof material, not the light green at the top, or the fencing, or the little jut out green piece before the lip on the entrance ways - so it can look even better.

So my "discovery" might be that all your "broken" color nodes in your imports, might just be that those materials had specular lighting enabled in the brawl file, meaning the color struct doesn't matter and could have been bad in the first place. I'm sure this is why some of your stages look too dark as well.

To enable specular lighting:
Bone struct flags contains (00010080)
Material struct flags contains (0000000C)
*I just updated my post from a few posts up for the specular lighting requirements to match these

For the above Hyrule example, the bone struct flags = 00150080 and material flag = 0000000C

Nice, Achilles! When I was playing with those flags I thought about those black parts on that stage, but never got around to testing on it. That looks so good now.

I'm very interested to find out exactly which bits/flags are for what. For example, when you say that the last byte of the flags needs to be 0x0C, 0000 1100, to enable specular lighting, do you know if it's the bit in blue or the bit in green, or actually both? And then which flags/bits of 00010080? Are they really all needed? Rather than being multiple flags for the same thing, I wonder if each enables some sub-component/function of specular lighting, and thus it only works when they're all set. In which case I wonder what the sub-components are.

Btw, what structure exactly are you guys referring to as "color nodes"?

 

[zankyou]

Color node is whats referenced by the material structure and colors the entire object. Color struct is whats referenced in the mesh attribute array and colors polygons. It was never really an issue until now because there were no color structs in vanilla. But the game supports them.

 

[DRGN]

Ah. The structure on the Material Structure for color I've been calling the Color Structure or Color Struct. That's how I labeled it in DTW, just as other documentation I've seen (like the mario kart wiki). From earlier in the thread, I thought that Color Nodes were tied more directly with meshes.

 

[zankyou]

Ah. The structure on the Material Structure for color I've been calling the Color Structure or Color Struct. That's how I labeled it in DTW, just as other documentation I've seen (like the mario kart wiki). From earlier in the thread, I thought that Color Nodes were tied more directly with meshes.

Yeah definitely a problem with different people naming everything. Originally learned everything from the melee dat format thread and its called color nodes there and in this doc. https://docs.google.com/document/d/...07BQ3c4Q2Wms5I1OU/edit#heading=h.1tza9z44vpir
But it also calls everything else a node so I am being inconsistent. Suggestions?

 

[zankyou]

So I just spent was too much of my life going through every material and bone flag of an import and changing the color node according the brawls light channels to see what exactly was being changed with those flags

material flag 0xXX000002

material flag 0xXX00000C

As you can see. Theres no support for colors with the 0x0c flag but you get the specular lighting effect thats been missing and making the stages too dark. Also the red steel beams in the background dont appear unless the flag is 0x0c. Ideally there would be a way of combining the 0x0c and 0x

 

[zankyou]

I thought I fixed it with 0x0e flag since looking at the bits it made sense. But it only works on the crystals and I cant figure out whats different about that one
edit:

0x0a combines the color node and the color struct with the specular effect. Its a little too shiny imo but it looks a lot better.
Now to go back through the imports I just did

edit: ignore that. what the bits do are talked about down below.

 

[Achilles1515]

DRGN Z zankyou

I have been using the terms of the HAL DAT Format Wiki. So we have object structs, material structs, texture structs, mesh structs....so I think it makes sense the one coming off the material struct is just the color struct. And I've been referring to the colors in the mesh struct as color nodes. I think "nodes" makes sense for this because mesh points come together at nodes.

DRGN, the bits certainly mean different things. Here is what I have labeled so far, but I'm still not sure about my descriptions, specifically the last two.

Material Structure Flags
(40000000) = enable color struct transparency
(04000000) = enable character shadow on material
(00000002) = enable mesh color nodes
(00000004) = enable diffuse lighting (?), incompatible with mesh color nodes
(00000008) = enable specular lighting (shininess?)

Zankyou, our crystals are still different because you did not enable the (00000004) flag in the material struct. If what you are trying to do is create a one-size fits all material structure flag value, I don't think that is a good idea - things need to be different for it to look right.

 

[zankyou]

DRGN Z zankyou

I have been using the terms of the HAL DAT Format Wiki. So we have object structs, material structs, texture structs, mesh structs....so I think it makes sense the one coming off the material struct is just the color struct. And I've been referring to the colors in the mesh struct as color nodes. I think "nodes" makes sense for this because mesh points come together at nodes.

DRGN, the bits certainly mean different things. Here is what I have labeled so far, but I'm still not sure about my descriptions, specifically the last two.

Material Structure Flags
(40000000) = enable color struct transparency
(04000000) = enable character shadow on material
(00000002) = enable mesh color nodes
(00000004) = enable diffuse lighting (?), incompatible with mesh color nodes
(00000008) = enable specular lighting (shininess?)

Zankyou, our crystals are still different because you did not enable the (00000004) flag in the material struct. If what you are trying to do is create a one-size fits all material structure flag value, I don't think that is a good idea - things need to be different for it to look right.

You cant enable the 0100 and 0010. 0110 does nothing. its why 0000000e didnt work as I wanted. Which makes sense as there really isnt a need for both. I figured out the root problem which is that the crystals have a shimmer effect which changes their colors dynamically. So thats why their colors arent perfect. Technically the textures scale is supposed to be 0404 but without the correct colors it looks absolutely horrible.
The weird thing is that 1110 has an effect that only works on the crystals. I think its because of the image type.
those are the bits btw.

So basically I dont see a reason to use a flag other than 0x02 or 0x0c if the color group (its what the mdl0 wiki calls it) isnt present and 0x0a when specular lighting is needed. At least until I understand what 0x0e actually does.

And yeah color struct is more consistent for the one in the material struct. I think tcll calls everything structs and revel calls everything nodes. But no one has named the things the mesh attribute array points to. So to be consistent vertex group, uv group, color group, normal group are the things it points to. Does this work?

Also does anyone know how exactly the specular and shininess values work. the specular color seems to ignore the alpha value and instead use the shininess value. But a shininess value of zero makes every thing facing the light source white.

 

[Achilles1515]

Can

You cant enable the 0100 and 0010. 0110 does nothing. its why 0000000e didnt work as I wanted. Which makes sense as there really isnt a need for both. I figured out the root problem which is that the crystals have a shimmer effect which changes their colors dynamically. So thats why their colors arent perfect. Technically the textures scale is supposed to be 0404 but without the correct colors it looks absolutely horrible.
The weird thing is that 1110 has an effect that only works on the crystals. I think its because of the image type.
those are the bits btw.

So basically I dont see a reason to use a flag other than 0x02 or 0x0c if the color group (its what the mdl0 wiki calls it) isnt present and 0x0a when specular lighting is needed. At least until I understand what 0x0e actually does.

And yeah color struct is more consistent for the one in the material struct. I think tcll calls everything structs and revel calls everything nodes. But no one has named the things the mesh attribute array points to. So to be consistent vertex group, uv group, color group, normal group are the things it points to. Does this work?

Also does anyone know how exactly the specular and shininess values work. the specular color seems to ignore the alpha value and instead use the shininess value. But a shininess value of zero makes every thing facing the light source white.

I'm fine with color group.

Can you upload a new WIP of metal cavern so that we are doing things on the same file?

 

[zankyou]

Can

I'm fine with color group.

Can you upload a new WIP of metal cavern so that we are doing things on the same file?

Yeah when I get home Ill do it.

 

[Achilles1515]

Yeah when I get home Ill do it.

And will this be the version with the new rotation radians/degrees thing you figured out?

 

[DRGN]

I have been using the terms of the HAL DAT Format Wiki. So we have object structs, material structs, texture structs, mesh structs....so I think it makes sense the one coming off the material struct is just the color struct. And I've been referring to the colors in the mesh struct as color nodes. I think "nodes" makes sense for this because mesh points come together at nodes.

And yeah color struct is more consistent for the one in the material struct. I think tcll calls everything structs and revel calls everything nodes. But no one has named the things the mesh attribute array points to. So to be consistent vertex group, uv group, color group, normal group are the things it points to. Does this work?

Yeah, I agree. Sounds good.

When talking about the flags, I like doing so in binary, so we don't lose sight of what we're really doing. I think we'll be more open to catch patterns and correlations. And this makes more sense when comparing or considering transferring properties from one structure to another. The game will probably be more stable if we only switch the specific flags/bits we need, rather than copying a whole string or byte of hex, since we don't know yet what the rest of the bits do. Also, it might be clearer for others that are new and following along to understand what's really being changed.

Also does anyone know how exactly the specular and shininess values work. the specular color seems to ignore the alpha value and instead use the shininess value. But a shininess value of zero makes every thing facing the light source white.

Documentation from my testing of the color struct's values is here, if you go to the Object Properties section and scroll down a little to Color Structure. It's expanded on from what I learned from Lanceinthepants' guide (the 'Changing Color Effects' thread).

 

[zankyou]

Material structure for the crystal is at 0x12da24
The diffuse and ambient colors in the color struct are the same as in the mdl0 file.
grnba
waters still broken but this is close to how it will probably look in the end.

 

[zankyou]

Im kinda curious of what the first byte in the material flag means. Its usually 00 10 40 or 60. I know the low bits are for the shadow. The high bit seems to correspond to what brawl calls the alpha function.

 

[Achilles1515]

Material structure for the crystal is at 0x12da24
The diffuse and ambient colors in the color struct are the same as in the mdl0 file.
grnba
waters still broken but this is close to how it will probably look in the end.

I left my laptop at work yesterday and haven't been able to check this out yet. I should be able to tonight, but let me know if there is a newer version to download by that time.

 

[Achilles1515]

grst
the object group is at 0xd1478. its kind of an old file from the first import and when I was testing this more extensively.

My thoughts are as follows:

Anytime I've seen a vanilla DAT file use "next texture struct" pointers, the two images in play have the exact same properties of size and image type. And they are the same image, just with different color schemes to presumably create some sort of accent effect.

In this uploaded GrSt file, you have a 128x128 texture calling a 256x256 texture (that I presume are going to try and use the exact same meshes?). Idk what the end effect you were going for is, though.

As you are aware, the file you gave me crashes the game. But this has nothing to do with texture structs. It is crashing because you have the same pointer twice in the relocation table, for two different pointers.

I'm guessing this was just a mistake during the import.

If you fix this issue, the stage boots up and looks like this.

I have no idea what else you did with this file, so I am not going to make anymore comments about it.

 

[zankyou]

My thoughts are as follows:

Anytime I've seen a vanilla DAT file use "next texture struct" pointers, the two images in play have the exact same properties of size and image type. And they are the same image, just with different color schemes to presumably create some sort of accent effect.

In this uploaded GrSt file, you have a 128x128 texture calling a 256x256 texture (that I presume are going to try and use the exact same meshes?). Idk what the end effect you were going for is, though.

As you are aware, the file you gave me crashes the game. But this has nothing to do with texture structs. It is crashing because you have the same pointer twice in the relocation table, for two different pointers.

View attachment 110181

I'm guessing this was just a mistake during the import.

If you fix this issue, the stage boots up and looks like this.

View attachment 110180

I have no idea what else you did with this file, so I am not going to make anymore comments about it.

Wow. I made a script to sort the relocation just so that it would catch these mistakes I make and this happens.
Anyways the reason this is important is because its how the mdl0 file does it. What I was doing before was making the object again but that also doubles the amount of polygons used. So hopefully we can make it less intensive if this works.
Im not sure why Im not getting your pic, but after doing that fix and playing with the flags a bit I can kinda see whats going on.
Changing the word I usually have set as 0x00340010 to 0x00340020 shows that the would you change will be predominant.

setting both yields

The end result should be an overlay of the two.

 

[Achilles1515]

Wow. I made a script to sort the relocation just so that it would catch these mistakes I make and this happens.
Anyways the reason this is important is because its how the mdl0 file does it. What I was doing before was making the object again but that also doubles the amount of polygons used. So hopefully we can make it less intensive if this works.
Im not sure why Im not getting your pic, but after doing that fix and playing with the flags a bit I can kinda see whats going on.
Changing the word I usually have set as 0x00340010 to 0x00340020 shows that the would you change will be predominant.

setting both yields

The end result should be an overlay of the two.

Can you actually give me this new file?

 

[zankyou]

Can you actually give me this new file?

grst
I tried swapping the texture structs but it didnt do anything.

edit: it might be the first byte. The 0x34. I tried changing them to random numbers and saw some success.
at 0xd0658 -> 0x00160010
at 0xd05c8 -> 0x00040010

So I got it. For those interested in the flags, the second byte of that flag determines if multiple textures are loaded or not. 0x16 seems to work consistently. This obviously needs to be tested with more meshes though. it seems the 0x04 bit needs to be set in order for the color struct to work. the 0x10 bit is the only one ive had success getting multiple textures.
To think I abandoned this because I didnt catch the relocation duplicates. Thanks for being my second eye achilles.

 

[Achilles1515]

L

grst
I tried swapping the texture structs but it didnt do anything.

edit: it might be the first byte. The 0x34. I tried changing them to random numbers and saw some success.
at 0xd0658 -> 0x00160010
at 0xd05c8 -> 0x00040010

So I got it. For those interested in the flags, the second byte of that flag determines if multiple textures are loaded or not. 0x16 seems to work consistently. This obviously needs to be tested with more meshes though. it seems the 0x04 bit needs to be set in order for the color struct to work. the 0x10 bit is the only one ive had success getting multiple textures.
To think I abandoned this because I didnt catch the relocation duplicates. Thanks for being my second eye achilles.

Lol nice! I just spent the last half hour looking at it and was going to come back here and say "I figured it out! Set the second byte of that flag to 0x10 and it will overlay." But it looks like you already figured that out. Good work.

Also, I might not have been able to figure that out without DRGN and the amazing DTW. When I loaded up the file, I noticed those wave textures had two image structure headers listed, but they were both stated as being at the same offset. And I know DRGN runs through the relocation table, so I figured it was reading the same thing twice - which it turns out it was because of the double entry.

 

[zankyou]

Seems like theres more to it than just setting the flag to 0x14 for multiple textures. The textures arent overlay in quite the right way. I think its a problem with the order or priority of the textures. Looks like only the first texture struct can have the flag 0x14. 0x10 seems to work for the rest.

 

[Tcll]

I really need an update... haha
I have no idea how this stuff even works XD

lol I'm just getting into GLSL and such, so I'm catching up
but heck, any pointers and such would really help, especially with UMC's viewer plugin

 

[zankyou]

Sorry I havent posted an update on the flags. I still cant figure out how exactly to get the textures to overlay properly. 0x14 makes them too dark but 0x17 makes them too bright.

 

[Achilles1515]

Z zankyou after hours of ASM reversing yesterday, I finally figured out how to modify the stage light source position. And it was right under our noses the whole time. I'll need to finish up looking at a few things later and will make a post about it.

 

[Achilles1515]

So...good news and bad news. My research was soley focused on changing the direction of character shadows via a lighting property and that's exactly what I found. Unfortunately, the lighting source I found is specific to characters and not stage elements.

Format seems to be (x,y,z) location of light source.
For the Hyrule Castle import (GrTPe.dat), these x,y,z float values start @ 0xffa4 with the default values of (-700,700,700) --> 700 to the left, 700 up, and 700 toward the camera. It's located amidst the map_plit stuff.

Original character light source (-700,700,700), with specular lighting effects and color structs enabled on the green roofing (so no use of mesh color groups):

As you expect, it seems like the specular lighting effects on the roof elements follow a similar light source as the shadow.

When I change the character light source to (700,700,700), I get this:

Shading on the character and the cast shadow are as you would expect, with the new light source on the right instead of the left, but the lighting on the roofs did not change. Sadface. But I have some other leads to check in on.

For anyone curious, ASM function @ 8000f38c is the main function in charge of placing a character shadow on-screen.

I also found other interesting functions.

Make the below picture happen (Gecko code):
0400f938 60000000

 

[Achilles1515]

Found the stage element lighting.

Original (specular roofs):

Modified light source:

Same format as the above post, @ 0x280 in the file.

 

[zankyou]

One step closer to perfect stage imports.

 

[zankyou]

So the layer flags in the texture struct seem to correspond to which UV is being used, with 0x4 being the first index.
Also I cant seem to have success with multiple textures on the same material struct. Ive come somewhat close but now quite. Here are a pair of eyes for mario. It should look something like this.

But the closest Ive gotten by changing the halfword at 0x40 in the texture struct.

Why are the Melee Gods so cruel.

 

[Achilles1515]

So the layer flags in the texture struct seem to correspond to which UV is being used, with 0x4 being the first index.
Also I cant seem to have success with multiple textures on the same material struct. Ive come somewhat close but now quite. Here are a pair of eyes for mario. It should look something like this.

But the closest Ive gotten by changing the halfword at 0x40 in the texture struct.

Why are the Melee Gods so cruel.

What are the flag values used in these pictures?

 

[zankyou]

The two halfwords at 0x40 seem to correspond to enabling certain color features and from where to get them. Playing with the second halfword has some pretty weird effects one of which Im pretty sure is used for metal textures.
for the first one
0x8 inverts the colors

0x7

0x2

The red is there because I was playing with the color group. Both of these are far lighter than the other flags: 0x3,0x4, 0x5 which are the normal ones.

0x0 doesnt have any texture. and I cant remember what 0x1 does or 0x6. Also didnt notice a difference between 0x14 and 0x4. Im not really sure whats going on here.

 

[Achilles1515]

I don't understand your use of the "next texture struct pointer" value, and it may just be because I don't get how these images are supposed to be tied together.

In my mind, the outer pink texture would point to the blue inner texture as its "next texture struct pointer". But you have one outer pink texture pointing to nothing, and the other pointing to that black texture.

But...hmm...so you have the blue inner textures calling the outer pink. What is the protocol for which texture calls which?

 

[zankyou]

Its just a linked list. I think the order matters in priority though but I didnt experiment much with the reverse order. I got a black image when I did, though I didnt play with the flags much.

 

[Achilles1515]

Its just a linked list. I think the order matters in priority though but I didnt experiment much with the reverse order. I got a black image when I did, though I didnt play with the flags much.

I figured it out btw. Kind of. I mean, I got it to show up like you wanted. But I'm at the pool getting some Vitamin D so I'll post about it when I get back.