Melee dat format...

[zankyou]

Hey Achilles, I hate to keep asking you to help me out, but could you look into how multiple images are loaded onto the same material struct So multiple texture structs per material. Right now I have to duplicate the object and give it the appropriate material but for the kalo league stage that adds around 6000 extra polygons, even though I didnt notice a performance difference without the duplicated objects itd be nice to not have to push the game any harder than I already am.

 

[Achilles1515]

Hey Achilles, I hate to keep asking you to help me out, but could you look into how multiple images are loaded onto the same material struct So multiple texture structs per material. Right now I have to duplicate the object and give it the appropriate material but for the kalo league stage that adds around 6000 extra polygons, even though I didnt notice a performance difference without the duplicated objects itd be nice to not have to push the game any harder than I already am.

I can try, but I’m a little confused as to what you are actually wanting to know. So you have the material structure, which points to a texture structure. And then this texture structure can point to another texture structure. But what about this are you trying to figure out? If you are wanting to know how it actually knows where to place the texture images, then…idk…that might be too complex of a thing to figure out by reversing assembly. But lemme know exactyl what you desire and I’ll try and poke around.

My services may or may not cost you an updated Hyrule 64.

 

[zankyou]

I can try, but I’m a little confused as to what you are actually wanting to know. So you have the material structure, which points to a texture structure. And then this texture structure can point to another texture structure. But what about this are you trying to figure out? If you are wanting to know how it actually knows where to place the texture images, then…idk…that might be too complex of a thing to figure out by reversing assembly. But lemme know exactyl what you desire and I’ll try and poke around.

My services may or may not cost you an updated Hyrule 64.

Funny you mention that. Because I just finished the import over peach target stage. I'll go into a bit more detail after my classes are over as to what I'm asking.
Essentially I know how the file is supposed to be structured for multiple textures per object, but the game breaks when I try to implement it. If you could look at where it loads the second texture struct for a file the supports it and the file Im trying to make and tell me what Im doing differently that would help. I know its kinda vague...

 

[zankyou]

Trying to find how bones are stored so I can rip animations and make it easier to reverse engineer them. I think the strings tcll were talking about contain data with the bit which is why its not really intelligible from looking at the aj file.

 

[zankyou]

So Im pretty sure the aj file uses the bone structures in the pl**.dat and that the bone structure in the pl**nr.dat is used for the things in there such as objects and animations it needs like blinking. So if you wanted edit the bone size or general translation you would need to do it in the pl**.dat file. Furthermore it seems that the pl**.dat has its own set of objects but all the images led to a 20byte image. Going out on a limb Id assume it was equivalent to brawls ref or metal texture.
Now that we can find where the bones are stored we should be able to look at the transformations that bones undergo, but analyzing that is beyond me.

 

[zankyou]

View attachment 74155

I don't actually know what this function controls btw.

r3 contains the location of where the bone will be stored in memory. This function also runs for every bone at the beginning of the match and loads all the bones. r4 is the location of the copy from the dat file. Should be pretty simple to make moving platforms and what not but more importantly is a good step in cracking animations.

Editing the bone doesnt always work at least on parent bones. Might have something to do with the bone flags.

 

[zankyou]

Tcll return to us.

Notes of animation files (pl**aj.dat)

Every animation in the file is formatted as its own dat file in a way and the next animation dat header is on the next line directly after the end of the previous animation.
There seems to be only 1 root node and its entry in the string table is the name of the animation.
The root node - General animation info
0x00 - unknown (possible int for number of animations (always 0x00000001)
0x04 - unknown (always 0x00000000)
0x08- frame count (float: the character will go on to the next animation as soon as this is over. If it is greater the character will just freeze until then)
0x0C- pointer to bone index table (no idea what to call this)
0x10- animation info

bone index table (0x1*number of bones- length)
The number of bones can be gotten by just counting up the bones in the pl**.dat or looking at the joint matanim section and counting those up.
0x1- number of sections (possibly animations). Bone index is given by the position from the start of the table.
Terminated by 0xFF

animation info (0x0C*(total number of sections in the bone index table)-length)
0x00- length of animation data (0x02 bytes)
0x04- unknown (my speculation is that it works like the animation format word in the chr0 file for the brawl format)
0x0C- pointer to animation data

This is as far as my ability to pattern recognize goes. Anything more would require I actually know what Im talking about and have any idea how animations work to confirm.
As far as my assumption about the animation data format. I believe the format is encoded in the bits like in the chr0 files. However the bits would be reversed, and melee bone transformations are ordered by rotation, scale, translation whereas brawls is ordered by scale, rotation, translation.

Considering writing a code to basically record the transformations of a particular animation every frame and just export the ram. But I wouldn't even know what to do with that.

 

[Achilles1515]

grGroundParam

0x04    Half    character shadow intensity
                       0x0000 = opaque (fully black)
                       0x00FF = transparent (no shadows)

 

[Tiberious]

Let me get this straight... there's ability to import models into Melee, but yet we still can't export fighter models? What gives?

 

[zankyou]

Let me get this straight... there's ability to import models into Melee, but yet we still can't export fighter models? What gives?

Melee, Hacks, and You -- *UPDATED 5/21/2015*
I understand more now though. But the main hold up is bone mapping. I think I have a lead on my random weight index issue. But if the bones arent in the right order then the animation file will break the game. If someone wants to reskin models to melee bones and import that into brawl I could give it a try but until then youre going to have to wait until figure out a good way of going about this.

 

[Tiberious]

No, no, I mean just extracting the models from Melee with their joints and vertex weighting (like we have tools and scripts for doing the same with Brawl and WiiU) to a format we can use on the PC.

So far, the closest thing I know of is something that can export to .obj, but that doesn't keep the joints or weighting on the vertices. Any attempt to rig to it would just be a guess at what the actual weighting was.

 

[zankyou]

No, no, I mean just extracting the models from Melee with their joints and vertex weighting (like we have tools and scripts for doing the same with Brawl and WiiU) to a format we can use on the PC.

OhI see. We're still limited to only exporting objs.

 

[Tiberious]

Yeah, see, that's the part that sucks. It's obvious the models have bones and weighting, but nobody seems interested in finding a way to convert that to .dae or something similar.

I mean, I'm only really interested in 3 fighters (Wireframes and Mewtwo), but I'd like to have it as accurate as possible, you know?

Worst case scenario, I could probably puzzle out the Male's joint locations, and maybe rig it up, but that seems like a lot of hassle, especially since it seems he actually has individual finger joints.

 

[zankyou]

Done with classes. Does anyone have experience with animation data formats?

 

[Tcll]

Done with classes. Does anyone have experience with animation data formats?

you could look into the IQM format
it's triangles only, but about as simple as OBJ, although in binary.
(I'll be working on an exporter for this)

I'll also be working on a PMD exporter as it's almost as simple
I might just get MD5 along with that as it's easier to export to.

EDIT:
oops, autism again, I thought you meant models supporting animation

 

[Achilles1515]

Tcll Can you update the material and image structures on the HAL Dat Format page?

http://smashboards.com/threads/return-to-dreamland-wip-model-import.417944/page-2#post-20211123

See that post and the one two below it.

 

[Tcll]

I'm actually doing alot in my current position, but you can feel free to edit
that's why it's a wiki

EDIT:
if I had a faster compy, I'd have no problem
#slowlaptop
#slowinternet

 

[Achilles1515]

I'm actually doing alot in my current position, but you can feel free to edit
that's why it's a wiki

EDIT:
if I had a faster compy, I'd have no problem
#slowlaptop
#slowinternet

Oh I had to confirm my email. Got it.

 

[TGE]

Dunno if it's of any use but I wrote my own DAT file parser for fun. Only non-stage models supported atm. Can probably work out an exporter with bones/rigging if there's a need for it. The joints seem line up with brawl's for the most part.
The transformation bit was kinda tricky as it didn't do what I expected it to at first but I wouldn't be opposed to attempt writing a model converter (probably something like BrawlBox, but not actually BrawlBox because reasons).

Oh yeah, made a quick mockup if what a 'BrawlBox-like' viewer for these files would look like.
http://puu.sh/oLFfV.png

Edit: Also yes I realize I accidentally moved both of his cape meshes to the right lol.

 

[Tcll]

Ugh, another 3ds script... >_<

Nice work though, at least it fills that void...

Have you been taking any pointers from me??

 

[TGE]

It's not a maxscript (though I'm pretty fluent in that as well)
The program is written in C#, I just wrote a quick obj exporter because it's easy to use for debug output when reversing 3d data.
I got most of the info from revel8n, the GC sdk helps too. I have prior experience with many model formats (including ones for GC specifically) so I guess I want to offer my help where I can.

Most of the unknowns lie in the texture descriptor struct at the moment. I've also been trying to find the instruction that loads the models but I haven't found it yet (if anyone knows the offset(s), I'd appreciate it)

 

[Achilles1515]

It's not a maxscript (though I'm pretty fluent in that as well)
The program is written in C#, I just wrote a quick obj exporter because it's easy to use for debug output when reversing 3d data.
I got most of the info from revel8n, the GC sdk helps too. I have prior experience with many model formats (including ones for GC specifically) so I guess I want to offer my help where I can.

Most of the unknowns lie in the texture descriptor struct at the moment. I've also been trying to find the instruction that loads the models but I haven't found it yet (if anyone knows the offset(s), I'd appreciate it)

Looks amazing, please keep going.

What do you mean by an instruction to load the models? Can you give an example where a model is not loaded?

Also, what is holding you up from doing stages at the moment? Is it any sort of lack of stage DAT file info?

 

[TGE]

Looks amazing, please keep going.

What do you mean by an instruction to load the models? Can you give an example where a model is not loaded?

Also, what is holding you up from doing stages at the moment? Is it any sort of lack of stage DAT file info?

I meant the instruction(s) in the disassembly ;p (or maybe I just suck using that dolphin debugfast build, had better luck with ps2dis for ps2 games)
I haven't really looked into the stage files too much, from what I've seen people know bits and pieces about it but there doesn't seem to be a real writeup as far as I know. It'd help me a lot of I had some good collection of findings about how the files are put together.

 

[Tcll]

Could alwayslook at my sscrip . Fo . UMC

I'ma bbi . Unabl . To ge . It on thi . Stupidpphone
But iIhhavea few ffriendswho mmightbe aable to hel.

See if yyoucan ccontac . Matt Sitton,
He's helpin . Me develo . UMC andmmigh . Be abl . To ge . Yo . A copy

 

[Tiberious]

Dunno if it's of any use but I wrote my own DAT file parser for fun. Only non-stage models supported atm. Can probably work out an exporter with bones/rigging if there's a need for it. The joints seem line up with brawl's for the most part.
The transformation bit was kinda tricky as it didn't do what I expected it to at first but I wouldn't be opposed to attempt writing a model converter (probably something like BrawlBox, but not actually BrawlBox because reasons).

Yes... YES PLEASE!! Nearing 15 years since this game's launch, and yet this is the last one in the series that has yet to have its fighter models ripped with weighting. I'm sure both TMR and the Brawl modding scene would love to get actual Melee models with their bones and rigging.

 

[Tcll]

No . Quit . Th . Las, smash64 is bbeingllookediinto

 

[Achilles1515]

Z zankyou
Stage lighting. Still a confusing topic, but I wanted to post my observations and initial research.

I think this was mentioned in the model hacks thread recently, but I do not believe the majority of stage models in Melee use any sort of lighting effects at all. I haven't looked through all the stages, but one exception is Fountain of Dreams.

About a year ago, I posted a code titled "Vibrant Melee". This code modifies one of the map_plit light structures via RAM hacking instead of hex editing.

If you change the lighting value in this code to red instead of white, this is the outcome:

Notice how stage elements are normal.

But, let's go to FoD...

So the lighting on many structures here are directly tied to the lighting color struct value.

My first guess is a Material Structure flag.

Let's look at the material structure flag for the texture on the side of the left platform.

Material structure @ 0xe04e0 in GrIz.dat
platform side texture mat struct flag value = 0x00000014

The green grass clearly is not influenced by the lighting color.
Green grass material structure @ 0xe0dac
mat struct flag = 0x00000012

Okay, so they each include (00000010), but the side platform that uses lighting effects also include the (00000004) bit. Let's remove it and set the material struct flag word to 0x00000010.

Cool! So turning that bit low seems to remove the use of lighting effects on materials.

Now, I went into your Metal Cavern HD import and tried to enable the (00000004) bit on the material struct of the green floor. While it did enable some lighting effects on the floor, it was doing that weird thing that we saw in your initial imports where the lighting effect would "flicker" upon strong character hits, etc.

Idk...there's more testing to be done. Possibly to look at the bone struct flags as well.

 

[Achilles1515]

Z zankyou
Big discovery! I think I got it working...

For all the below, from your GrNBa file:
Material struct flag @ 0xe1d6c
Bone struct flag @ 0x111364

Current Metal Cavern Floor
Material struct flag = 0x10000022
Bone struct flag = 0x10140000

--------------------------------------------------------------------------------

Specular Metal Cavern Floor
Material struct flag = 0x0000001c .......... (0x00000014 might be the same?)
Bone struct flag = 0x00040088 ............ (I copied this value from the bone struct flag of the side platform texture mentioned above on FoD)

No flickering.

 

[zankyou]

This is looking promising. Definitely gonna take into consideration when I finish metal cavern. Hopefully it doesnt take long to fix those with the current imports Im almost done with.

 

[zankyou]

So the first byte of the material flag is alpha function related. Because playing with this value gives similar results as playing with the alpha function in brawl box.

 

[Achilles1515]

So the first byte of the material flag is alpha function related. Because playing with this value gives similar results as playing with the alpha function in brawl box.

That and (04000000) causes character shadows to be cast on the material.

 

[zankyou]

Hey achilles. That offset for the object group with multiple textures in the plmsbk.dat is at 0x1638c. I know theres a pointer to the next texture in the text struct but I feel like theres more since the game breaks when I try it.

 

[Achilles1515]

Hey achilles. That offset for the object group with multiple textures in the plmsbk.dat is at 0x1638c. I know theres a pointer to the next texture in the text struct but I feel like theres more since the game breaks when I try it.

So these multiple textures are forced to use the same mesh? Can you give me an example file of where you wish to use this?

 

[zankyou]

grst
the object group is at 0xd1478. its kind of an old file from the first import and when I was testing this more extensively.

 

[Achilles1515]

How you like them crystals!
(still testing some things)

EDIT:
Nvm. Even f**king better.

 

[zankyou]

How you like them crystals!
(still testing some things)

View attachment 109837

EDIT:
Nvm. Even f**king better.

View attachment 109838

I have to ask while youre playing around with those flags. Do you think its possible to have both a color struct and a color node.

 

[Achilles1515]

I have to ask while youre playing around with those flags. Do you think its possible to have both a color struct and a color node.

Idk. I'm still trying to work out color structs and transparency flags. I also don't think you can enable specular lighting and color nodes at the same time. The metal cavern import is confusing me, because anytime I put a (40000000) in the material struct flag, the object completely disappears. Do you know why this occurs? Because I think you have to have that enabled to use the transparency value in the color struct.

 

[zankyou]

Idk. I'm still trying to work out color structs and transparency flags. I also don't think you can enable specular lighting and color nodes at the same time. The metal cavern import is confusing me, because anytime I put a (40000000) in the material struct flag, the object completely disappears. Do you know why this occurs? Because I think you have to have that enabled to use the transparency value in the color struct.

Whats the bone flag? Ive switched to using 10140088 as soon as I read your post this morning because I think that happens otherwise.
And yeah I didnt really think you would be able to have both. Seems really redundant since the color struct does the same and more. The color struct is like a color node for each polygon instead of the entire object. You might have more success if you played with the color struct in the mdl0 file before I import it.

 

[Achilles1515]

Whats the bone flag? Ive switched to using 10140088 as soon as I read your post this morning because I think that happens otherwise.
And yeah I didnt really think you would be able to have both. Seems really redundant since the color struct does the same and more. The color struct is like a color node for each polygon instead of the entire object. You might have more success if you played with the color struct in the mdl0 file before I import it.

I tried a whole bunch of bone flags and never could get anything to make it not disappear. Try it with that bone flag you posted.

Do you have color struct and color node switched in your above statement? Is there ANY way for me to modify color nodes with a file you have uploaded? Or would that be ludicrous and it has to be created before importing it into the file. Idk even know where color nodes are located in the file. Do the mesh attributes point to them?

Also, to get the "true" specular lighting or whatever you want to call it, like in my second crystal picture:

Bone flag contains (00010080)
Material Struct contains (000000C)

Including the (00010000) in the bone flag actually adds the "shine" from the camera as you move the camera around. It looks amazing.

I'm not sure what the (00000008) does in the bone flag, and we can probably remove it for the time being. What does (10000000) do in the bone flag?

 

[zankyou]

I tried a whole bunch of bone flags and never could get anything to make it not disappear. Try it with that bone flag you posted.

Do you have color struct and color node switched in your above statement? Is there ANY way for me to modify color nodes with a file you have uploaded? Or would that be ludicrous and it has to be created before importing it into the file. Idk even know where color nodes are located in the file. Do the mesh attributes point to them?

Does it also disappear when the last byte is set to something like 22 or 1c. I havent really experimented with 00.
Also yeah the mesh attribute array points to it. Id guess that about half of them are simply one color of gray though so those could be replaced with the color node. Thought I know playing with the alpha value of the color there makes the object transparency change accordingly which sounds like what you want to try.