Melee dat format...

[zankyou]

I figured it out btw. Kind of. I mean, I got it to show up like you wanted. But I'm at the pool getting some Vitamin D so I'll post about it when I get back.

Any step towards understanding the text struct a little better sounds amazing

 

[Achilles1515]

So first off, it's hard to understand the big picture of what the purpose of textures are without knowing the entirety of what you are trying to import.
I have no idea what the textures circled in red are being used for.

Nonetheless, I did what I said in my above post, and tweaked a flag, and it worked.

(When I say "call" below, I am just referring to linking the texture structs.)

Again, idk what the black texture is used for so I just ignored it and made nothing call it. I made the white image (data @ offset 0x7c680) call the pink/tan texture. Then made the pink/tan texture call the blue texture. Changed the 0x40 text struct flag of the blue texture to 00010010 and then it worked. Blue texture doesn't call anything.

Idk for certain if the order matters, but it just makes sense to me that the last linked texture is the highest layer. Like Marth's cape, the cape texture links to the cape logo which is on top of the cape texture.

Side Note: For the final product, whatever it may be, can we convert these really simple _6 textures to _14? I feel like they are unnecessarily taking up way too much space for how simple they are, and how small the object they are being cast onto is (just small eyes).

 

[zankyou]

The black texture flag really shouldnt be getting called in the same way. It doesnt have texture coordinates. It uses normal coordinates which Im not sure of its purpose. I definitely plan on putting them in one small texture. The eyes take up 2/3s of the final size. Its just multiple textures come in handy else where and the eyes were the easiest way of testing it.
The reason brawl uses multiple textures here is because for the eyes you can just have the blue one move around. However I dont really feel like making a joint matanim section because no one knows the purpose of the string.

 

[Achilles1515]

The black texture flag really shouldnt be getting called in the same way. It doesnt have texture coordinates. It uses normal coordinates which Im not sure of its purpose. I definitely plan on putting them in one small texture. The eyes take up 2/3s of the final size. Its just multiple textures come in handy else where and the eyes were the easiest way of testing it.
The reason brawl uses multiple textures here is because for the eyes you can just have the blue one move around. However I dont really feel like making a joint matanim section because no one knows the purpose of the string.

You had one of the pink/tan textures calling the black texture. Was this a mistake, then?

 

[DRGN]

So the layer flags in the texture struct seem to correspond to which UV is being used, with 0x4 being the first index.
Also I cant seem to have success with multiple textures on the same material struct. Ive come somewhat close but now quite. Here are a pair of eyes for mario. It should look something like this.

But the closest Ive gotten by changing the halfword at 0x40 in the texture struct.

Why are the Melee Gods so cruel.

Sorry I didn't respond to your texture question in the other thread. I was gonna tackle it once I got home and put my comp back together, but Achilles beat me to it.

Anyway, what structure are the UV lists/maps a part of?

 

[zankyou]

You had one of the pink/tan textures calling the black texture. Was this a mistake, then?

I just linked it in the same order its in for the brawl file. Im not even sure if its showing up since I didnt set it to a texcoord.

Sorry I didn't respond to your texture question in the other thread. I was gonna tackle it once I got home and put my comp back together, but Achilles beat me to it.

Anyway, what structure are the UV lists/maps a part of?

No worries.

 

[zankyou]

So first off, it's hard to understand the big picture of what the purpose of textures are without knowing the entirety of what you are trying to import.
I have no idea what the textures circled in red are being used for.
View attachment 115813

Nonetheless, I did what I said in my above post, and tweaked a flag, and it worked.

View attachment 115814
(When I say "call" below, I am just referring to linking the texture structs.)

Again, idk what the black texture is used for so I just ignored it and made nothing call it. I made the white image (data @ offset 0x7c680) call the pink/tan texture. Then made the pink/tan texture call the blue texture. Changed the 0x40 text struct flag of the blue texture to 00010010 and then it worked. Blue texture doesn't call anything.

Idk for certain if the order matters, but it just makes sense to me that the last linked texture is the highest layer. Like Marth's cape, the cape texture links to the cape logo which is on top of the cape texture.

Side Note: For the final product, whatever it may be, can we convert these really simple _6 textures to _14? I feel like they are unnecessarily taking up way too much space for how simple they are, and how small the object they are being cast onto is (just small eyes).

What did you set the flags for the pink texture to?

 

[Achilles1515]

What did you set the flags for the pink texture to?

Kept them the same as you had them. If I remember correctly is was 00070010.

 

[SinsOfApathy]

Isn't the black texture just the pupil? lol

I would assume the one above it is the twinkle in Mario's eyes.

 

[zankyou]

Alright so I think I understand the first halfword flag a bit. It seems to control what type of transparency is used for the image. 01 seems like the normal transparency, 0x2 seems like it makes the blacks transparent. Id assume 0x7 only saves the highlights from the image. It seems more like this flag is a comparison function used to determine what colors show. Im sure this makes since to someone who knows more about model formats than I. 0x8 throws in a wrench though because it inverts the colors so maybe it has something to do with src/dst modes.

Achilles youre definitely right about the order of the textures. I had the same thought a long time ago and even reversed all of the textures before I posted here about it. But I never tried to 0x1 flag so I never got it to work and assumed it wasnt the issue. Just shows why its good to have multiple people reverse engineering things.

Isn't the black texture just the pupil? lol

I would assume the one above it is the twinkle in Mario's eyes.

You would think that but it shows up that way even without those textures.

 

[zankyou]

So it seems like the joint matanim structure is needed for weighting. Found that out the hard way.

 

[zankyou]

Has anyone looked at the bones in YS. Theres some weird structure at 0x18538 thats supposed to be an object group but its definitely not in a format documented.

 

[Achilles1515]

Has anyone looked at the bones in YS. Theres some weird structure at 0x18538 thats supposed to be an object group but its definitely not in a format documented.

Omfg, Randall data. We can manipulate his path. Need to do some more tests later.

Zankyou, begin importing Smashville over this stage. Immediately, and that is an order. Attach the moving platform object as the new Randall.

 

[zankyou]

Omfg, Randall data. We can manipulate his path. Need to do some more tests later.

Zankyou, begin importing Smashville over this stage. Immediately, and that is an order. Attach the moving platform object as the new Randall.

So it doesnt take me long to get here.

but thats over a target stage. I have to make my current import method compatible with ys especially since this file is on the larger side. After removing everything most of the stuff that was previously there Ive gotten my Yoshis Fortune import to around 2mb. But the game crashes if you hit a certain shy guy. I probably just deleted the mesh for the shy guy but Im too burnt out right now to figure out which bone the shy guys are on so ill do it later.

 

[zankyou]

Punkline Achilles1515
Can either of you look into why this file of YS is crashing when you hit a shy guy. If I had to guess its because I shifted the data around so the shy guy function doesnt know where to look. If thats the case then it means I cant shrink the filesize of YS.

 

[Achilles1515]

Punkline Achilles1515
Can either of you look into why this file of YS is crashing when you hit a shy guy. If I had to guess its because I shifted the data around so the shy guy function doesnt know where to look. If thats the case then it means I cant shrink the filesize of YS.

Do you have a good idea as to where the shy guy animation data is stored?

Also, I just hit some shy guys with the file and it was totally normal.

 

[zankyou]

Do you have a good idea as to where the shy guy animation data is stored?

Also, I just hit some shy guys with the file and it was totally normal.

It seems like it doesnt crash if you it a shy guy with a strong attack.

 

[Achilles1515]

It seems like it doesnt crash if you it a shy guy with a strong attack.

Just tried with Fox jab, laser, first hit uair. No freeze. What is a "weak" attack you are using to make it freeze?

 

[zankyou]

Just tried with Fox jab, laser, first hit uair. No freeze. What is a "weak" attack you are using to make it freeze?

I just tried fox lasers on a relatively clean iso and it still triggered for me. Im getting
invalid read from 0xffffffff, PC = 0x80268c98

Ill test some more and see if Im doing anything dumb to cause this.

 

[Achilles1515]

I just tried fox lasers on a relatively clean iso and it still triggered for me. Im getting
invalid read from 0xffffffff, PC = 0x80268c98

Ill test some more and see if Im doing anything dumb to cause this.

K. I guess I should note that I'm using 20XX. Just using a 20XX root folder with random stage WIPs added and rebuilt with GCR.

 

[Punkline]

Do you have a good idea as to where the shy guy animation data is stored?

Also, I just hit some shy guys with the file and it was totally normal.

I just tried fox lasers on a relatively clean iso and it still triggered for me. Im getting
invalid read from 0xffffffff, PC = 0x80268c98

Ill test some more and see if Im doing anything dumb to cause this.

I wasn't able to reproduce it either, but I just remembered something...

Achilles1515 I rely on this version of Dolphin for memchecks, and have noticed that when I use it it makes these kinds of errors not appear. I wrote a code for a request recently, and I couldn't understand why I wasn't reproducing a similar error message L.O.S.T.30-3 was reporting until I tried using the latest version of Dolphin. The error I'd made in my code was likely causing arbitrary data to be read, but for some reason I wasn't able to see it happen on this old copy of Dolphin...

Running it in Dolphin 5.0 makes the errors appear for me. No idea why.

---

Anyway, earlier this morning I used the collision records in player hitbox data to see what the shyguy was doing. I’m not 100% sure, but loading 0x4 from the pointer left in the hitbox collision leads to what looks like an entity structure.

Collision record -> 0x4 -> 0xC8 points to an array of 3 pointers that appear to be located in this area of GrSt.dat, and all point to other areas in the file. I have only this clue telling me that it's related to shyguys, and may have nothing to do with the problem:

0xB8 seems to use the pointers for something. I haven’t looked into it much, but it was the only thing I could find (from a quick look) that pointed to loaded GrSt.dat data. I'll give it another look later knowing that I can reproduce the error message in Dolphin 5.0

Here’s a gif of the supposed entity, internal data, and possibly related file data. The clip shows the entity loading with the match, loading the data as the shyguy spawns, and what happens as a shyguy collides with a hitbox:

 

[zankyou]

I wasn't able to reproduce it either, but I just remembered something...

Achilles1515 I rely on this version of Dolphin for memchecks, and have noticed that when I use it it makes these kinds of errors not appear. I wrote a code for a request recently, and I couldn't understand why I wasn't reproducing a similar error message L.O.S.T.30-3 was reporting until I tried using the latest version of Dolphin. The error I'd made in my code was likely causing arbitrary data to be read, but for some reason I wasn't able to see it happen on this old copy of Dolphin...

Running it in Dolphin 5.0 makes the errors appear for me. No idea why.

---

Anyway, earlier this morning I used the collision records in player hitbox data to see what the shyguy was doing. I’m not 100% sure, but loading 0x4 from the pointer left in the hitbox collision leads to what looks like an entity structure.

Collision record -> 0x4 -> 0xC8 points to an array of 3 pointers that appear to be located in this area of GrSt.dat, and all point to other areas in the file. I have only this clue telling me that it's related to shyguys, and may have nothing to do with the problem:

0xB8 seems to use the pointers for something. I haven’t looked into it much, but it was the only thing I could find (from a quick look) that pointed to loaded GrSt.dat data. I'll give it another look later knowing that I can reproduce the error message in Dolphin 5.0

Here’s a gif of the supposed entity, internal data, and possibly related file data. The clip shows the entity loading with the match, loading the data as the shyguy spawns, and what happens as a shyguy collides with a hitbox:

I dont get the issue on that version of dolphin either. Im assuming though the 5.0 is more true to console.

 

[Achilles1515]

I dont get the issue on that version of dolphin either. Im assuming though the 5.0 is more true to console.

Tested on console. Froze.

 

[zankyou]

Rip. My new import method puts the structures inside of the file rather than just at the end, so its not compatible with this stage. Ive mainly just been working on target stages so now Im curious what stages break when you remove data.

 

[SinsOfApathy]

Achilles1515 I rely on this version of Dolphin for memchecks, and have noticed that when I use it it makes these kinds of errors not appear. I wrote a code for a request recently, and I couldn't understand why I wasn't reproducing a similar error message L.O.S.T.30-3 was reporting until I tried using the latest version of Dolphin. The error I'd made in my code was likely causing arbitrary data to be read, but for some reason I wasn't able to see it happen on this old copy of Dolphin...

Running it in Dolphin 5.0 makes the errors appear for me. No idea why.

This is because Dolphin now accurately emulates BAT registers, which Smash Bros. Melee apparently uses to some extent.

Basically, games are now reproducing crashes that would occur on console due to reading invalid memory when previously that would not occur.

 

[Punkline]

This is because Dolphin now accurately emulates BAT registers, which Smash Bros. Melee apparently uses to some extent.

Basically, games are now reproducing crashes that would occur on console due to reading invalid memory when previously that would not occur.

That’s very interesting. I don’t know how to make use of these new registers, but it’s cool that Dolphin is accurately emulating this now when it wasn’t before. I'll have to keep this in mind while testing codes in the future.

I’ve noticed the error message seems to fire as soon as it has a straight branchless path of instructions to read. If you haven’t put down any breakpoints, it will usually happen right after a bl. Stepping through the code line by line is the only way I’ve been able to find the offending instruction.

In this case, this appears to be ground zero:

iirc, this calling function appears to do something mysterious alongside new behaviors in the various items/articles/projectiles on stage.

The call it makes after doing something with some bone data uses the BA in r23, which points to something loaded from GrSt.dat.

When the shyguy is moving, the call appears to do something as it resets its flying animation; but after hitbox collision, r23 is displaced by 0x20 in order to point to some neighboring data that I presume is used somehow in the behavior of a shyguy after being hit.

Here are the 2 neighboring tables of data referenced by r23 in these particular cases:

r6 passes a value from r23 to the offending function call; and that would be the FFFFFFFF value I’ve highlighted there. What’s strange is that even though this value appears in both files, in the unedited Yoshi Story level this loaded value becomes “0” but in the edited one it stays as “FFFFFFFF”

Using a memcheck on the address that the value gets loaded into (with savestates) doesn’t trigger any breakpoints for some reason. I’m at a loss as to where this “0” comes from when using the unedited stage. r6 gets saved to r30, and from there, instructions like this freeze the game:

---

I don't really know what to do about this as I can't figure out what's editing the values. A nice lead though is the fact that the r6 value loaded from the adjacent table for normal shyguy flying behavior is also a value that turns into 0.

Even more curious is the fact that it's a file offset instead of "FFFFFFFF" and hasn't been modified in the shrunken GrSt.dat Z zankyou

 

[zankyou]

Punkline This is really intriguing and not a case that I would have found without your help. I think that 0xffffffff might have something to do with loading external data. Similar to if you look at pokemon stadium, the pointers to the bones in what I call root structures are set to 0xffffffff for what I assume is where pointers to the stage changes go. So my import method might not work for that stage either. With the way I shrink files, that reference was definitely missed because its not a pointer in the pointer table. So the program that does it just assumed it was data and left it unchanged.
I changed the value to 0x0002af38 and it seems to have fixed it but it does need to be tested more. If I had to guess what the function is for, since it only broke on weak attacks the idea I have is that its related to the shy guy turn around animation. Strong attacks knock them away but on weak attacks they turn around.
Edit: Looks like its part of the itemdata rootnode. I often forget shyguys carry items.

 

[Punkline]

Very interesting. I'm still learning a lot about the basics here, but I've run into fun problems with the relocation table while modifying "goto" and "subroutine" subaction events. Looking at the stage data has me wanting to learn more about this stuff.

Punkline I changed the value to 0x0002af38 and it seems to have fixed it but it does need to be tested more. If I had to guess what the function is for, since it only broke on weak attacks the idea I have is that its related to the shy guy turn around animation. Strong attacks knock them away but on weak attacks they turn around.

I'll try the same adjustment later this evening and see how it goes.

BTW, I recall that the arguments for the DATfile_boneStructure_copy function that gets called right before this one pointed to a table that seemed to resemble the kinds of bones I mentioned in these notes.

---

I just checked and the table passed as argument r3 for the bone copy has a pointer appear in the same spot you would expect an "interpolation timer" pointer to appear (as I called it for player bones) after the error message appears.

That also means that the pointer 0x8 from the location interpolation timer should be the default bone data in GrSt.dat (similar to the kind of data I mentioned in this transN bone adjustment for Marth that didn't work very well)

I think this is where it is in the dat:

 

[Achilles1515]

Hmmm. A reference node. I need to look into these.

GrdStoryHeiho_TopN_shapeanim_joint

That’s very interesting. I don’t know how to make use of these new registers, but it’s cool that Dolphin is accurately emulating this now when it wasn’t before. I'll have to keep this in mind while testing codes in the future.

I’ve noticed the error message seems to fire as soon as it has a straight branchless path of instructions to read. If you haven’t put down any breakpoints, it will usually happen right after a bl. Stepping through the code line by line is the only way I’ve been able to find the offending instruction.

In this case, this appears to be ground zero:

iirc, this calling function appears to do something mysterious alongside new behaviors in the various items/articles/projectiles on stage.

The call it makes after doing something with some bone data uses the BA in r23, which points to something loaded from GrSt.dat.

When the shyguy is moving, the call appears to do something as it resets its flying animation; but after hitbox collision, r23 is displaced by 0x20 in order to point to some neighboring data that I presume is used somehow in the behavior of a shyguy after being hit.

Here are the 2 neighboring tables of data referenced by r23 in these particular cases:

r6 passes a value from r23 to the offending function call; and that would be the FFFFFFFF value I’ve highlighted there. What’s strange is that even though this value appears in both files, in the unedited Yoshi Story level this loaded value becomes “0” but in the edited one it stays as “FFFFFFFF”

Using a memcheck on the address that the value gets loaded into (with savestates) doesn’t trigger any breakpoints for some reason. I’m at a loss as to where this “0” comes from when using the unedited stage. r6 gets saved to r30, and from there, instructions like this freeze the game:

---

I don't really know what to do about this as I can't figure out what's editing the values. A nice lead though is the fact that the r6 value loaded from the adjacent table for normal shyguy flying behavior is also a value that turns into 0.

Even more curious is the fact that it's a file offset instead of "FFFFFFFF" and hasn't been modified in the shrunken GrSt.dat Z zankyou

00000000 gets written to 0x32d78 in the vanillla DAT at code line 803804C8.

 

[Achilles1515]

Also, Z zankyou
I tried the Sm4sh Captain Falcon (super high poly) on console and it lagged as expected, but looked like this.

Not sure if that is specifically related to the high poly, or something else in regard to the model import.

 

[zankyou]

Also, Z zankyou
I tried the Sm4sh Captain Falcon (super high poly) on console and it lagged as expected, but looked like this.

View attachment 117427

Not sure if that is specifically related to the high poly, or something else in regard to the model import.

Definitely something wrong with the model. But its possible that the image dimensions are not compatible. I remember reading about how dolphin could take dimensions of any kind while the wii cannot so it could be that this import was only meant for dolphin. Did you try hooded marth?
I also got a similar issue when I first starting modding related to upscaling dreamlands background. I had to change the texture wrap settings to fix that. Maybe on of the textures is too high res.

Btw my import method works for YS so do you still want a smash ville over it. Not sure if Ill add an exception for this stage in any future programs.

 

[Achilles1515]

Definitely something wrong with the model. But its possible that the image dimensions are not compatible. I remember reading about how dolphin could take dimensions of any kind while the wii cannot so it could be that this import was only meant for dolphin. Did you try hooded marth?
I also got a similar issue when I first starting modding related to upscaling dreamlands background. I had to change the texture wrap settings to fix that. Maybe on of the textures is too high res.

Btw my import method works for YS so do you still want a smash ville over it. Not sure if Ill add an exception for this stage in any future programs.

I did not try hooded Marth. I'll try to in the near future.

Sure. I was a little disappointed with the results from my Randall testing. I can successfully make him travel in a straight horizontal path back and forth, but idk how to reliably change the speed or x-location for the stopping points. But more testing still needs to be done.

Also, can you post the nuked Yoshi's?

 

[zankyou]

I did not try hooded Marth. I'll try to in the near future.

Sure. I was a little disappointed with the results from my Randall testing. I can successfully make him travel in a straight horizontal path back and forth, but idk how to reliably change the speed or x-location for the stopping points. But more testing still needs to be done.

Also, can you post the nuked Yoshi's?

I guess I should probably call it scrubbing the file.
Unfortunately because YS is an atypical stage (ref nodes, weird obj groups) its fairly hard to get the programs Im working on to not crash when you select that file. I had to do Yoshi's Fortune slowly and Ill probably do it for this import since the filesize is so large.

But it might be a while before I do it to the default stage. Im not sure why but a lot of things went wrong when I tried it before

with crashes that also happen as soon as shy guys or possibly randall loads. For now Ive just set randall to not visible. Ill have to test it more since punkline found that pointer thats not really a pointer.

 

[Achilles1515]

Z zankyou Punkline
The synopsis of my rough notes about this.

Take reference node data pointer from end of file. Load word at that pointed to address. If not FFFFFFFF, then it is a "pointer". Take the pointer and go to that location in the file data block. Load word at this address. If not FFFFFFFF, it is a pointer. Repeat....

EDIT: accidentally said zero instead of FFFFFFFF.

 

[Achilles1515]

Definitely something wrong with the model. But its possible that the image dimensions are not compatible. I remember reading about how dolphin could take dimensions of any kind while the wii cannot so it could be that this import was only meant for dolphin. Did you try hooded marth?
I also got a similar issue when I first starting modding related to upscaling dreamlands background. I had to change the texture wrap settings to fix that. Maybe on of the textures is too high res.

Btw my import method works for YS so do you still want a smash ville over it. Not sure if Ill add an exception for this stage in any future programs.

"The length and width of these textures should not exceed 1024x1024, and each value must be a power of 2."
- GameCube SDK Version 1.0, Character Pipeline for Artists, page 14

 

[zankyou]

"The length and width of these textures should not exceed 1024x1024, and each value must be a power of 2."
- GameCube SDK Version 1.0, Character Pipeline for Artists, page 14

Btw what fps would you say it ran on console. Im honestly surprised it loaded. I figured it was going over the input limit.

 

[Achilles1515]

Btw what fps would you say it ran on console. Im honestly surprised it loaded. I figured it was going over the input limit.

Uhh idk. Maybe like 30 fps or something.

DRGN worth putting a note or a "warning" symbol or something in DTW next to textures that don't follow this convention?

 

[Achilles1515]

Z zankyou
Say you wanted to find the total space taken up in a DAT file from one object's mesh structure vertices points data. Am I correct in saying that the only way to do so, is to walk through the entire displaylist and find the highest vertice index entry, then multiply it by the stride? Sounds like a pain in the *** since you'd need to know all the components of the displaylist and their formatting and type too. Does your scrubber do this?

 

[zankyou]

For the program Im working on, I completely parse the file as soon as its open. Part of why it crashes when choose YS.

 

[Achilles1515]

For the program Im working on, I completely parse the file as soon as its open. Part of why it crashes when choose YS.

Sure, but how does your program calculate how much space each mesh structure attribute takes up in the DAT for a single object?