Melee Animation / Model Workshop

[adjective]

D3WD suggested breaking work on the animation/model data out of the main melee modding thread.
This is for work on trying to figure out the internal animation and model formats in the melee datfiles, (Pl**Nr/Re/Bu/Gr/Yl.dat and Pl**AJ.dat files)

for convenience:

• revl8n's post on joint and material information
• my post on jobj & figatree exporting in the main thread
• TCLL's Universal Model Converter

 

[adjective]

I dumped the output of my inspector on the Pl**AJ files for each subaction offset, which you can find here.

The dumps of anonymous data are the result of following the probablyAnOffset value in the figatree_headers with relation to the internal HAL file in the Pl**AJ file, continued until it runs into a previously mapped area (i.e. until it hits the beginning of the figatree header or the relocation table of the data file, etc).

Some interesting features of this pointed data are:

1. it is far too small to be animation data for any of the characters, even if it's compressed somehow
2. It is replicated between animations, but at different offsets
3. it does not look like floating point data or offsets. If anything the alternating zeroes make me think it is a series of chars / uint8_ts, but it doesn't look like a string in any character encoding I'm familiar with.

For example, all of Pichu's Passive animations (tech / techroll animations) share the same values in their anonymous block

Regardless, I'm pretty sure it's some kind of animation metadata, NOT actual animation data.

EDIT: SOMETHING I DIDN'T REALIZE WHEN I MADE THIS POST
It's probably bone-specific flags.

Pichu has 45 bones and his char[] is 44 elements long.
Pikachu has 52 bones and his char[] is 52 elements long.
Mewtwo has 67 bones and his char[] is 68 elements long.
Ganon has 83 bones and his char[] is 84 elements long

If you ignore flags on the grounded bottom-ecb bone (bone 0) and possibly the other ecb bones (vertical and horizontal ecb bones), these numbers work out very nicely.

 

[Tcll]

could I get some info on the bone flags plz

the only flag I actually know from looking inside the DOL is:
JOBJ_USE_QUATERNION
but I havn't yet found a bone that uses a quaternion to test this flag with.

EDIT:
may be a tad outdated, but it's still helpful
http://wiki.tockdom.com/wiki/HAL_DAT_(File_Format)
(Please edit)

it's outdated because of the pathfinder.
my new pathfinder actually helps you locate undefined pointers.
(or rather it locates them and tells you where they are in a particular struct)

EDIT2:

here's a definition for the material struct with a pointer found from KAR:

    @structpointers( 24, {
        0:'_pass', # string
        8:'_texture',
        12:'_pass', # (A,D,S color data)
        20:'_pass', }) # unknown (not documented) (detected by pathfinder in KAR)
    def _material(offset,_apply=True):

@structpointers( struct_size, {
ptr_offset: substruct_name
})

 

[zankyou]

lol theres like 5 animation threads going on. Ill just repost about the structure of the pl**aj files.

Tcll return to us.

Notes of animation files (pl**aj.dat)

Every animation in the file is formatted as its own dat file in a way and the next animation dat header is on the next line directly after the end of the previous animation.
There seems to be only 1 root node and its entry in the string table is the name of the animation.
The root node - General animation info
0x00 - unknown (possible int for number of animations (always 0x00000001)
0x04 - unknown (always 0x00000000)
0x08- frame count (float: the character will go on to the next animation as soon as this is over. If it is greater the character will just freeze until then)
0x0C- pointer to bone index table (no idea what to call this)
0x10- animation info

bone index table (0x1*number of bones- length)
The number of bones can be gotten by just counting up the bones in the pl**.dat or looking at the joint matanim section and counting those up.
0x1- number of sections (possibly animations). Bone index is given by the position from the start of the table.
Terminated by 0xFF

animation info (0x0C*(total number of sections in the bone index table)-length)
0x00- length of animation data (0x02 bytes)
0x04- unknown (my speculation is that it works like the animation format word in the chr0 file for the brawl format)
0x0C- pointer to animation data

This is as far as my ability to pattern recognize goes. Anything more would require I actually know what Im talking about and have any idea how animations work to confirm.
As far as my assumption about the animation data format. I believe the format is encoded in the bits like in the chr0 files. However the bits would be reversed, and melee bone transformations are ordered by rotation, scale, translation whereas brawls is ordered by scale, rotation, translation.

Considering writing a code to basically record the transformations of a particular animation every frame and just export the ram. But I wouldn't even know what to do with that.

 

[adjective]

haha dang looks like all the work I've been doing is redundant, but it feels good to have my hunches confirmed by other people.

 

[zankyou]

Feel free to continue. I havent taken that further than it was in november. I have some ideas as to what direction to take if you want help.

 

[Tcll]

feel free to jump on my dev-chat too
adjective : I'll PM you my invite

 

[adjective]

Update on animations:

I looked at the stuff Z zankyou posted and I made upated animation dumps for each character available here

Each folder has a series of text files, which are dumps of the animation with the corresponding offset in the Pl**AJ.dat file.
The text dumps are a series of tables representing the memory pointed to by the animDataOffset of the figatree header, up until the offset of the next animData,

For example, these are both 2 word long structures, holding a single word of relevant information

I think this is the animation data zankyou was talking about? I had a bit of a hard time understanding what he was trying to say in his quote.

EDIT:
I changed the dump format to just display the char[] s. Currently looking for repeating patterns to find modulos for animations. ifl moves animating scale will be a good place to start.

It's possible this another situation like the subaction commands situation where you have a union type representing different animation-actions .

 

[revel8n]

Some interesting features of this pointed data are:

1. it is far too small to be animation data for any of the characters, even if it's compressed somehow
2. It is replicated between animations, but at different offsets
3. it does not look like floating point data or offsets. If anything the alternating zeroes make me think it is a series of chars / uint8_ts, but it doesn't look like a string in any character encoding I'm familiar with.

Regardless, I'm pretty sure it's some kind of animation metadata, NOT actual animation data.

EDIT: SOMETHING I DIDN'T REALIZE WHEN I MADE THIS POST
It's probably bone-specific flags.

Pichu has 45 bones and his char[] is 44 elements long.
Pikachu has 52 bones and his char[] is 52 elements long.
Mewtwo has 67 bones and his char[] is 68 elements long.
Ganon has 83 bones and his char[] is 84 elements long

If you ignore flags on the grounded bottom-ecb bone (bone 0) and possibly the other ecb bones (vertical and horizontal ecb bones), these numbers work out very nicely.

Your observations on joint counts are correct (at least in my findings so far).

This format is actually somewhat "simple" just encoded in a non-obvious way. i can readily say i probably would not have been able to figure out the latest information i am gathering from just static observation of the file data. The process to decode and interpret it all is a little more involved though.

struct FIGATREE_DATA
{
    // 0x00
    uint32 unknown0x00;
    uint32 unknown0x04;
    float  unknown0x08;
    uint32 trackCountsOffset <format = hex>;
    // 0x10
    uint32 trackInfoOffset   <format = hex>;
 
    local uint32 jointCount = 0;
    if (trackCountsOffset != 0)
    {
        PushFP();
        FSeekOfs(trackCountsOffset);
     
        // determine number of joint entries
        while (ReadUByte(FTell() + jointCount) != 0xFF) ++jointCount;
     
        uint8 jointTrackCounts[jointCount];
        PopFP();
    }
     
    if (trackInfoOffset != 0 && jointCount > 0)
    {
        PushFP();
        FSeekOfs(trackInfoOffset);
     
        local uint32 jointNum = 0;
        struct
        {
            for (jointNum = 0; jointNum < jointCount; ++jointNum)
            {
                struct
                {
                    struct TRACK_INFO
                    {
                        uint16 dataSize;
                        uint16 unknown0x02 <format = hex>;
                        uint32 dataFlags   <format = hex>;
                        uint32 dataOffset  <format = hex>;
                     
                        PushFP();
                        FSeekOfs(dataOffset);
                        uint8  trackData[dataSize];
                        PopFP();
                    } jointTrackInfo[jointTrackCounts[jointNum]] <optimize = false>;
                } jointTrackInfo;
            }
        } jointTrackInfo;
        PopFP();
    }
};

This has not been updated with my latest findings as i am still debugging through the game code.

But essentially the Aj files appear to contain a list of animation tracks representing individual elements that can be animated. i.e. there are individual tracks for the x,y, and z values of a position, and similarly for rotation, scale, etc.

The 'char[]' array you mention that is the same length as the number of joints contains the number of tracks associated with that joint. So starting from the beginning of each array take the count from the 'track count array' and assign that many tracks from the current index point in the TRACK_INFO array to that joint, the next joint with a count greater than 0 would start gathering tracks from where the previous one left off. So essentially joints with a track count of 0 are not animated.

The 'dataFlags' in TRACK_INFO is actually 3 bytes: track_type, data_flags_0, data_flags_1.

The track type can range from 0 - 57, but for main animations the most notable ones are:
(1, 2, 3) - (x,y,z) rotation
(5,6,7) - (x,y,z) translation
(8,9,10) - (x,y,z) scale

The 'data_flags' as i currently have them are a combinations of the GXCompType (actually the inverse) shifted by 5, or'ed with a 5-bit exponent used to calculate a power of 2 scale to divide values in the track by when the GXCompType is not floating point.

As an example, a data flags value of: 02 87 88 00
Would comprise the track type: 2 (y rotation)
data flags 0: 0x87 which breaks down into 0x80 and 0x07 when split into component type and scale, so would be (0x80 >> 5 = 4) for the component type. Since it is inverted (GX_FLOAT - 4 = 0) which represents a data type of GX_U8 (unsigned byte) values. 2^7 is 128 so then the values are divided by 128.
And similarly for the second data flags value.

How all this is used to put everything together is the process i am currently investigating.

Kind of a rush explanation of things. Will clarify and update this as i find things out.

it actually a good bit more involved than this, but i am still debugging through the logic.

Will let you know when i have things worked out to a point where it is somewhat useful.
May take a bit as i still need to verify the track types, joint ordering/assignment, and the origin of a few values in the process, but will have to see how things go.

Well that is it for this initial teaser of things to come.

 

[revel8n]

Minor update.

i have finished reversing the main processing of the track data.

Not as complex as i though it would be.

The 2 data flags are the 'value format' and the 'tangent format'.

There appear to be 4 main interpolation types: step, linear, hermite?, and "constant" (in quotes as i am not sure, mainly continues using the set value until another is set).

The data is mainly a list of keyframe "commands" structured as:

"command": 4-bit command/interpolation type, or'ed with keyframe count (up to 16-bits; continue reading bytes as long as the current byte has 0x80 set, + 1)

Then for "keyframe count" times, depending on the "command" value:
either read a value, read a tangent, or read both a value and tangent
- the data format structuring from my previous post are used here to read the specified sized value/tangent (as little endian)
followed by reading a keyframe duration value (up to 16-bits; continue reading bytes as long as the current byte has 0x80 set)

i am speculating that once the 'keyframe count' values are read it is possible to continue this process as needed to included as many sections of values using a particular interpolation type as needed (i.e. it would be possible to mix keyframes that are initially linearly interpolated, followed by stepped keyframes, etc., but have not encountered it yet.)

i am glossing over a lot of the details, as i still need to verify that everything works when put into practice, and of course there are still a lot of unknowns (mainly around joint flags).

The game code does not do it all at once, but by using the passage of time in frames to determine whether more keyframe data needs to be read or if the 2 values (and if needed, tangents) it already has are enough to interpolate and update the corresponding value.

A current and next value (and tangent) are gathered and used to calculate the current value interpolated against the current duration between frames. The interpolation method of the current key is used (it is possible for the next key to have a different interpolation type).

i have also only really looked at the track types necessary to update the most important values of a joint. i have not checked to see if any of the other track type values may also be relevant yet.

i am currently deciding how i want to go about verifying the joint ordering of the associated tracks to player skeleton, but will see where things take me.

Given the current simplicity observed i am a little sad it took me this long to get around to looking into it, but on the other hand it has shown me that proper tools are very helpful to accomplishing a task efficiently (possibly more on this later, but enterprising dolphin debuggers and reverse engineers looking to improve their debugging user experience might find some interestingly useful repositories on github under a username suspiciously similar to my own, heh).

Oh well will let you know when i have more information and/or results.

Hopefully this will at least pique some interesting in deciphering the data. Feel free to ask for any clarifying information.

 

[SinsOfApathy]

Given the current simplicity observed i am a little sad it took me this long to get around to looking into it, but on the other hand it has shown me that proper tools are very helpful to accomplishing a task efficiently (possibly more on this later, but enterprising dolphin debuggers and reverse engineers looking to improve their debugging user experience might find some interestingly useful repositories on github under a username suspiciously similar to my own, heh).

That IDA plugin gave me a stiffie, not gonna lie.

 

[Tiberious]

So, is this going to lead to a tool that can extract models from the .dat files with their weighting onto PC? We have 'em for Brawl, and even Smash WiiU, and Melee's going to be 15 this year, so it kinda should have one too.

 

[revel8n]

That IDA plugin gave me a stiffie, not gonna lie.

lol. i would say that this is the response of a true debugging enthusiast, but that might be admitting too much...

...Anyway, yes, being able to debug with proper support for comments, structures, code flow graphs, cross references, etc. is extremely useful.

So, is this going to lead to a tool that can extract models from the .dat files with their weighting onto PC? We have 'em for Brawl, and even Smash WiiU, and Melee's going to be 15 this year, so it kinda should have one too.

This will lead to support for animations in any eventual tools, but as far as being able to extract mesh, texture, skeleton, and skinning information from .dat files has been possible for a while (at least the information to do so has been available).

Making it easier and less technical to do so on the other hand...
But in order to not raise hopes too high, all i can say about that for now is, good things come to those who wait.

Will see what can be done.

 

[SinsOfApathy]

lol. i would say that this is the response of a true debugging enthusiast, but that might be admitting too much...

...Anyway, yes, being able to debug with proper support for comments, structures, code flow graphs, cross references, etc. is extremely useful.

Would you mind tagging up a release of the plugin on your Github? Compiling it myself will probably be a PITA.

Edit: Nevermind. Resolved the issue I was having and it's compiled.

For anyone interested, here's an IDC script from, I believe Shuffle, which will load Dolphin maps into IDA. I've been using it for the CSM, but the dual-naming causes some problems.

Spoiler: ReadDolphinMap.idc

/* ReadDolphinMap.idc

   Loads Dolphin .map files into IDA Pro.
   Carl Kenner, 2014
*/

#include <idc.idc>

static main(void)
{
    auto fh;
    auto fname;
    auto pusha, popa;
    auto start, end;
    auto success;
    auto count;
    auto line;
    auto ea; // column 1
    auto name; // column 30
    auto p;
    auto code;

    fname = AskFile(0,"*.map","Load a .map file from Dolphin emulator...");

    fh = fopen(fname, "r");
    if (fh == 0) {
        Message("Can't open %s\n", fname);
        return;
    }

    Message("Loading %s dolphin map file:\n", fname);

    for (count = 0; 1; count++) {
        line = readstr(fh);
        if (line == -1)
            break;
        if (strlen(line)>30 && line[0]!=" ") {
            ea = xtol(substr(line,0,8));
            name = substr(line,29,strlen(line)-1);
            if (substr(name,0,3)!="zz_") {
                if (!MakeNameEx(ea,name,SN_NOCHECK | SN_PUBLIC | SN_NON_AUTO |SN_NOWARN)) {
                    MakeNameEx(ea,name+"_2",SN_NOCHECK | SN_PUBLIC | SN_NON_AUTO );
                }
                Message("ea='%x', name='%s'\n", ea, name);
            } else {
                MakeNameEx(ea,name,SN_NOCHECK | SN_PUBLIC | SN_AUTO | SN_WEAK | SN_NOWARN);
            }
        } else if (strlen(line)>30) {
            ea = xtol(substr(line,18,18+8));
            p = strstr(line, " \t");
            if (p>=30 && ea!=0) {
                name = substr(line,30,p);
                code = substr(line,p+2,strlen(line));
                SetFunctionCmt(ea, code, 0);
                if (!MakeNameEx(ea,name,SN_NOCHECK | SN_PUBLIC | SN_NON_AUTO |SN_NOWARN)) {
                    MakeNameEx(ea,name+"_2",SN_NOCHECK | SN_PUBLIC | SN_NON_AUTO );
                }
            }
        }
    }
    Message("Dolphin map file done.\n");
}

 

[SinsOfApathy]

R revel8n Mind explaining your process here?

I've tried attaching a GDB server to the Dolphin process, then running the plugin. When IDA tries to connect, it just insta-closes the connection. Also tried using it without the process selected, but IDA only returns "pid 0 dolphin" for the attach list.

 

[revel8n]

Would you mind tagging up a release of the plugin on your Github? Compiling it myself will probably be a PITA.

Ok, i will see what i can do.

Will require my fork of dolphin as well currently. Hopefully i can clean up the additions at some point and get it merged into master, but not sure that will happen.

The additions i made to dolphin start TCPGecko and GDBThread instances on the server ports set in the configuration file.

The TCPGecko support is compatible with my fork of GeckoDotNet.
The GDBThread support is compatible with the ida pro plugin.

Eventually there will be another ida pro plugin with both TCPGecko and USBGecko support so that you do not require GeckoDotNet anymore.

 

[SinsOfApathy]

Ok, i will see what i can do.

Will require my fork of dolphin as well currently. Hopefully i can clean up the additions at some point and get it merged into master, but not sure that will happen.

Dolphin has a GDBStub. If you fixed it up to not be ****, they'd probably take it.

 

[revel8n]

The GDBStub in dolphin is only partially integrated into the interpreter and does not compile under Visual Studio due to the use of gcc extensions to the language.

The additions i made work in both Jit an Interpreter mode.

I could probably rework it to actually replace the old functionality. Right now it is a separate addition.

The official ppc gdb stub support also does not fully support the Paired Singles registers, so my implementation is non standard with the official gdb layout.

 

[adjective]

The data is mainly a list of keyframe "commands" structured as:

"command": 4-bit command/interpolation type, or'ed with keyframe count (up to 16-bits; continue reading bytes as long as the current byte has 0x80 set, + 1)

Then for "keyframe count" times, depending on the "command" value:
either read a value, read a tangent, or read both a value and tangent
- the data format structuring from my previous post are used here to read the specified sized value/tangent (as little endian)
followed by reading a keyframe duration value (up to 16-bits; continue reading bytes as long as the current byte has 0x80 set)

So to clarify, Is this how you're saying the data is laid out?

struct command {
uint_4bit command;
uint_8bit[] keyframe_count;
<keyframe data>
}

where <keyframe data> is of length

• keyframe_count * 8 bits in the case of just keyframe values
• keyframe_count * 16 bits in the case of keyframe and tangent values

and keyframe_count is a list of chars that when added together, ignoring the first bit, make the keyframe count?.

keyframe_count = [10000000, 10001000, 00000001]
= [ 0 + 8 + 1 = 9
= 9 (+1) keyframes

or is it by concatenation?

keyframe_count = [10000000, 10001000, 00000001]
= 000 0000 000 1000 000 0001
= 1025 (+1) keyframes

 

[revel8n]

A full example (from PlPkAj.dat):
Track Info:

00 06 00 00 06 85 88 00 00 00 00 00

becomes:

Track type: 06 (y-translation)
Value format: 85 (data type: u8, scale: 1 / 32)
Tangent format: 88 (data type : u8, scale: 1 / 256)

Track data (6-bytes):

11 B0 AE 03 B0 00

read command byte:
First byte is: 11
Interpolation type: 1
Frame count: 1 (+ 1) = 2

Interpolation type 1 specifies reading a value (if it did not you would skip the value):
Value format is u8, so read 1 byte for value:
Value = B0

Interpolation type 1 does not read a tangent. If it did you would use the tangent format to read its value like the previous step.

Next read duration:
AE
Mask with 0x7F to get base value:
2E
Since 0x80 bit is set, continue reading duration from next byte:
03
Shift 7 bits and 'or' with current duration value:
1AE
Byte does not have 0x80 bit set, so duration value is complete.
1AE = 430 (which you can verify is the same as the float value in the figatree header.

So we have a single frame with a value of 0xB0 (176), scaled by 1/32 = 5.5 for a duration of 0x1AE (430) frames.

Since this is the first frame, and at least 2 are needed to interpolate:
First check the Frame count against the number of frames read so far, if it is the same then we would start from the 'read command byte' step again. Since it is not, start from the read value step.

The next value is also 0xB0, for a duration of 0 (specifies the end in the case; our initial frame count was only 2 to begin with.)

i guess you could do:

struct keyframe
{
int interpolation_type;
float duration;
float value;
float tangent;
};

And create an array using the process outlined above. That way you can essentially treat all keyframes the same when evaluating them.

 

[Tiberious]

[A]s far as being able to extract mesh, texture, skeleton, and skinning information from .dat files has been possible for a while (at least the information to do so has been available).

[snip]

[G]ood things come to those who wait.

Will see what can be done.

The main reason I asked is that, while I know that Tcll is working on something similar, I've kinda given up hope on him actually getting something out that works fully for this title. Every time he gets close, he scraps everything he has and starts fresh.

To illustrate what I mean, I have the most recent release of his tool, which only outputs to .obj, and it was released in 2013, going by the Last Modified date on the files. That's why I'm putting my hopes on this project.

 

[zankyou]

The main reason I asked is that, while I know that Tcll is working on something similar, I've kinda given up hope on him actually getting something out that works fully for this title. Every time he gets close, he scraps everything he has and starts fresh.

To illustrate what I mean, I have the most recent release of his tool, which only outputs to .obj, and it was released in 2013, going by the Last Modified date on the files. That's why I'm putting my hopes on this project.

Im not sure why this is an animation and model workshop but the stuff in this thread looks solely to be cracking animations so it shouldnt really help you for what you want.

 

[adjective]

@revl8n
Thank you for the detailed description! It really helped clarify some things.

I've got my thing printing out all the keyframes of every track for an animation right now, and it seems to be going all right. I think all that's really needed to start experimenting with an exporter now is the meaning of the interpolation-type flag. I'll spend some time playing around with different values in the future, but this is finals season for me so I probably shouldn't be doing too much with it right now.

 

[revel8n]

@revl8n
Thank you for the detailed description! It really helped clarify some things.
...
I've got my thing printing out all the keyframes of every track for an animation right now, and it seems to be going all right. I think all that's really needed to start experimenting with an exporter now is the meaning of the interpolation-type flag. I'll spend some time playing around with different values in the future, but this is finals season for me so I probably shouldn't be doing too much with it right now.

Looks good so far, though i think you may have a small bug. The 03 byte for the second keyframe shown, is part of the duration value of the first keyframe, so the next value byte should actually be B0 again with the next duration byte being 0.

What are you writing your tool in?

// command : interpolation type
1 - read value : step
2 - read value : linear
3 - read value : hermite
4 - read value and tangent : hermite
5 - read tangent : hermite
6  - read value : constant

// data formats (little endian)
0x00 - float (no scale)
0x20 - s16
0x40 - u16
0x60 - s8
0x80 - u8

 

[adjective]

ah, thanks for the catch. I'm writing it in C++. The code is a little spaghetti (especially for the animation stuff, because I misunderstood the structure when I was first writing it), but it's up on github.

 

[D3WD]

adjective - nice, way to pick up the ball and run with this.

 

[Tcll]

why have I not heard word of this adjective :/
here I've been giving ploaj all the credit.

and yes T Tiberious , the only reasons UMC only exports OBJ in that time are because I had a lack of knowledge to actually export anything else, and because nobody with knowledge offered to help me or build scripts.

even to this day UGE still only supports exporting OBJ, however, the API has come a long way and is actually able to export far more.

so don't put me on the line because nobody wanted to help me.
(I'm actually getting help now, but I still need internet before I can really do anything, and that's coming soon)

it is niceto see some competition though, sorry I couldn't do anything a bit faster... blame life.

I'll have more to offer here once my API is actually functional and I'm able to look into this stuff again.