• Welcome to Smashboards, the world's largest Super Smash Brothers community! Over 250,000 Smash Bros. fans from around the world have come to discuss these great games in over 19 million posts!

    You are currently viewing our boards as a visitor. Click here to sign up right now and start on your path in the Smash community!

Melee dat format...

Tcll

Smash Lord
Joined
Jul 10, 2010
Messages
1,780
Location
The Gates of Darkness
NNID
Tcll5850
Tcll, what's with the multi posting? Keep it to one post, bro.
shut it... lol :p

yea...
it's been weeks (almost months), and I've been the only one keeping this thread alive...
(I hate dead threads)

besides... I usu have a good reason for multiposting...
I've explained everything like 3 times already though, and don't feel like doing it again...

Required self bumping because nobody cares about this hidden gem of a thread.
this is one of my explanations... heh

thanx

EDIT:
*sigh* why are people so overobsessed with multiposting...
if it's pointless multiposting then yes, by all means, start a commotion...

otherwise there's nothing to worry about...

^as the rules for my forum state >_>
 

Tcll

Smash Lord
Joined
Jul 10, 2010
Messages
1,780
Location
The Gates of Darkness
NNID
Tcll5850
for the bases (root nodes),
how does the indexes work

index[0]: name

index[1]: 'Share' or 'ACTION'

index[2]: ACTION( name )
index[2]: share( 'joint' or 'matanim' )

index[3]: share( matanim( 'joint' ) )

hmm
I think they may reference the names of functions

BUT

I think the matanim applies to the 'joint' in the first base
(like using 'joint' as a reference)

IDK
I need something with more 'Share' info

sry...
shoulda mentioned I'm calling the root_nodes "bases"
since that's what they are (the base of the heirarchy)
 

Tcll

Smash Lord
Joined
Jul 10, 2010
Messages
1,780
Location
The Gates of Darkness
NNID
Tcll5850
I think I've figured out the purpose of the offset table...

basically, it's a list to make the heirarchy easier to handle...
I think it breaks it up into specific parts to order it in an easier way...

but IDK for sure yet

on a side note, I've finally gotten the bones (well offsets) to log w/o errors
couldn't get the freakin thing working right fpr the longest time DX

anyways... so that's that :)
 

Tcll

Smash Lord
Joined
Jul 10, 2010
Messages
1,780
Location
The Gates of Darkness
NNID
Tcll5850
hey revel...

what are the attr types??

type 1 is ??
type 2 is ??
type 3 is vector

and the unk value in the attr...
I think that's 2 8bit values
the first of which would be a multiplier

the 2nd is I'm not sure

ATTR_DATA
0x00(4) = Attr
0x04(4) = AttrType
0x08(4) = compCnt
0x0C(4) = data_type

0x10(1) = multiplier
0x00(1) = unk
0x00(2) = entry_stride
0x00(4) = dataOffset

another thing is,
I think each attribute represents a value of the facepoint...

Pika has 5 attributes, meaning each facepoint should have 5 values

it's amazing how much simpler this format is compaired to mdl0 :cool:

complicated to figure out, yet way simpler than mdl0 XDD

EDIT:
here's a few things I've figured out:

the value of 'Attr' fits directly into the value of the facepoint index list:
(the list contains 20 values)

[
vert/normal_mtx ,
uv0_mtx ,
uv1_mtx ,
uv2_mtx ,
uv3_mtx ,
uv4_mtx ,
uv5_mtx ,
uv6_mtx ,
uv7_mtx ,
vert,
normal,
color0,
color1,
uv0,
uv1,
uv2,
uv3,
uv4,
uv5,
uv6,
uv7
]

the value of 'AttrType' fits into the CP index type

0 - null
1 - direct data
2 - 8bit index value
3 - 16bit index value

that's all I know for now...
I'ma try to get a model exported >:D
 

Tcll

Smash Lord
Joined
Jul 10, 2010
Messages
1,780
Location
The Gates of Darkness
NNID
Tcll5850
Hey did you steal my text color?
dude...
I had my text color long before you asked me about making your own converter

not to mention yours is a lighter version of mine :p

but speaking of your converter, are you still working on that?? :/
 

Fortress | Sveet

▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀
Joined
Dec 21, 2005
Messages
16,256
Location
Northern IL
No i havent worked on it in a while. I have been busy with school and work, and i have sort of lost interest in the format
 

Tcll

Smash Lord
Joined
Jul 10, 2010
Messages
1,780
Location
The Gates of Darkness
NNID
Tcll5850
for the unk value in the attributes:

Pika:
Nr:
vert: 0B00
norm: 0600
UV: 0C00
Re:
vert: 0B00
norm: 0600
UV: 0C00

Pichu:
Nr:
vert: 0B00
norm: 0600
UV: 0D00

I'll add more later >_>

EDIT:
was checking PlPkRe.dat, and found this:
00 00 00 00 00 00 00 01
00 00 00 00 00 00 00 04 00 00 00 00 00 00 00 00

00 00 00 09 00 00 00 03 00 00 00 01 00 00 00 03
0B 00 00 06 00 00 00 00 00 00 00 0A 00 00 00 03
00 00 00 00 00 00 00 01 06 00 00 03 00 00 34 00
00 00 00 0D 00 00 00 03 00 00 00 01 00 00 00 03
0C 00 00 04 00 00 4D 80 00 00 00 FF 00 00 00 02
00 00 00 00 00 00 00 04 00 00 00 00 00 00 00 00

00 00 00 09 00 00 00 03 00 00 00 01 00 00 00 03
0B 00 00 06 00 00 00 00 00 00 00 0A 00 00 00 03
00 00 00 00 00 00 00 01 06 00 00 03 00 00 34 00
00 00 00 0D 00 00 00 03 00 00 00 01 00 00 00 03
0C 00 00 04 00 00 4D 80 00 00 00 FF 00 00 00 02
00 00 00 00 00 00 00 04 00 00 00 00 00 00 00 00

^that's a single block
but it contains a copy of attr's [1:4]

actually... normal Pika has copies too where as Pichu doesn't

wonder what that means >:D

added PkRe above btw
 

Tcll

Smash Lord
Joined
Jul 10, 2010
Messages
1,780
Location
The Gates of Darkness
NNID
Tcll5850
I figured out that unk value in the attributes :D
well... partially >_>

it's actually 2 8bit values
the first is the modifier, the 2nd is still unk

but Pichu is now imported into Blender (verts) :)
 

L-a-t-e-r-a-l-u-s

Smash Ace
Joined
Dec 18, 2009
Messages
811
I figured out that unk value in the attributes :D
well... partially >_>

it's actually 2 8bit values
the first is the modifier, the 2nd is still unk

but Pichu is now imported into Blender (verts) :)
Sorry, I've just kind of skimmed this thread. Is this related to model hacking in Melee? Because if so, I'm interested :B
 

Tcll

Smash Lord
Joined
Jul 10, 2010
Messages
1,780
Location
The Gates of Darkness
NNID
Tcll5850
would this img tell you anything :3



well actually, it's more related to the format itself...
the img can't really tell you that... eh-heh
 

Tcll

Smash Lord
Joined
Jul 10, 2010
Messages
1,780
Location
The Gates of Darkness
NNID
Tcll5850
may as well add something useful to the thread...

Calculating Transforms:


as for returning the matrices to be calculated with a vert,
you will have to inverse them after retrieving them

then append them to a matrix list:
matrix_list = [M1,M1,M2,M4]

then append that to a transform list:
Transform_list = [[M3,M0],[M1,M1,M2,M4],[M5]]

once you have that you can use facepoint[0]/3 to give you the index of the transform:
calc_transform(Transform_list[facepoint[0]/3],Vert_list[facepoint[9]])

the calc_transform function above would multiply the current matrix with the vert to give you the transformed vert.
but it only sounds simple...

to get the transformed vert, you'd have get the value from each matrix in the group,
and then add the values together

as for calculating a single matrix:

TX = (VX*M[0])+(VY*M[1])+(VZ*M[2])+M[3]
TY = (VX*M[4])+(VY*M[5])+(VZ*M[6])+M[7]
TZ = (VX*M[8])+(VY*M[9])+(VZ*M[10])+M[11]

this would get you the transform value for a single matrix in a group
once you've calculated all the values for the matrices, add the values together

I personally prefer this method:
Code:
def calcTransform( matrices, vert ): #to be performed on the current vert
    tx,ty,tz = 0,0,0
    for mtx in matrices:
        matrix = mtx
        tx += (vert[0]*matrix[0]) + (vert[1]*matrix[1]) + (vert[2]*matrix[2])  + matrix[3]
        ty += (vert[0]*matrix[4]) + (vert[1]*matrix[5]) + (vert[2]*matrix[6])  + matrix[7]
        tz += (vert[0]*matrix[8]) + (vert[1]*matrix[9]) + (vert[2]*matrix[10]) + matrix[11]
    #transform to be added to vert (during export conversion)
    return [ tx-vert[0] , ty-vert[1] , tz-vert[2] ]
where 'matrices' would be the current transform group,
and 'vert' would be the current vert you're applying

now you have your transformed vert ;)



I only subtract the vert from the transform vert because of the fact that my format gives you the option weather or not to apply the transfroms to the verts

thus you can chooze between the RAW vert positions and the transformed positions
 

Tcll

Smash Lord
Joined
Jul 10, 2010
Messages
1,780
Location
The Gates of Darkness
NNID
Tcll5850
well, after a few years of being homeless, and then taking some time to study the depths of advanced GL,
as well as learning a little C++ to translate other sources...


I've finally decided to look into the format again, and I've gotten pretty far into figuring out the relocation table. :)


The relocation table holds nothing more than the offsets of EVERY (valid) pointer in the file.
The order of this table is simply the order of the pointers in the file from start to end.

it's literally nothing more than a pointer array.
(that's what I'm renaming it now btw)


I think the pointer array is used in building the structures of the file before the root pointers (root nodes) direct you to the first structure.

how the structures are built, I'm still working on...

but recent findings in Kirby Air Ride have led me to looking up new info about nintendo's memory handling interface.

I have a new template, which I'll upload when I can, which is able to identify structures of Melee Pl/Ty files, as well as KAR VcStar files.

it also highlights the data fields with sub-fields and values properly.
 

Tcll

Smash Lord
Joined
Jul 10, 2010
Messages
1,780
Location
The Gates of Darkness
NNID
Tcll5850
interesting tool :D
and just in Magus's area too XDD

too bad I had to go and make a complete idiot of myself for mis-understanding that it's a logic editor, and not a model viewer >.<


EDIT:
May I ask for some help in deciphering this??

I'm still a complete noob at C++ to know how any of this memory structuring stuff works yet...

so I've figured out the pointer array...
but knowing how tricky this format can be,
there's something I need to do before I can begin to look at the root pointer...

how could I manage these (undefined) structures in memory before referencing their offsets and parsing the src tree??


I'm asking because just me talking about this stuff usually helps me figure it out...
it's wierd, but it works. XDD
 

Fortress | Sveet

▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀▄▀
Joined
Dec 21, 2005
Messages
16,256
Location
Northern IL
No worries, I find voicing my questions to be very self productive as well.

Is that all rhetorical? I might be able to give you a few leads if you're really stuck
 

Tcll

Smash Lord
Joined
Jul 10, 2010
Messages
1,780
Location
The Gates of Darkness
NNID
Tcll5850
lol, yea, I'm really stuck on this...
it's not that I don't have any leads...
it's more like KAR confusing the crap out of the format...

I'm not sure which lead to start with. :(
(I'm so confused that some of my leads may be leads in the middle of a lead)
^ I don't even know anymore >_<


so there's my standpoint :p

I could actually use some input, so plox :3


EDIT:
I do have an idea about the structure types, but I'm not sure about conflictions with this idea...

I think the (root) structure type is defined by the size of the structure...
though I'm not sure because of my works with this format... :/

currently: (in bytes)
64 - bone
32 - unknown (KAR) (0x00(4) is a pointer with the value of 16)
8 - matanim

I havn't looked at everything yet, so I'm not sure how well this will work :/
(the string is just a string, there's nothing significant about it)

I also think structuring info is defined by the DOL...
but I still need to learn ASM to get a hold of that XD
 

Tcll

Smash Lord
Joined
Jul 10, 2010
Messages
1,780
Location
The Gates of Darkness
NNID
Tcll5850
Kirby Air Ride :p

btw, I was wrong about the size...
it's 48, not 32. D:
(hadn't looked at it since 2 days ago and forgot)
^ compy's been acting up and I've got 3 HDDs dieing on me :(
- one of them being my 232GB (it randomy loses power) ;_;
- the other just stopps spinning and picks up again as if it's got a corrupt sector...
- the last one just completely shuts down randomly when dealing with data...
I need another HDD... T.T
(at least the 2nd and 3rd HDD's don't act up often, so they're still usable for now...)
^ to transfer data from my 232GB

anyways, I've been moreso looking into Melee since then :p
 

Tcll

Smash Lord
Joined
Jul 10, 2010
Messages
1,780
Location
The Gates of Darkness
NNID
Tcll5850
o_o
I just got the idea working.

here's what I've got:
I'm going to explain it in code rather than text...
Code:
parray = [ ] #our array of pointers (pointing to the file's structures)
rarray = {} #our (named) root pointers

struct = 0
for ptr in parray: #set 'struct' to the offset of the last structure in the file
    if ptr > struct:
        struct = ptr

for k in rarray: #get the names of our root pointers
    root_ptr = rarray[k] #get the root pointer via it's name
    #NOTE: dictionaries are ordered by a hash value of the string rather than an index
    #and that's all the purpose of the string in the file serves.

    for ptr in parray: #set 'struct' to the first structure in the range
        if ptr > root_ptr and ptr < struct:
            struct = ptr

    struct_size = struct-root_ptr

    if struct_size == 64: #joint structure
    elif struct_size == 48: #KAR vcStar start sruct
    else: #unknown struct
lol, sry if this confuses you...
I speak better in code than english when it comes to stuff like this :p

I tried to add enough commenting...
lemme know if you need anything explained :)
 

shuall

Smash Apprentice
Joined
Jun 26, 2013
Messages
155
Location
Philly
Too much time on my hands, and no internet. Made a technical specification document for the dat format as far as I could glean from the first page and libOGC documentation. Don't know if this will be helpful to anyone, but at least now I have all the structs written.
https://docs.google.com/document/d/1TIFquuPCrFEANPAH_CM7A138SX07BQ3c4Q2Wms5I1OU/edit?usp=sharing
Wii hackers seem fond of c# and windows to make all the useful tools they need, which is a problem for someone on linux. So I'm working on a simple C lib and a terminal front end for editing dat files.
 

Tcll

Smash Lord
Joined
Jul 10, 2010
Messages
1,780
Location
The Gates of Darkness
NNID
Tcll5850
sounds nice :D
too bad I can't view it... :<
(Google doesn't like wii)

also, how'd you like to join our convo?? :3
 

Tcll

Smash Lord
Joined
Jul 10, 2010
Messages
1,780
Location
The Gates of Darkness
NNID
Tcll5850
has anyone else been working on this format in the shadows??
I mean, even the private convo's been collecting dust and all...

if anyone has any updates to add to anything, I've created a wiki page to centralize development :)
http://wiki.tockdom.com/wiki/HAL_DAT_(File_Format)
if anyone has any new findings to add, please be sure to update the the wiki with them. ;)

also... I'll be posting more info from KAR about the new structures found in that game... :)
everything from them is currently unknown except for each structure's size and pointers,
so I'll try to keep an active update on that...

but seriously... am I the only person making any sort of progress to anything with this format?? :/
 
Last edited:

Achilles1515

Smash Master
Joined
Jun 18, 2007
Messages
3,211
Location
Cincinnati / Columbus OH
Pretty much everything on that thread goes over my head atm. Sorry I can't help.

I have a question for you though. Do you know the code line(s) in the game that refers to a specific character .dat file to load information from? For example, instead of loading data from PlCaGr.dat (regular green captain falcon), I want the game to load data from PlCaGr2.dat (which would be green ranger captain falcon or something). If this is possible, then I think you could theoretically use this to have more costumes than normal if you had some sort of toggle to switch between what files it loads.

I'm probably really oversimplifying this process, but it was just a thought I had.
 

Tcll

Smash Lord
Joined
Jul 10, 2010
Messages
1,780
Location
The Gates of Darkness
NNID
Tcll5850
yea, most all the game's script is believed to be in the game's DOL.
I have yet to run across possible external modules hidden inside DAT files... heh

but yea, if ya wanna help out, I'm trying to make the wiki page more understandable for noobs who want to get into hacking the format. ;)
would you like to know anything that could help you understand the wiki page?? :)

EDIT: also... the "code" that can be found in the DOL is in the Wii's ASM format.
I'm still learning ASM, so I can't exactly edit the DOL directly yet... heh
(for such a large DOL too)
 
Last edited:

shuall

Smash Apprentice
Joined
Jun 26, 2013
Messages
155
Location
Philly
EDIT: also... the "code" that can be found in the DOL is in the Wii's ASM format.
I'm still learning ASM, so I can't exactly edit the DOL directly yet... heh
(for such a large DOL too)
You can get dolphin to act as a disassembler by dumping the entire ram as assembly right when it loads the main.dol (so nothing is changed). It tries to translate data blocks to assembly too which is usually easy to spot. It's nice, but I agree, it's not so much that assembly is hard to understand, it's just that there's so much of it.

I've still been working on dat files, and the main.dol, when I have the time but I haven't been nearly as prolific as you(Tcll) or achilles (20XX codes are great).
 

OninO

Smash Journeyman
Joined
May 19, 2014
Messages
289
Hey dudes, there was talk in Achille's 20XX thread about a shared symbol map, I think I've figured out what a couple of functions do, but I still haven't found the main loop. Should we get moving on a shared map, and if you guys are keen, how should we share it?
 

Tcll

Smash Lord
Joined
Jul 10, 2010
Messages
1,780
Location
The Gates of Darkness
NNID
Tcll5850
don't worry slash, we havn't stopped ;)
I'm pretty much the only one working on HAL-DAT models that I know of

so what's this about a shared symbol map??
I'm interested to learn more :/
 

shuall

Smash Apprentice
Joined
Jun 26, 2013
Messages
155
Location
Philly
Dolphin lets you name 'functions' (blocks of assembly). There's a file it saves ending in '.map', I think. It is modeled after a file found in old wind waker discs which had debug info accidentally left in.

A shared map would be real useful.

github gist might be useful. Shows changes and allows merges etc. A little less than an entire git repo.
 
Last edited:

OninO

Smash Journeyman
Joined
May 19, 2014
Messages
289
Yeah, I was just linking the site, will link the file when I have a bit more done.
 

OninO

Smash Journeyman
Joined
May 19, 2014
Messages
289
Yeah, that does look very similar, and also very cool. I'm just doing it the dumb way of tracking through function calls by hand. I've already found some interesting functions which appear to be important (lots of sub-calls). Will post a link tonight after I've gone a few layers. The map I've got so far just starts from where Dolphin boots into pause.
 

Tcll

Smash Lord
Joined
Jul 10, 2010
Messages
1,780
Location
The Gates of Darkness
NNID
Tcll5850
thanks :)
it's supposed to do alot when it comes to analysis and testing, including step-rewinding (you can rewind what the CPU did to see where an error is)

the lighter connections display the current step

I don't know enough about CPUs though to fully design this things as it should be designed... heh
(not saying I won't get there though)
^when I do, this thing should be able to emulate just about any CPU with the same features provided


when UGE comes around, you'll be able to actually code with this language ;)
(where "coding" is more like connecting wires between points on a bread-board) :p


how this applies to Melee is DOL and REL files which are pretty much the same as EXE and DLL files
linux executables don't seem to have an extension (just about any format can be executable),
but they too work in the exact same way, requiring libraries and such.

my point is you should be able to convert between them with this :)
(and even detect flaws and further optimize the code)


and I'm just 1 person writing this stuff :p
normally this would take a group of people to solve. XD
 
Top Bottom