Universal Model Converter - Side Development Thread

[Tcll]

even bettwe news!

I've invited a dev to my project, and no he's inviting me to merge UMC with his project called Hobby.

I still have my option fully open, so I can back out any time I want
I don't think APott (Nardcake) would be one to lead me into any kind of situation that could screw me over,

as far as UMC being part of Hobby, it's possible, though me and them operate on ENTIRELY different threads.

I stick with Python 2.7 on WinXP (x64) while they use C on Linux.
don't get me wrong, C is indeed a great language, but with python, you need to jump 1000 less hoops to accomplish what you want.

why 2.7 and not 3x?
because of PyOpenGL and some other libraries.

 

[MopedOfJustice]

even bettwe news!

I've invited a dev to my project, and no he's inviting me to merge UMC with his project called Hobby.

I still have my option fully open, so I can back out any time I want
I don't think APott (Nardcake) would be one to lead me into any kind of situation that could screw me over,

as far as UMC being part of Hobby, it's possible, though me and them operate on ENTIRELY different threads.

I stick with Python 2.7 on WinXP (x64) while they use C on Linux.
don't get me wrong, C is indeed a great language, but with python, you need to jump 1000 less hoops to accomplish what you want.

why 2.7 and not 3x?
because of PyOpenGL and some other libraries.

What is the goal of Hobby?

 

[Tcll]

not exactly sure... heh
just a bunch of highly complex projects cluster'd together is what I'm thinking.

I think it's called "Hobby" just cause all the projects are free-time projects done as a hobby.

I got laughed at when I told them about UGE being able to take EXEs and convert them to DOLs, or essentially logically decompiling them.
they were laughing at me due to how complex the idea was, but they love it. :3

the only downer I have about them is they don't support WinXP due to deprecation. >.<
but their reasoning is the crappy WinAPI interface not being needed anymore, which is decently acceptable...
I can't argue with it cause yes, even I know it's a big mess.


I'm a mis-fit cause I refuse to use anything above XP until MS pulls their heads out of their A's.

yes, I'm basically telling you guys to get XP if my programs don't work on 7.
otherwize, you'll have to write the code to get it working on 7.

I'm not one to make sure my programs only work with XP.
heck my GUI is done entirely in OpenGL which is supported across almost all platforms. (even Andriod)
all I need is python interpreters that work across these systems and my code is good to go.
(my code DOES NOT use single-platform limitations)
^ I don't even use WGL which is GL for Windows

 

[Tcll]

finally got a website for Hobby, and my thoughts were correct
http://hbywre.org/index.html

UMC should be added shortly.

 

[MopedOfJustice]

finally got a website for Hobby, and my thoughts were correct
http://hbywre.org/index.html

UMC should be added shortly.

Awesome! So does joining them mean you get more help?

 

[Tcll]

I hope anyways... heh
like I said... me and them are from 2 different planets...

not one of them even knows Python XD

also... I've been updating the MDL0 format on the MKW wiki. :3

updated it with the v10+ fur specs as well as replaced the "Has Bounds" with "Component Count" and updated the component specs.

I have my own specs I'm working on on my forum as well...
I don't own the wiki, so I'm a little hesitant when it comes to editing it.
(I can do what I want on my forum, excluding PB limitations)

bout time the format saw an update XD

EDIT:
here's the URL:
http://wiki.tockdom.com/wiki/MDL0_(File_Format)

NOTE: I'm currently posting off my wii to save energy.

 

[Tcll]

in even more recent news, I've been working on a wiki page describing my updates to the HAL Labs DAT file format
http://wiki.tockdom.com/wiki/HAL_DAT_(File_Format)

I don't think the MKW community likes it too much, but I know the site creator Tock has the same thoughts as me about the page.
hopefully it won't get taken down... more info about this is posted on the format's talk page.

I'm still working on adding more structures myself, but if anyone would like to help and add even more, just copy and modify one of the existing structures.
or if you just wanna correct minor flaws on the page, that's fine too. XD

the more people we have working on this format, the better

 

[Tcll]

new idea for supporting compression:

reading:

decompress( data_size, 'zlib' ) # automatically creates a temp file

start = bu32() # read from the decompressed data

SwitchFile() # switch back to previous file

writing:

FileID = TempFile()

# file data to compress

compress( FileID, 'zlib' ) # automatically switches back to the outer file.

# optional arguments:
# (including writing the compressed data to an alternate file)
#compress( FileName, 'method', AltFileName/ID )

I'm still working on a new idea for archives... heh
but I think that sums up the compression

 

[Tcll]

finally some progress on something UMC-related


^I've completely rebuilt the hex viewer to be LOADS faster than what I was using previously.

there's still a few large kinks to work out: (one of which can be seen in the image)
- everything is marked as selected. (made a mistake on the selection logic)
- any activity on the window forces the hex viewer to redraw and lose all text. (only the vertical address line is drawn)

but for some real progress:
the file I have loaded is ACTUALLY loaded through dev5's file system

the color of the hex values was derrived from HexEdit's ANSI format
I'm thinking about including 0x0B and 0x0C in the list of printable (black) characters as found in python's 'string' module

EDIT:
well I fixed the first bug:


but this bug has me feeling like Markiplier playing the impossible quiz/book:

 

[Tcll]

just now fixed the 2nd issue:


I'm such an idiot... XDD
I was making modifications to the file's offset in the view-code...

the offset wasn't being reset because it's referenced as local to the file, which I hadn't considered

all that's left to do now is fix a small selection issue...
(I currently can't select anything)

 

[Jabberwockxeno]

How functional is this tool as of now? Can it extract ALL models? I know that the tools used in the extraction of assets from the halo games aren't capable of extracting level geometry, for example, are there similar issues with this tool?

Right now, all I need is to be able to extract the rayquaza model. I don't need the textures. I don't even need the rig data as long as he's already in a straight pine, pose wise.

 

[Tcll]

ah, then you should have no problem building a script for dev4.5 then
I'm still working on material/texture export functions for the dev4.5 update which is why I havn't released yet, but even the export system is getting a slight revamp here...
as far as importing, you can't really set weights in the currently released dev4.5... you can set bones however, but yea... it's kinda crappy...
rig-less models should import and export perfectly though. (as in up to UVs)

the next release of dev4.5 will fix the material and weighting issues in both import and export.

as for dev5, I've rebuilt the text viewer for UMC_SIDE: (using a QPainter widget)

it doesn't scroll or select yet though, which I'll get to.

this will allow me to more easily highlight specific areas of text where the script operates on the file.

I'mma work on a quicker hex/text toggle at the right of the bar.
(the view menu will still be there, but will contain the options for what each side displays)

 

[Jabberwockxeno]

ah, then you should have no problem building a script for dev4.5 then
I'm still working on material/texture export functions for the dev4.5 update which is why I havn't released yet, but even the export system is getting a slight revamp here...
as far as importing, you can't really set weights in the currently released dev4.5... you can set bones however, but yea... it's kinda crappy...
rig-less models should import and export perfectly though. (as in up to UVs)

the next release of dev4.5 will fix the material and weighting issues in both import and export.

as for dev5, I've rebuilt the text viewer for UMC_SIDE: (using a QPainter widget)

it doesn't scroll or select yet though, which I'll get to.

this will allow me to more easily highlight specific areas of text where the script operates on the file.

I'mma work on a quicker hex/text toggle at the right of the bar.
(the view menu will still be there, but will contain the options for what each side displays)

As I said, texture/material/rig/weighting isn't important, as long as the model geometry is there and it's posed the right way to begin with. I assume then this can export the model to an obj or some other format I can use?

 

[Tcll]

eh... sorry... I was only trying to mention basic models was currently it's thing if by chance you did want anything advanced.
but yea, it only exports OBJ currently.
(you can build export scripts for it, but I'd rather wait until the next update for that if I were you)
^I'll have better functional support for exporting

currently though the problem with the next dev4.5 update is the fact I'm more focused on dev5's interface...

basically, dev4.5 gives you a kiddie-pool to play in with back-yard room,
while dev5 gives you a full-out water park with room for many more toys.

and not to mention UMC_SIDE, which btw I've gotten text-view scrolling working...

UMC_SIDE gives you everything you need for noobs to build highly complex script-interfaces for UMC.
(let's not forget UMC's scripting interface is already designed for noobs who havn't done a mathematical equation in their life) XD

 

[Jabberwockxeno]

eh... sorry... I was only trying to mention basic models was currently it's thing if by chance you did want anything advanced.
but yea, it only exports OBJ currently.
(you can build export scripts for it, but I'd rather wait until the next update for that if I were you)
^I'll have better functional support for exporting

currently though the problem with the next dev4.5 update is the fact I'm more focused on dev5's interface...

basically, dev4.5 gives you a kiddie-pool to play in with back-yard room,
while dev5 gives you a full-out water park with room for many more toys.

and not to mention UMC_SIDE, which btw I've gotten text-view scrolling working...

UMC_SIDE gives you everything you need for noobs to build highly complex script-interfaces for UMC.
(let's not forget UMC's scripting interface is already designed for noobs who havn't done a mathematical equation in their life) XD

OBJ will work just fine, as long as rayquaza is exportable. I might end up doing some other stuff down the road, but I have other projects I am involved in, I can't first start to get into smash stuff now.

 

[Jabberwockxeno]

Actually, it seems the download link for the dev build is down.

 

[Tcll]

Actually, it seems the download link for the dev build is down.

oh woops. looks like I forgot to update the links ._.
thanks

I'm using 4Shared now, since DB has become more gov't affiliated...
http://www.4shared.com/archive/ktmXUSiiba/UMC30ad45.html?

 

[Jabberwockxeno]

oh woops. looks like I forgot to update the links ._.
thanks

I'm using 4Shared now, since DB has become more gov't affiliated...
http://www.4shared.com/archive/ktmXUSiiba/UMC30ad45.html?

Somewhat off topic, but what do you mean by "more government affiliated"?

 

[Tcll]

meaning the gov't can peek into your DB files even easier than they could before

 

[Tcll]

just spent my time rewriting the hex editor again, and this time, I'm extremely proud of my work. =D
take a look:

notice that little clip-off at the bottom?

the data that's drawn is literally the only thing put through the display loop,
meaning this thing is LOADS faster than it was when I got the original code.

and now that I can test this out, the FitPikachu00.mdl0 took about 2 seconds to import in dev4.5
using dev5's file system, it loads up in less than half a second! =3
(note that this is for my speeds on my system, where the results will vary on other compys)

looks like my work and research paid off after all. XD
you can expect dev5 to be extremely fast and efficient in everything it does.
(and for Python, the slow language it's built in, that's saying a heck of alot) :D


just give me time guys,
this gem has only just begun to shine. ^_^

/me is excited

EDIT:
btw, that clip-off is mendable by drawing 1 extra line

 

[Tcll]

just rewrote the text viewer doing the same thing as the hex viewer:

^ loaded a large text file that used to cause it to hang, and it loaded instantly! =D

everything's just so fast now, I love it ^_^

also, I did the endpos+1 thing, so now the cut-off is past the viewport height

 

[Tcll]

for development purposes, who here likes the idea of an automated author and credits system??
(if you publish a script/library, you're the author, if you edit someone else's script, and the author likes it, you get credit for it)

I actually wanna make the credit system point out what your credit is for to other developers :D

this will come with a plugin for UMC...
CREDIT.py for DEVEL.py with an optional COLLAB.py (ideas from Nitrous.io)


anyways... I'm working on the code editor for UMC_SIDE now...
I'll work on selection and copy later on...
I just wanna get the basic automated interface working... heh

 

[Tcll]

finally figured out a simple syntax highlighter...
here's a test:


selection and advanced keyboard features don't work at all...
using a QPainter() to do this is quite new.

you can bet there's gonna be syntax highlighting for UMC functions
I'mma actually draw a BG color around them to represent the colors of the hex/text-highlights.
(so you can easily correlate the functions to the data)

EDIT:
oh also...
if you're wondering about those 2 extra buttons near the minimize button, those are for me...
they're Nvidia controls which allow me to maximize a window to a single monitor,
or send the window to the same position on the other monitor.

right now due to power issues, I'm only using 1 monitor, but the options handle the grid exactly the same.

 

[Tcll]

just posting some quick progress before going to bed:

yeh, I prefer IDLE's syntax colors over all

and this thing is still blazing fast! :3

I'll work on bold and italic styling later on...

still gotta work on input events...
all this thing can do atm is scroll... heh

 

[Tcll]

nearly finished with the syntax highlighter

all that's left to implement is the function and constant collections from UMC's interface
(as in any updates made to UMC's functions will also show up in the IDE)

a little about the shown syntax:
Header() is red because it's the function that actually registers the UMC script in particular areas.
(currently the script is registered as a Model I/O script, which is validated by the functions)

the classes Import() and Export() with the Model() functions are statically used by UMC which is why they're highlighted as teal

the class usage makes the function structure usage a little more viewable.

there's also an extension I can do with this method for the GUI plugin to allow custom dialogs.

 

[Tcll]

well dang... never thought Giga Bowser would look near perfect... o.o

^ his skin's white because it's a bump-map

I've done nothing new to the Melee script... dev4 still doesn't support NBT...
the normals for his skin are taken from the NBT vectors while the bi-normals and tangents are discarded...

what got me to look into it was the brawl community looking for NBT data...
Brawl does NOT have NBT support...
(Brawl's GBowser uses basic normals)

thus, Melee is higher quality than Brawl

 

[Tcll]

here's some progress on UMC_SIDE:

^the colors for the highlights are derived from a hash value for the base-functions.

bu32 = bu(4)

the purpose of these highlights is the colors for these functions will also be used to show the data they cover in the test files.

also, as a new idea for the interface, I'm thinking of putting yet another splitter for the test files side, which will display a previewer above the file you're testing.
the previewer will have tabs for showing the model with it's images, so you can know what you're importing

EDIT: small update, highlighted UMC constant variable definitions:

all that's left to support is the formatting functions

 

[Tcll]

so here's the final update to the syntax highlighting:

now let's see if I can get to work on some of the advanced features:
- outlining (collapse anything with a ':')
- VS2010/PyTools-like intellisense and code completion (best intellisense I've seen yet, and the reason I love VS2010)
- Aptana/Eclipse code convenience functions ( type "def ", it automatically adds "():" )

but before I can get to those, I've still gotta get a cursor going and fix the scrolling and what not

even make this thing faster by tracking horizontal positioning and skipping what's past the width of the code editor.

other future intentions:
- function catalog with info (D-Click a function to add it to your code)
- split-window, dual-view, and multi-editor view

also, the code-collapsing is not fixed to a single line like alot of IDEs...
for example:

[+]- class Import:
 |
 |  [-]- def Model( UMC, FT, CMD ):
 |
 |  [+]- def Animation( UMC, FT, CMD ):
 |   |
 + - + -     pass

hope you guys like it


EDIT:
actually, that's not exactly the final syntax update...
I intend to fully highlight a struct as well as the variable it's stored on.

colors will also be picked for an array in this manner

basically, instead of just "struct" or "array" being highlighted as displayed above, the whole thing will be highlighted,
where the color for the highlight will be based off the struct's contents.
and because a struct() counts as a UMC-type, the variable it's over will be highlighted with that color as well.

here's what I mean:

^ the magenta would be an array
^ the gray would be a struct
^ the rest of the highlights would be the values of the struct

 

[Tcll]

just wanted to post I've made alot of updates to UMC_SIDE
the code editor actually functions just about properly now :D
(there's just one small issue when pressing Up on before a blank line, when the line gets skipped)

as you can see, I have a cursor now
the only selections that work atm are Ctrl+A, which entirely works perfectly
one of the biggest things, dynamic type definition works properly now :D
(as in you can type "bu(5)", which isn't defined upon startup, and it'll now highlight)

I still have yet to implement highlighting a variable given a type-def, so as you can guess, struct() doesn't work yet

anyways... I've fixed the scrolling as well as line numbers no longer scroll with the text.


up next, I'mma try to get the file data to highlight with the script...
but first I'm gonna go back to the old function layout...
using a class makes things overly complicated,,, heh

 

[Tcll]

here we are, and I got structs working properly

^ messed up on the export function, but oh well

EDIT: aaand I see a few bugs with the builtins and such... heh
oh well, I'll get to those

 

[Tcll]

btw, if anyone wants to help, I do have my forum set up as a sort-of wiki for anyone placed in the UMC Helper group

for here, you can edit the function documentation for both dev4.5 and dev5
http://tcll5850.proboards.com/board/42/umc-related

as for UMC_SIDE, right now I'm taking a break while I await some help for some new features I'm trying to work on...
(I can't concentrait when I want to work on the other things I want to add to it because of this, so I call a break)

meanwhile, I'm taking a hit at dev5's dual-ended model interface...

only progress I got on it atm is all code... (nothing to really show)

 

[Tcll]

from now on, dev5 will be known as v3.0 (no more 'a')

dev5 is proving to be the exact interface I wanted since I started v3.0a, so now I can officially call it v3.0. =D

no joke, I wanted the whole update system and everything, including mods to the system
and now I actually know most of what I want to pull off :D

and I'm changing the dev system a tad... it's gonna be 'rev' (revision) just to fit the standards for development.

hopefully now, the final project, UGE, can start coming into play here soon.

the biggest things this new version will have to offer:
- a better, more extendable, and easier to understand and work with, scripting interface
- updates (both minor and major) including plugins
- plugins (UMC itself is just a console API)
regarding plugins: (when they're developed)
- a fully functional, customizable, and very pretty GUI (after a very simple and informative UI)
- Animations
- Archives/Compression
- Script/Library updates (with notifications) including user-end script uploading (for UMC script developers)
later on:
- an in-program script IDE
- a game script recompiler using a custom language designed for decompilation
upon name change to UGC:
- ability to play an imported game while viewing the actions in the script. (don't expect things to run perfectly, it's not exactly an emulator)

for the pretty GUI part, think of dev4's GUI as it was originally meant to look.
I'll pass around a very simple UI which will display what's needed and render the model as it should look in game. (excluding how it's imported)
NOTE: this UI won't give you any features for modifying the model... nice things take time, so this is something just to scrap with an update.

EDIT:
if anyone happens to skim across this who knows a thing or 2 about modern game development,
I could use a hand:
http://tcll5850.proboards.com/thread/224/umc-help-proper-scene-graph

I'mma try to get some guys guys from the GL dev team helping me out here also.
I want this thing to work right on the very first release

 

[Tcll]

btw, I was wrong about my PT definition.

model formats such as OBJ and DAE are Pre-Transformed (T-Pose), where formats such as Nintendo's are RAW or Un-Transformed

so I'm gonna be changing the function definitions to UT instead of PT

at least I didn't catch this too late


EDIT: btw, a new feature of v3.0 finally defines something I've been wanting to do for a while.

I call them Roots, what they do is define a space for a collection of Scenes...
you can also call them Project Spaces, but I like Roots better.

what does this mean?
it means you'll be able to import multiple models at the same time and save all of them in the SES.
you can merge Roots to combine separate data's for exporting into multiple scenes, and much more later on.

you won't be able to control Roots at script level... these can only be managed via the GUI...
they basically just define editable namespaces for game data.

 

[Tcll]

ok guys, since UMC is getting to be to the point of where I wanted it, I'd now recommend you guys start following me on here:
http://tcll5850.proboards.com/thread/143/official-universal-converter-development-thread

I'm centralizing it's development on my forum now and bringing everything together.
twas fun while it lasted :D

you can still reply here, but just stay updated with that thread

 

[Tcll]

ok, I just need some input: http://tcll5850.proboards.com/thread/144/public-umc-discussion-thread?page=1&scrollTo=937

as for right now, the functions won't return or do anything other than set the data

 

[MopedOfJustice]

A friend showed me this:
http://wiimmfi.profeibe.at/index.php
What say you?

 

[Tcll]

I've questioned Wiimm about it, regarding the fact that I've only just gotten MKW...
it's still nothing I'll play over Brawl (including the addition of a "cutesy" character).

with my self-standards, I can't really say anything about it yet...
(not sure if this is bad or good)

also, I'm reworking a few of the functions I've defined in the v3.0 docs.

compress() and decompress() have been documented as well as ugeSetScene()
http://tcll5850.proboards.com/board/55/data-management

I could use some help here though... writing the functions in the code is a big enough task...
you'll need to ask me for the link to my box if you want to help.
if you're not on Nitrous.io, please ask me for an invite link as well.
(like DropBox, you will give me "N2O" by following that link so I can increase my box's storage space)

 

[Tcll]

btw, work hasn't stopped on UMC_SIDE
got some new ideas and added a few old ideas...

new:
- QGL model viewer with tabbed texture viewers
- Go button, instead of automation to run a script (automation can really slow things down)...

old:
- UMC function key (D-click a function to insert) extended with types and constants

for the new idea, I figured I'd make use of UMC's interface a little bit better
in the tree view, if you select a function dealing with UMC data, the data in the viewer is highlighted, just like the hex/text viewer

anyways, here's an image for what I have so far:


in the key, those types, functions, and constants are defined by UMC's interface entirely
nothing is static to UMC_SIDE.

everything UMC_SIDE does with files and data revolves entirely around UMC's API.

 

[Tcll]

well... I just woke up this morning to get scared out of my witts:

(You have insufficient privileges to reply here.)
^every thread I went to, and even the profile VM post-box was gone... ._.

glad whatever happened is fixed now
though now I'm leary... >.>

EDIT:
if something should happen, here's the main place you guys can reply to:
http://tcll5850.proboards.com/thread/144/public-umc-discussion-thread

I think I'm actually gonna start talking about UMC_SIDE there as well...

cause yea... now I'm scared, since this isn't exactly my domain,
and I don't want you guys to think I've stopped working on this if I should stop replying here... heh

 

[Tcll]

well, I got enough people making me hit myself for not making the release...
so here's a pre-release for you guys.
*4shared link for UMCv30ad45 removed*


what doesn't work yet:
- proper exporting (this still uses the previous functions and exports only up to bones (not weights))
- the GUI (the port didn't work out well, and it's still a little laggy and buggy)

I'd hoped to have these improved and more stable by the actual release.
sorry I couldn't make it guys.

what you can do with it is import materials and textures, and actually read the available features
(these will also be available (including weights) in the SES file if you'd like to try to convert directly to another format)
^ I've provided the SES v1 documentation

you may also edit the source of this release to try to make improvements
as well as release your improvements, but please notify me, I have a group just for doing this. :-\


also, there are known bugs with the OBJ importer, the model is there, it just has a clear texture over it.
there's also another known bug: