Universal Model Converter - Side Development Thread

[Tcll]

just another typical standpoint in the help I recieve:
http://python-forum.org/viewtopic.php?f=6&t=10941&p=16583#p16583

EDIT:
if you think this type of help brings me down...
it actually fuels me in a way...
I'm starting to look at stranac as a horrible helper and can't wait to punch her (along with hundreds of others) with what my program does.
heh

no I don't want to literally punch anyone, but once they see what my program is capable of,
they're gonna take all those comments of my code being stupid back when they see just how easy it is to build a script for it.

^that my friends is how I'm fueled, and why I'm still active on this.
I know what I'm capable of, and I deathly want to show everyone else what I know I can do.

 

[Tcll]

boom: (UMC_SIDE)
http://tcll5850.proboards.com/thread/144/public-umc-discussion-thread?page=1&scrollTo=957

the go button now recognizes the functions being called and returns the appropriate data
now to start feeding the previewers :3

 

[Tcll]

UMC_SIDE update: http://tcll5850.proboards.com/thread/144/public-umc-discussion-thread
put alot of work into this and had to change the interface a bit to get it working... heh

EDIT:
btw, sorry if work on this seems to be slowing down...
I'll be honest, I'm actually trying to speed it up,
but I've only just released another plugin I've been working on (Unlimited Custom User Smileys) for my forum.

EDIT2:
don't forget about my biggest forum plugin I'm working on though:
Unlimited Role-Play Character Dialogs with sync-age between forums

 

[MopedOfJustice]

Thanks for continuing to update this thread. I though you weren't going to.

 

[Tcll]

only to mention updates to my thread, that's all

not everyone wants to jump to my forum to recieve updates

 

[Tcll]

just to update here, I'm taking a short break to work on a few games to gain knowledge for UMC

in the mean time,
I'm also working on another plugin addition you guys might find useful towards UMC development reporting on my forum:
http://tcll5850.proboards.com/thread/249/post-progress-bars-development

^I'm doing it with pure CSS, and actually using another plugin of mine to pull off the display.
(actually these BBC plugins are all part of a plugin pack... heh)

might I mention, PB devs don't like my ideas. XD
@PB devs: it's not my fault... I like to give people what they want. :3

 

[Tcll]

major update to scripts announced:

http://tcll5850.proboards.com/thread/144/public-umc-discussion-thread

also, if I must point out a misconception, I don't build the scripts for converting formats, everyone else does.
UMC provides a scripting interface simple enough for non-programmers and hacking noobs to use and learn from,
and by 3.0, an IDE to make things even easier and more convenient for noobs to write scripts for the interface.

 

[Tcll]

hey moped, if you like this so much, why not lend me a hand in a few small areas?? ;D

for something productive, the documentation really needs some work...
I've got about 3-4 people who are supposed to be working on it, but you see how well that's going.

for something counter-productive, your input on things with any new ideas always helps.

if you want to get into it really big, just ask. ;D

 

[MopedOfJustice]

hey moped, if you like this so much, why not lend me a hand in a few small areas?? ;D

for something productive, the documentation really needs some work...
I've got about 3-4 people who are supposed to be working on it, but you see how well that's going.

for something counter-productive, your input on things with any new ideas always helps.

if you want to get into it really big, just ask. ;D

Not sure how much I can help, given that I know practically nothing about coding.

 

[Tcll]

Not sure how much I can help, given that I know practically nothing about coding.

lol, I was trying to point out you don't have to know coding unless you want to get into the thick of things.

all I'm asking for is help with the documentation, or simple ideas.

just think of things you'd like conveniences for. :3
(things that I'm not seeing)
^ keep in mind, this goes all the way up to UGE which is a full-out decompiler and practically a game-building AI of sorts.

just take a random shot and pick off a strand of hay until you find a needle


for the documentation, I need threads created, and you will need the code for reference to read the comments of the finished functions.

 

[Tcll]

also, one of the things I'd like to do by the release is NBT support, which is MORE than possible,
buuut, I'd have to support both old and new, which complicates things a tad:

old: ["_Mesh", "Name", Material_index, [ Verts, Normals, [ Colors, ... ], [ UVs, ... ], Weights, Primitives ] ]
new: ["_Mesh", "Name", Material_index, [ Verts, Normals, BiNormals, Tangents, [ Colors, ... ], [ UVs, ... ], Weights, Primitives ] ]

one way I could make this easier is to make UMC support only the new and rewrite my SES import script.

this is what makes SESv1 inefficient... heh

EDIT:
forget this... I'm gonna have to make SES1.1 just to support weights in an easy fashion >.<
may as well add NBT support... heh

so as of this, facepoints themselves will be extended to:
[ VertID, NormID, (Col0ID,Col1ID), (UV0ID,...,UV7ID), ((BoneName, WeightID), ... ) ]
for NBT:
[ VertID, (NormID, BiNormalID, TangentID), (Col0ID,Col1ID), (UV0ID,...,UV7ID), ((BoneName, WeightID), ...)]

 

[Tcll]

updated the 3.0 function layout for the 3.0a port-release:
http://tcll5850.proboards.com/thread/182/layout
^the reddish functions cannot be used in 3.0a

I've still got a few updates to make on the functions as the release will be able to both import and export bones and weights
(since the SES format in the current release already supports it)
^the problem was the provided interface being a tad too complex to modify, to where it was a tad too overwhelming to do anything.

 

[solitonmedic]

I can really use this to extract models from Punch-Out Wii?

 

[Tcll]

you'll need to build a script to import the model and another to export it to whatever format you want.

I'd wait till the next release though which will come as soon as I finish porting the import functions from the 3.0 interface.

and yea... if UMC's name isn't obvious enough as to what the program does. XD

also... UMC3.0a doesn't have an archive interface... I'm working on that to be released by UMC3.0

in comparison, comparing UMC3.0a to UMC3.0 is like comparing Minecraft Alpha to Minecraft 1.7.10

 

[Tcll]

ok, so here's the final facepoint format for SES1.1:
[ VertID, (NormID (, BiNormID, TangentID)), (Col0ID,Col1ID), (UV0ID,...,UV7ID), ((BoneID,WeightID), ... ), MaterialID ]

the material ID is local to the materials referenced by the mesh.

the weight usage kindof bamfs the weight-groups method used in SES1, but it's in trade for simplification.
if a format needs a weight group, simply create a dictionary and store each weight in the group, then pass the dictionary to the format.

IK, it's a little more code... I'll have a better solution for UMC3.0

 

[solitonmedic]

I have zero knowledge about programming, I don't think I'll be much help.

 

[Tcll]

yeh... I'm trying to make it to where you don't need to know alot about coding to build a script for UMC
it's not like learning the stuff you need to build a script for Blender ;D

just let me know what difficulties you may run into, and I'll try to make things smooth for you.

documentation is a huge issue atm... I'm trying to work on a decent format >_>

 

[solitonmedic]

I think I could help you on that.

I could help organize everything.

 

[Tcll]

eh... I think organization is pretty decent
here's what I got so far:
http://tcll5850.proboards.com/thread/182/layout
^everything in red won't be in 3.0a
though some of the simple things can be implemented, but later, when ready

really all I need is people to create threads and document the functions... heh
(I'm overwhelmed simply writing them in the code and linking everything together)

I want the threads to be more descriptive than the function.__doc__ though...

 

[Tcll]

just showin some progress

^ this is an OBJ export from v3.0a using v3.0's functions

the problem here is with:
while ugeObjectsExist():
and either:
while ugeFacepointsExist():
or:
while ugePrimitivesExist():

the model only has 1 object, and doesn't get either the last Triangle-Strip (primitive),
or the last facepoint from a set of primatives (thus not drawing the last triangle).

if this is a facepoint issue, the OBJ script could also be responsible >_>

 

[Tcll]

update:
http://tcll5850.proboards.com/thread/144/public-umc-discussion-thread?page=1&scrollTo=1042

once this gets fixed, I can start work on the material, texture, and image export functions, and then hit up the import interface as well as the header function.
(everything should work the same as it should (and more) in the 3.0 release)
I also have to work on the data functions, which unfortunately won't be able to type-check like in 3.0...
the interface there is a bit complex to simply merge

so I can promise a release soon, but I can't say how soon

 

[Tcll]

I... am an idiot *facepalms*
update:
http://tcll5850.proboards.com/thread/144/public-umc-discussion-thread?page=1&scrollTo=1043

and my last update is included with me being an idiot

 

[Tcll]

and here's why Blender's OBJ importer doesn't help people:

^warning, large image

note, the normals are exported to the OBJ as is.

so yea, the developer of the OBJ plugin either doesn't know what normals are,
or the blender developers made it too complex to import them...

I'm assuming the primary option because every other model import script supports normals
(not all model formats support normals, so the scripts for those are excluded)

 

[Tcll]

just wanted to post an update

finally figured out a decent method for the export functions for materials:

def ugeMaterialsExist(Global=False): #defaulted for materials in current object
    global ActiveObject,ActiveMaterial
    materials = range(len(Libs[4])) if Global else Libs[3][ActiveObject][3][2]
    if materials==[]: return False # no materials
    if ActiveMaterial!=None: # continue operations on current material
        index = materials.index(ActiveMaterial)+1
        if len(materials)==index: ActiveMaterial=None; return False
        else: ActiveMaterial=materials[index]; return True
    else: ActiveMaterial=0; return True # start

def ugeGetMaterials(ObjectName=None): #use ugeGetMaterials(ObjectName=False) for global
    global ActiveObject
    if ObjectName==None: return Libs[3][ActiveObject][3][2]
    if ObjectName==False: return range(len(Libs[4]))
    elif ObjectName.__class__ == int:
        try: return Libs[3][ObjectName][3][2]
        except IndexError: pass
    elif ObjectName.__class__ == str:
        for name, vpID, lrs, d, pID in Libs[3]:
            if name == ObjectName: return d[2]
    else: pass

what these do:

while ugeMaterialsExist(): #use this for traversing the materials held by the current object
while ugeMaterialsExist(True): #use this for traversing the materials of the entire import (materials can be shared between scenes)

for MID in ugeGetMaterials(): #use this for referenced-traversal with the materials held by the current object
for MID in ugeGetMaterials(0 or "Object0"): #use this for referenced-traversal with the materials held by the referenced object
for MID in ugeGetMaterials(False): #use this for referenced-traversal with the materials held in the entire import

yea, the same functions work with global and local materials

 

[Tcll]

if you guys are wondering why I updated here, I'm currently having an issue with code-boxes on my forum...
'\n' seems to be getting removed...

 

[Tcll]

skype chat: (discovery)
[5:02:54 PM] DarkPikachu: guess what I just found out
[5:03:10 PM] DarkPikachu: IDK how this thing is even working
[5:03:22 PM] DarkPikachu: the material system is completely borked
[5:03:53 PM] DarkPikachu: the material for the first mesh object is getting placed on the 2nd
[5:04:02 PM] DarkPikachu: yet it still renders properly O.o
[5:04:49 PM] DarkPikachu: so I'm looking up the old SetMaterial function, and yea... idk WTF is going on
[5:05:14 PM] DarkPikachu: I may as well just redo the entire thing and forget about export testing
[5:05:40 PM] DarkPikachu: cause as far as it looks, I'm getting an error because this thing isn't importing right

 

[Tcll]

I must be the world's stupidest person when it comes to 3D development >3<

how long have I been using Blender, and how long have I known about everything the bones contain? (note ? = rhetorical)
http://wiki.blender.org/index.php/Doc:2.6/Manual/Rigging/Armatures/Bones

yet here I am building 3.0 upon primitive bone deformation techniques >3<
how have I not relized this for so long... wow... just... wow

jsyk, I'm not mad, I'm actually LMAO-ing at my stupidity


on another topic, like I've said on skype, development on this is still going on
(I won't stop until it's finished)
I'm just held back, and have been getting into Minecraft recently as I was introduced to the AT Launcher's MadPack.
so I've fallen in love with a few mods revolving around Magical Crops,
and have been rebuilding (multiple times) an ME that basically can give a user Creative Mode with whatever items can be crafted.

I also have alot of mental backup from being overwhelmed which is slowing me down,
Proboards is one cause of that as I can't save edits to my plugins...

just give me time... all I'm getting is slowed down, not stopped
for two on the slow down, I typically don't like to build unless I know what I need to do,
and getting that information takes a pain-staking amount of work and time (because big forums like to take a month to respond).

3.0a however, I'm replanning, which has currently halted it's development till I can piece everything together.
(I want a trophy of DP (my ava) on my desktop (yes, I can place 3D models on my desktop) so don't think for once that 3.0a is stopped)

also... one last project... if you're into mystcraft, you might already know what this is intended for:
http://tcll5850.hostoi.com/myc/help.htm
you can read up on it here:
http://binarymage.com/forum/viewtopic.php?f=21&t=2418&start=40 (more towards the end)

yeh, just more slowdown... IK... this is a small project though, so it shouldn't take me too long
(unlike some of the plugins for my forum) >3<

 

[Tcll]

ok, so I'll be making a few changes to 3.0's functions for bones following these guidlines:
http://www.blender.org/documentation/blender_python_api_2_71_release/bge.types.BL_ArmatureBone.html

- bones have both a head and an optional tail with individually set coords
(if no tail is specified, by default it'll use the parent's head, or +Y/+Z (major axis) if no parent or if specified otherwize)
- new functions will be specified to return the bone's matrix (both global and local positioning)
- changes to ugeBonesExist() with a boolean global attribute which will scan the current bone for children
(ugeBonesExist() scans the rig object (like before); ugeBonesExist(False) scans the current bone)


also, not to confuse people, UMC follows XYZ (W,H,D) (2D+Depth/Z-buffer) (not XZY) standards.
(this means Y is up and Z is toward you (you don't look down to view a monitor))

 

[Tcll]

just recently been modifying my HAL DAT format page, and posted my code in UMC format...
(since UMC makes python even simpler for stuff like this)
http://wiki.tockdom.com/wiki/HAL_DAT_(File_Format)#Method_2

this is mainly to help anyone following in my footsteps who wants to write their own support for the format.
(simply describing the format is one thing, I'm also displaying how it works)

 

[Tcll]

in recent news, I've switched to Linux, and have been able to test UMC with it
it almost works out of the box
it needs a little tedious work right now, but I'm working on fixing that to make it as decent as running on Windows

http://tcll5850.proboards.com/thread/143/official-universal-converter-development-thread

also, I havn't stopped Windows testing.
I intend to install a VM when I can get around to it, and reinstall XP64 to test with.


the reason I switched to linux is I planned it for the next Windows death
this time it wasn't XP's fault, but yet another dieing HDD.
the OS still works enough for me to boot, but it freezes when the HDD cuts it's power.

let this be another time Windows died with no viruses :3

so I've installed linux on my laptop HDD which shouldn't wear as much, and (being smaller) should do things much faster

 

[Tcll]

I'm gonna switch to PyGLFW before the next update...

switching will:
- get rid of the Minecraft-like window resize (everything will resize as it did with freeglut)
- hopefully fix the screen resolution issue I'm having (it's either SDL or wgl)

oh yea, I have a safe method for running it in linux (won't play with your res)

^configure a virtual desktop with winecfg in the terminal

EDIT:
btw, the loader:

UMC.sh

#!/bin/sh
wine data/Python/x86/python.exe loader.py 'x86'

^I don't have an x64 interpreter just yet
(I think wine is x86 anyways)

this works the EXACT same as UMC.exe, so you're not missing anything

 

[red skull fire]

........im confused lol XD does this thing allow you to edit a mdl0 in blender and then export back out in the correct data to be put in brawl box?

 

[Tcll]

yes exactly
however, I havn't worked on the blender script to be able to export a .blend file after importing the MDL0

after this next release (which patches the functions of 3.0a with most of the functions from 3.0), I might be able to have someone work on that.
I'll be working on 3.0 while improving the scripts for DAT and MDL0, and possibly a brute-forced DAE 141 exporter.
(I'll need an XML lib for UMC before I can fully work on a DAE script)

all of this stuff is extendable for UMC, and the extensions have been made easy to program with UMC's provided functions which should be easy to understand
I'm boasting 73% less code comparing UMC3.0 scripts to Blender27 scripts

 

[red skull fire]

ok then i think i understand what your saying lol XD cuzz this das donkey does not want to work for me way to hard to try and figure out whats going on with it ....when your finished i might just use your converter thing....also will it work for all versions of blender?

 

[Tcll]

I'd hope to support 249b along with the latest...
but with the BDNA being as picky as it is, I may implement a version selector as each blender version is really just minor changes to the blend file...

if I'm the one working on the blender script...
I designed UMC's scripting interface so everyone else could build the scripts and extend it's support

hopefully this next update, having a solid model I/O interface, will encourage that. >_>
I can't be the only one working on the support for over 150 different model formats.
to add, I have future plans for UMC which I need to work on.

EDIT:
technically I am working on that support, but at the lowest level

 

[Tcll]

sorry for the long wait...

just wanted to post some progress.
first visually successful import after porting my MDL0 script to the new format.

^ignore the dev4.5 part, I'll be removing that shortly

I'm still patching ALOT of issues that happens with an untested interface...
(I couldn't test until I had a ported script)

the new interface supports a clean and solid script I/O system for models that's easier to understand than the previous release.
for info on the new functions you'll be using, see here: http://tcll5850.proboards.com/thread/182/layout
(unfortunately I've had no help in actually building the docs, so what you see is what you get)
sorry guys, I've tried getting people to look at what I've documented in 3.0's src and forward it to the docs on my forum while I worked on UMC...
(about 7 people in my skype group to be exact)
and still the only work done is what I've been able to do...
hopefully this'll pick up in the future, but it looks like the docs aren't gonna make it by the release.

I'm not finished fully porting either... this is JUST the format interface.
the data interface has also had a few ports and improvments that need to be implemented in 3.0a:
bu_(5,'') --> bu(5)()
StructArr() --> array()

the data interface for 3.0a will NOT support pointers or auto-recursive data structures, this new support is a clean and extendable fully rebuilt interface in v3.0

EDIT:
sorry about complaining about no help...
I'm basically the only one actually managing all of this, so while being overwhelmed beyond belief, I try not to complain, but I can't help the results from it as I can only do so much.

EDIT2:
actually, instead of removing dev4.5, I think I'll rename the thing to dev5
I do intend to push a few more small devs out for improvments before I release 3.0

3.0 is still pretty far away from anything that can be called "decent" in design, but it's a huge step in the right direction.

one example, I might be able to make this thing support animation (the SES format supports it, sotechnically it kinda already does), but in playback, it'll be extremely slow.

 

[Tcll]

update: (not quite good enough for my forum yet)


the display issues here is with VIEWER.py except for Pichu's main texture which is something with the UMC-library RVL_IMG.py
(I'm not sure what format the img is being converted from, but the library is also used by my TEX0 script, so it'll be found in time)

if anyone would like to help me out who has knowldge src-wize for any various interface used in UMC, please let me know, I really need help knowledge-wize in these areas, or the interfaces given will be noob-ish and poorly structured by the release.
(this is why knowledge should NOT be sold)

 

[Tcll]

somewhat fixed MDL0 materials:

the eyes are a transparent black for some reason... since it's MDL0, hopefully it'll give me a lead on Pichu's problem above...

I can't believe there's actually a problem to begin with... I did nothing to the viewer except add SESv1.1 support which is only a few more things for meshes...
I havn't even done anything with the material index in the facepoints yet.
I've done nothing to cause this issue, it SHOULD be working just like before >3<

 

[Tcll]

fixed one issue, still got many more to go:

fixed by renaming a global var
(not sure why it affected this as it already had a unique name, I just renamed it to fit what it holds rather than what it does)

 

[Tcll]

good news for image support! =3

I've just recently talked with the developer of Pillow who has a fork of PIL.
I'm hearing Pillow is portable enough to where I can build a library out of it and support over 30 image formats as common formats.

when you specify:
ugeSetImage('imageName') # no extension.
you should be able to place imageName.png (or any supported image format) along with the model file, and see it show up in the viewer.