[WIP] Elo Ratings System

[Fortress | Sveet]

nintendude, i don't think it is that difficult to implement the more complex one other than the additional math (correct me if i'm wrong). In any case, I think we should do this right as I don't think there will be a time in the future to correct or modify the system.

 

[Skler]

I agree with Sveet on this one, all you need to do is make sure it crawls through the bracket/pools in order and if we already have the math figured out it's no problem.

I'd figure it goes something like...

The function for elo calculation of a game will be EloCalc(p1, p2)
It can save the CHANGE in values of p1 and p2's Elo rankings.

The function for changing the Elo values can be EloChange(all players)
It saves the new Elo values for everyone and then resets the variables used in EloCalc to 0.

Just run EloCalc, having it edit a variable for each player, and then run EloChange when it runs into a special character.

The controlling function would just need to be something that calls EloCalc (p1, p2) until it runs into a special character, which is when it runs EloChange. The best part about the special character method is you can use it to separate bracket matches, different groups of pools, and it lets you easily include "other ranked matches (MMs, etc)" in the file. All you need to do is add it in with the special character.

Once the function hits the end of the text file it ends.

tl;dr: Crawling through a tournament (even with multiple pools that you want to separate) is really easy to implement.

 

[hectohertz]

k so elo-update algorithm aside:

--Do we have anyone who's going to be the "admin" of the database (approving tournament results and granting TO's permission to submit)

--Do we have anyone who's willing to host the DB?

 

[Nintendude]

Point taken on the algorithm.

Do you need to know that so you can set up user accounts (or something along those lines)? There
definitely has to be a group of admins (not one person, but not many either) to handle what you mentioned. Is there a way of keeping it simple (no SQL), like you wanted to, if you need user accounts?

Is there any way that Smashboards can host this? I think they were using their webspace to host some kind of Brawl matchmaking system back in the day, but I have no idea how that worked.

 

[hectohertz]

yeah we can keep it really simple (no SQL) and still have user accounts

smashboards could totally host this (the traffic-load would be extremely light), we just need to talk to someone whos a webmaster/moderator/has-privileges to do so. also we should figure out if this would be through vBulletin, or if we'd be writing some separate stuff (vBulletin is written in PHP, so smashboards can definitely handle php code, I could whip up some simple PHP to allow someone to login, approve/disapprove touraments, and a little "reciever" php script that would recieve a .tio file and put it somewhere for approval, again all of this is assuming the vBulletin doesn't have mechanisms to do any of this it very well may in which case we should use them, no need to duplicate functionality)

tl;dr: who's the webmaster here and how do they want us to do things

 

[Cactuar]

We should probably talk it over with Bionic.

 

[Nintendude]

Can you talk to Bionic for us?

 

[Fortress | Sveet]

We could just send him a PM. I don't think its a big deal, he seems open minded about site additions. Does someone want to right up a letter and maybe post it here for some editing/critique before we send it out? I don't really consider myself "in the loop" or anything so i don't know if you would consider something i would produce.

 

[Cactuar]

I'll contact Bionic and talk it over with him.

 

[hectohertz]

thanks cactuar.


so it seems to me this project is really two pieces:

- a server-side ELO db that keeps track of ELO ratings and updates them when new tournaments have been approved. It would include a submit page, as well as a page for the "admins" to log in and review tournament results. it would also allow anyone to download the current .elo file. ideally we'd want to use as much of the code that's already been written for ELO stuff (I think Cobalt wrote it, is he onboard with this project and if not we should recruit him)

-a tio plugin that would support fetching the elo file from the server (easy cause its just a flat file), parsing it, using it to seed brackets, and then submitting the resulting .tio file to the server when the tournament is over


since we have two lead programmers and two "projects" i'd like to suggest that i'd lead the server-side stuff (since that's where i'm more experienced), and tirno would lead the tio plugin stuff. tirno feel free to tell me if you're not cool with that

 

[Nintendude]

Can I get an update on the progress of things?
Also, it might be useful to set some relaxed deadlines in order to keep the project moving. What do you guys think you can have done by the end of July?

 

[hectohertz]

I'm waiting on cactuar/bionic so I can start working on the online db. Has anyone contacted the programmer of the Elo rankings thing that's on the public boards? We should bring him in.

 

[Isatis]

Someone linked me here, so I assume it's OK to discuss it here.

I also wanna say it's totally possible. But as it stands our site is in limbo at the moment... we have no way of really uploading files as our system admin (Agora) is busy, but we can add custom pages (which include PHP code) in vBulletin and inserting/updating/deleting database-specific stuff. Plus the fact that we may overhaul the forum software in favor of something else later this summer, if not later this year... which I hope won't interfere with the ELO project itself.

Also who're the PHP/MySQL programmers and who're the .NET programmers you have for this?

 

[hectohertz]

I suppose I'm the PHP programmer and tirno is the .NET programmer, although I have a descent amount of experience with .NET

just to clarify, would it be possible to insert some custom pages written in PHP, or would we need Agora for that?


also, has anyone tried to bring cobalt in on this project yet?

 

[Tirno]

Yeah some deadlines might do us some good. I think the first thing we should aim for is getting the plugin to submit results. Since nobody's going to be using the rankings to seed for quite some time, accumulating a database of results should be our first priority.

 

[Isatis]

Standalone files need Agora. Custom pages (like the Connect one we have, which is made through vBulletin and not as a separate file) we can do.

 

[Nintendude]

Cobalt apparently hasn't logged in in over 2 months...

 

[hectohertz]

Alright, well i went ahead and messaged the guy who wrote the brawl ELO software, hopefully he'll login soon.

Basically what I'd ideally like to do is use one of their pieces of software on server and then write some php around it to control it.

@Bionic: Do you know if php scripts in custom pages are allowed to use exec(). And if so, is there a jvm installed?

If not, we're going to be talking about either porting them to PHP, or just writing the PHP backside from scratch.

 

[hectohertz]

okay, so i just talked to UTDZac, the guy who wrote the exisitng EloCalc that brawl uses for rankings. And he'd like to help on the project. Can someone give him access to the backroom so that he can help?

edit: and i just talked to eet, who had posted in the public thread a while ago that he could help program. could he get access?

oh yeah and Zivilyn Bane just offered to help us do some data-mining

 

[Nintendude]

Before we just start letting all these people in I need to know what kind of role they will have in the project. Things can get really confusing if you have too many people collaborating on programming the same thing. For now I'll talk to UTDZac since he has experience working on an extremely similar project. Did he work on the system that they currently use or some other system? Cause the one they currently use is not Elo - it doesn't work with relative skill levels. I think their system places too much emphasis on things that don't matter (like actual tournament placing, pot, etc.) but that's for another discussion.

Also, I think we can't just go and collect tons of tio files from people. We have to set some guidelines for what we will accept. To start, I think everything should be from the year 2011. All of the entrants must be entered with their actual tag and not some kind of joke or alternate tag or else it may throw off some results. Finally, I'm not sure how to get the regional stuff to work if the system will use old results to get started.

 

[hectohertz]

too many chefs do spoil the cake. talk to UTDZac and figure out exactly what he's using.

i don't think we need to collect tons of random tio files, but i think it be awesome if someone could go through and make .tio files out of all the big nationals. there's no reason we have to use only stuff from 2011.

finally, im not too sure about the whole region id thing. people need to remember their id any that's a pain in the ***. look what happened with WII friend codes vs Xbox gamertags. If we have a comittee that's approving .tio files, they can make sure that all the spellings match previous spellings.
also we can include location in the DB if we want to be able to do regional rankings


anyway, eet has experience in c#, so I was thining he could help tirno out on the plugin. since utdzac wrote the elocalc i was thinking he could work with me on the server-side thing.


@nintendude: I know you're heading up this project, much respect. I have a lot of experience coordinating programmers, perhaps I could be lead programmer / project manager. Not trying to steal your thunder, its just often useful to have a programmer in charge of the other programmers.

 

[Nintendude]

If we want to accept things from before 2011, we need some kind of cutoff. I definitely think stuff from before 2010 is too old.

The reason I like ID numbers is that it just seems like the most professional and organized way to do it. Usernames and gamertags can get really confusing, but numbers are numbers. Everything in society, from social security, to credit cards, to drivers licenses, to bank accounts, are kept track of using ID numbers (in addition to names). Ideally, a robust system wouldn't require people to remember their ID number. If we could have a system capable of finding ID numbers based on gamertags, that would be great. Or imagine if, when registering, a person could say "I'm so-and-so," and then the TO can find and click a name in the database rather than having to type it himself.

I don't think the Xbox Live / Wii Friend Code thing is really an accurate comparison. The only handling of accounts by people are when you are trying to add people as friends and stuff. Everything else in XBL is taken care of by XBL, so your account can't ever get mixed up by the system. When you have TOs entering information for hundreds of people, you need a data entry method that minimizes errors.

If you want to manage the progamming that's perfectly fine. I basically see myself as the adviser just trying to make this a reality.

Before I go ahead and invite eet to the team, can tirno comment on how he feels with working with him?

 

[hectohertz]

Can you explain why we'd need a cutoff? Just curious.

And yeah I like the ID number system as you described it. We could definitely integrate something into the plugin for that.


And awesome. We couldn't do this without you man .

Yeah let's get tirno's opinion.

And we need to hear back from UTDZac about his implimentation.
And we need to hear back from Bionic.


glad we've got you to coordinate this

 

[Nintendude]

I think we need a cutoff because pumping the ratings system with old results wouldn't really produce accurate results imo. There's definitely a point where results are so old that they don't indicate any useful information. Having a bunch of tournaments from the era where Azen consistently got top-3 definitely inflates his rating to a lot higher than it should be.

I suppose you could also argue that any and all results are relevant though, and that inaccurate ratings would quickly be fixed by new tournament results. However, you run into the issue of inactive players. Ken would have a really high rating if you put in all the MLG brackets, but Ken is not very relevant anymore, and his really high rating would not mean the same thing as PP's high rating.

 

[hectohertz]

I'm sort of just playing devil's advocate here, but couldn't we introduce a ratings decay to fix that? Or maybe just count more recent results a lot stronger than old ones. So Azen got 4-stocked by M2K at Zenith, and when Ken came back out I'm pretty sure wobbles and some other guys beat him...

Also Zivilyn Bane says that he's an expert on chess ELO rankings, and offers to help with K-Values and equations. He also volunteered to do some data mining.

 

[Nintendude]

I kind of want to get Zivilyn Bane in here just so he can participate in the discussion, cause I feel really bad excluding him when he's really interested in it. Not sure how that would fly with MBR staff though...

I'm not really a fan of ratings decay, but I like the idea of weighing old events less than new ones. Like, what if we just halved all of the K-factors used for events before some certain cutoff date? I'm just uncertain of the implications of rating a player who never really voluntarily entered an event with the understanding that he'd be entered into a ratings database.

 

[hectohertz]

I kinda do too, see if cactuar will let him. it be good to have someone who knows ELO really well write up an ELO spec, or explain what changes we should make to UTDZac's, before we code it

and yeah I like that with da k-factors

 

[Isatis]

@Bionic: Do you know if php scripts in custom pages are allowed to use exec(). And if so, is there a jvm installed?

If not, we're going to be talking about either porting them to PHP, or just writing the PHP backside from scratch.

Yes, exec() is allowed, but restricted to our site... which will be fine considering we won't need to access anything other than SWF anyhow. It's used in the Albums/Stage Browser without a problem. But I don't think JVM is installed unfortunately. (Does PHP even support JVM?)

 

[hectohertz]

Well what I was hoping to do is since UTDZac already wrote an ELOCalc in java, was to run it via the command-line like:

exec("java -jar ./elocalc.jar")

which would require having java installed, and having execute permissions for the file elocalc.jar

can you check and see if either of those are available. because even if there's no java, if exec() let's you run arbitrary binaries, we can compile the java code to a native binary and run it like a CGI script

 

[Isatis]

Execute permissions are not a problem, but we don't have Java or its binaries... and well, since that'd require us uploading a file to the server... yeaaaaaaaaaah =/

 

[hectohertz]

that's fine, we can use gcj (http://gcc.gnu.org/java/) to compile it to a native binary. if we did that, could you put it on the server (with execute permissions)?

 

[Isatis]

It's actually getting it onto the server that's going to be the problem... unless you're willing to wait a few weeks (or a month) for it to be uploaded. Everything else shouldn't be a problem.

 

[hectohertz]

alright, if you're sure about that, we can work on refining UTDZac's elo calc into a lean mean elo-crunching db-maintaining machine. i mean this project is going to take some time to complete, maybe we aim to have something ready in a month or whenever this sysadmin is back. Seems like a natural time-target, right?

 

[Isatis]

It's not that, it's just Agora is usually slow on things because they're busy developing stuff for MLG and I usually have to email them constantly to remind them to upload things. Took a month for them to upload something related to premium subscriptions =/

but yeah, set some timetables and goals, I'll try and hopefully have the java binary deployed onto the server by... whenever I guess

 

[hectohertz]

alright, awesome man, thanks.

so we need to hear back from UTDZac, nintendude, are you on that?

 

[hectohertz]

also

@Bionic: can you help me out with some details about custom php pages. how do we upload them. what libraries/API's are we using? Is there a template? What access do we have to the file system? To an SQL database? Sorry if this is too many questions

 

[Nintendude]

UTDZac said he's interested in helping and hopefully he'll be joining us soon.

 

[hectohertz]

alright sounds good, looking forward to having him in. should i pm him or just wait till he's admitted?

 

[Nintendude]

I think we're either waiting on Cactuar to let him in or for him to apply to the BR usergroup. Either way it'll hopefully be any day now.

 

[hectohertz]

double post