[WIP] Elo Ratings System

[Strong Badam]

http://www.ssbpd.com/tournament/589
Cactuar withrdrew from this tourney after 2-0'ing everyone in his pool; the TO changed it to 0-2's so that tio would seed it correctly. so he at least shouldn't have those losses but prolly should have the wins, just not addd to the bracket.

 

[Nintendude]

I fixed it. It now says he 2-0'd his whole pool. btw, I doubt this would ever be important, but it seems to be impossible to change a result to 2-1 rather than 2-0.

 

[Zivilyn Bane]

I fixed it. It now says he 2-0'd his whole pool. btw, I doubt this would ever be important, but it seems to be impossible to change a result to 2-1 rather than 2-0.

You can flip it to show the other guy won, then switch it back.

 

[Cactuar]

I'd rather just be removed from that tournament.

 

[Nintendude]

What actually happened at that tournament? Did you actually play matches? I've heard that you did and I've heard that you didn't.

 

[FoxLisk]

I fixed it. It now says he 2-0'd his whole pool. btw, I doubt this would ever be important, but it seems to be impossible to change a result to 2-1 rather than 2-0.

that's correct. i figured that it's so rare that anyone would remember it that well and care (since it's not used in anything except the display anyway), that i didn't bother implementing it. inevitably someone's lust for their record to show that 1 game they won on rainbow cruise in round 1 pools 3 years ago will prove me wrong yet again and I will make it possible but i'm holding out hope that no one will do that to me.

 

[Cactuar]

I originally thought that I didn't play any of my matches before leaving, but after I talked to chain-ace he said I played matches and then withdrew so I went with that. Point is, I withdrew. The matches I won shouldn't be counted against those players. It would be a different case if I had lost all my matches in the pool and then decided I wanted to withdraw just to not have losses.

 

[Nintendude]

Ok, thanks. I have the corrected file on my computer so once the old one is deleted I'll add it.

 

[Zivilyn Bane]

Long time, no post in here. I think it's time we light the fire back under this project.

A quick status update:

• The creator of the SSBPD (FoxLisk) has appeared to lose interest from the project, and for Melee in general.
• We do however have a database compiled of nearly 600 .tio files, with tournaments ranging all the way back to 2003.
• A programmer friend of mine (Terral) has expressed interest in helping develop the project further. He is no amateur either. He knows his stuff, and would be a great candidate to assist. He has actually already has his program extracting sets and parsing files. He does a lot of freelance work, and makes money just by cleaning up different companies web sites - making the code cleaner and run smoother. He has expressed many times to me the general amateurish nature of the current SSBPD. For example, taking 1-2 minutes just to load some pages (FoxLisk typically blaming this on hosting).
• DippnDots also sent me a PM about a month ago about possibly providing web hosting for a new project.
• FoxLisk has agreed to release the source code for the current project, if we want it
• I still have the ability to extract results directly from AllIsBrawl
• For the most part, the community was definitely supportive of this project

This essentially means we are not set back all that far. So why aren't we continuing?

The first thing we should decide is whether or not we want to
A. Have FoxLisk release the source code and bring the SSBPD back to life. [Or]
B. Start from scratch with a much more focused project, created with a more democratic team (rather than FoxLisk running the gauntlet like some kind of dictator).

Whatever we do, we need to avoid the previous situation where one person had all the power and we all basically had to bend to his will and rely on him for everything.

Thoughts?

 

[Fortress | Sveet]

I think a type of democratic team would be best. In my opinion, a team should consist of programmers (people who make changes), designers (people who decide a change should be made) and users (people who use the program, test and make suggestions). In an organic team, members can contribute in any or all of those categories, but decisions should never fall to a single person.


my $0.02

 

[Zivilyn Bane]

I agree, and from my post above I'm definitely partial to option B. I would like to start this thing over from scratch. If done right, it shouldn't take all that long due to the experience we've had thus far, and already having a sizable compendium of files.

 

[Fortress | Sveet]

I would say do both. Honestly, writing it over from scratch is a pain.



Thursday, April 06, 2000
Netscape 6.0 is finally going into its first public beta. There never was a version 5.0. The last major release, version 4.0, was released almost three years ago. Three years is an awfully long time in the Internet world. During this time, Netscape sat by, helplessly, as their market share plummeted.

It's a bit smarmy of me to criticize them for waiting so long between releases. They didn't do it on purpose, now, did they?

Well, yes. They did. They did it by making the single worst strategic mistake that any software company can make:

They decided to rewrite the code from scratch.

http://www.joelonsoftware.com/articles/fog0000000069.html



It'd be best to go through the code and make changes where needed. All the code that processes a TIO file, files player data, calculates ELO, etc is perfectly fine. Not only would it be tedious to do all that work, it would be a total waste of time.

The only reason I would suggest starting over is if the project were going to be converted to PHP. If that is the case, though, the process should consist mostly of converting C# to PHP (PHP is basically C++) which is a lot easier than taking the time to reinvent the wheel.

 

[Zivilyn Bane]

The existing code was not perfectly fine though. For one, the tournaments were not rated in chronological order. He used each event as a single unit of time. This alone is a huge mistake (you know this, you agreed when we debated this way back). He used 1,000 as a base rating instead of a provisional rating formula. The code had to have been extremely sloppy and poorly written to have all the issues it had. Even the format was absolutely terrible and completely non user friendly. Recalculation took nearly a full day to complete at the end of the DBs life. It also updated the site in real time causing massive confusion during that phase. There would likely be more work cleaning up what was already made than starting over. Then of course if this thing actually picks up, there are issues with ownership and rights. With FoxLisk out of the picture, starting over is the best idea.

I'm going to put together some documents referencing the USCF (United States Chess Federation) database and why we should use that model rather than this one.

Let's see what others think.

 

[Fortress | Sveet]

No it won't be more work to clean it up.

The recalculation speed is partially due to the language its written in and probably partially due to how the code was written. I am sure when converting over I can point out many places where the code can be moved around to be more efficient (without rewriting much of anything at all). Others will probably notice more than me.

As for the update happening in real time, that would be because foxlisk didn't do the update as a transaction lmao. Easy fix.

Adding a provisional rating would be as simple as adding 1 function. Instead of:

player.Rating = 1000;

we would put instead

player.Rating = CalculateProvisionalRating();

As for the first issue about how tournaments are ordered, i'm not exactly sure how he has it structured. It may be an easy fix or it may be a little complicated, but if thats the worst of our problems its better than rewriting the ENTIRE thing from scratch

For structuring the database like the USCF, i'd probably be ok with it. I'm not sure how much of a programming background you have, but most of the output is superficial. The data is usable and thats the hardest (most tedious) part to code. Making pages or formatting output is the easy part.

 

[Nintendude]

I'd go with option B. I felt like the most time consuming part of the project was collecting tio files and that's really come a long way, so it really shouldn't be that hard to get a ssbpd v2 up and running provided your friend is as good as you claim he is (and I have no doubt that he is). Honestly, the code behind something like ssbpd does not seem like it would be complicated at all.

 

[Tirno]

Options A and B aren't mutually exclusive. We can still ask for the source and it might come in handy, even if we decide against trying to improve it incrementally. Two suggestions about a new project:

1) Having too many people and nobody with final say could lead to cumbersome development and/or bad design by committee. Of course, one-man dictatorship is often not ideal, I just feel like there's a balance to be had here.
2) We should definitely strive to keep this open-source with open development. I feel like Smash (and probably FGC software in general) is plagued by closed projects where the community is at the mercy of the developer losing interest.

 

[Fortress | Sveet]

A & B can both be done. Tirno knows whats up.

Nintendude and Ziv seem to be good candidates for the designer [committee]

Also we should have multiple programmers. That way its much harder to hit a design or programming bottleneck

 

[Nintendude]

In terms of the design itself I don't really have any input besides not treating entire tournamentsas a unit of time and implementing provisional ratings so the ratings list isn't full of one and dones.

 

[Fortress | Sveet]

I think a lot of that comes from how tournaments are handled in TIO. Foxlisk probably just saw it there and rolled with it. Easy enough of a fix to just ignore it and handle time another way

 

[Zivilyn Bane]

I wouldn't mind being the designer - but I'm not entirely sure what that would entail. Would it just be the one with final say on how the site looks/runs? I could deal with that - and of course would keep it open for all input. Like I said before though, if I were the programmer, I would model this thing after the USCF players database. It's professional, informative, and easy to use.

I'm going to create a document that shows how the USCF site works and then I think it would be more easily understood why it's better in every way than the ssbpd.

 

[Tirno]

Alright, so this is as good a time as any to mention that I've decided to try my hand at making an application for tournament organizing. I've already written code for creating a double elimination bracket data structure and seeding it by skill level. It's written in Python for now cause Python is easy and super simple to make cross-platform, though it has some drawbacks with regards to packaging a final product/executable. I only started days ago so it has no UI yet. I'm leaning towards using the Qt framework for the GUI through the PyQt bindings. At this point, nothing is really set in stone though, and I'm open to suggestion, ideas, or concerns, especially about UI design.

 

[Fortress | Sveet]

I don't know anything specific about python. One tip i'll give is before making the GUI, go find a comfy place to sit and think about how you would want to use this program. Not how you will make it, think of the problem you want to solve and imagine an ideal tool for solving the problem.

Like for me I imagine a tool that is really quick and easy to add players, set up a bracket and REALLY easy to customize. All pools are visable at once and players can be dragged and dropped between them. Clicking the pool name opens the pool bracket for score submission, and its possible to cycle through the pools using the mouse wheel. There are keyboard shortcuts for all menus and pools can be entered by typing in the score (numpad: 2, 0, enter - moves cursor to next tile).

Goals and ideals like these are important to begin fleshing out early. Just writing that last paragraph I went through 3 revisions; if I had just gone and done without thinking too hard about it I would end up 3 months down the road noticing that it would all be better if it was different -- the only difference is its hard to scrap something 3 months were spent on, no matter how bad it is.


Food for thought. IM me again sometime man if you wanna chat

 

[Tirno]

Yeah, definitely. I didn't even start work on the UI cause I want to think it through. Jumping in blind and hoping something sensible comes out isn't going to work, especially since this will be my first GUI. I think drawing mockups first will be helpful cause it's easier to make (and throw away) a hand drawn mockup.

 

[Zivilyn Bane]

Terral has already started working a bit on this and says he could have it to the same point feature wise as the current DB in 5-10 days. Can we get him an invite to the MBR?
http://www.smashboards.com/member.php?u=132730

That's probably for the best. I can probably get something to about where ssbpd is feature-wise in 5-10 more days of actual work, since I have done what I thought was going to be hardest. Do you have a reference for how ELO should be properly calculated, and what is the solution for rankings during round robin brackets where we can't tell the order in which people played? Should I just use a before-bracket value and pretend they all happened simultaneously?

To which I linked to the formula
http://math.bu.edu/people/mg/ratings/approx/approx.html
Which actually includes the provisional ratings formula and bonus factors, as well as a sliding formula for K values. All of these things will give us much more accurate elo's with shorter time frames than the other formula. We'll see obvious impact to players like Javi, Mango, ect. With the more accurate formula we should even see players like Kels have their elo's adjusted to better reflect relative skill. He may still be 1800s. But other players will likely be higher.

FYI the formula that FoxLisk used was the ancient Elo formula from wikipedia, with the only adjustment of a sliding set value of K dependent on Elo groupings > again not nearly as accurate.

 

[Nintendude]

Sounds good. I'll contact Cactuar to sort out the MBR membership stuff - removing foxlisk and adding Terral.

 

[Zivilyn Bane]

SSBPD vs USCF Database

To start off, I just want it to be known that I'm not saying our database should completely mirror the USCF. I just believe there are several things we could learn from it, and implement it in to our database. Here are the images I made. The first one is about the "List" portion of the two, the second one is about the "Player Profile" options of the two. Here are some links for reference.
Player/Ratings Lookup
Top Players Lists
Top Player Bios
State Top Players Lists

Please excuse the amateurish presentation, these were done in Paint.
[COLLAPSE="Lists"][/COLLAPSE]

[COLLAPSE="Player Profiles"][/COLLAPSE]

And if it's going to be a real "Player Database", or even a "Competitive Melee Database" (since it's for tournaments and characters, in addition to players; we could include player profile pages to have descriptions of players. Possibly with a picture, and a paragraph or two of the player. Much like a wiki page. It could evolve in to a compendium of the history of Melee as well. It could encompass several other projects we have talked about here in the backroom but never had a means to implement. Such as the Hall of Fame. The Melee History Timeline. And the Knowledge Compendium.

 

[Fortress | Sveet]

Most of those things are just design changes. Foxlisk didn't really take any time to make the interface good. All of the underlying data (the database itself) doesn't need to be replaced. We should keep all of that.

As for the USCF's site, I think there are a lot of good ideas on there. For example, I really like the ability to sort players by all sorts of statistics. I don't like how static much of site design is, though. Are we opening discussion for site layout?

 

[Zivilyn Bane]

I think it would be more work for the programmers to try to clean up and rearrange everything FoxLisk did than it would be to build a clean, organized database the way we want it from the get go. Terral seems to believe so also.

Any other opinions?

Even if I'm wrong, we should do what's best for the project and not what's easiest anyways.

 

[Fortress | Sveet]

Get the source code anyways. If you build your own from, at least you have reference material. I honestly think you will want to save at least the foundation. I'm talking the objects that describe the properties of a tournament, a player, a game, a set, processing a TIO file, etc. Change the contents to your desires and use them however you want, but use them. To reinvent the wheel would be a waste of time, and trust me, writing proper get/set functions for every attribute is really boring.

 

[Strong Badam]

Yeah, there's no reason not to get it. Having it is strictly better than not having it.

 

[Zivilyn Bane]

Today I sent Terral a sample size of .tio files to test his sw on.

Why isn't he in the MBR yet?

 

[Nintendude]

Sorry, I just haven't had time to contact Cactuar yet. Will do it today or tomorrow. Is his smashboards alias just "Terral"?

 

[Zivilyn Bane]

Yeah. I linked to his profile earlier.

 

[Nintendude]

I spoke to Cactuar in person a couple days ago at a tourney. He had a lot of praise for what we've accomplished with this project so far. He said that something came up (due to MLG? I don't quite remember) with MBR membership which is why there's been a delay on getting Terral in here. He's working on sorting it out.

 

[Fortress | Sveet]

I had a similar problem getting armada and others into the mbr for the tier list project.

 

[Isatis]

If Cactuar is okay with it (sounds like he is based on the conversation), I'll add him now.

 

[Terral]

Hey, thanks for adding me into this. It seems I just happened to check the boards right as I got access.

Right this moment I have the basic tournament database functional, minus ratings (you can register, upload .tio tournaments, view player match history, and search for/view tournaments and brackets). It also divides by game (Brawl, Melee, PM, Street Fighter, or whatever) and singles/doubles (could be extended to "with items", "random characters", "low tier", etc. for fun), just to try that out. It's very basic, but I'm only six days of work in. Everything's in Python using mod_wsgi on the server side, and the database is currently handled using sqlite, for any curious programmers. Brackets are implemented in JavaScript/Canvas for bandwidth/performance/usability reasons. I'm pretty happy with what I have so far, and I'm looking forward to sharing it once it's more complete and stable.

 

[Fortress | Sveet]

Sounds like you have a lot done, Terral. Good job!

 

[Juggleguy]

I'm a couple weeks late here, but I definitely think it's better to start from a clean slate rather than try to fix FoxLisk's previous work. We have clear goals in mind this time, whereas the old SSBPD seemed to be something that added more and more features as it went along, so I can't imagine the code would have been very elegant.

Also, I want to contribute to this project however I can. I know how important it is to document Smash history and (possibly) create new tools for TOing, and this could be huge for the Smash community. Let me know what I can help with.

 

[Terral]

Ziv suggested I post an update about the project. There was a while where I didn't have the time or initiative to work on it, but I picked up where I left off the other day.

Rating infrastructure is finished, though it's not using the final rating algorithm. Using the Wikipedia Elo calculation as a placeholder with each match as a unit of time, it can get done rating the results of 4700 matches (from 50 tournaments) in a third of a second (running on an ancient 1.8GHz single core laptop), covering Melee, Brawl, Smash 64, and a couple random other games that we did at Pathways tournaments, plus doubles. It's probably not worth mentioning, but matches from different games and singles/doubles don't affect ratings for other games. This should scale linearly with more data, and I'm not expecting a significant increase in calculation time using a more complex rating algorithm. It's possible to look at a player's match history and see their rating change on a match-by-match basis, and see the ratings of the people they played in those matches at the time.

I have advanced search infrastructure in place, and it's currently possible to search by name, game/type, and Elo ranges. So it's possible to find all tournaments that feature Brawl, or all Melee doubles teams, or all teams containing a certain player, or all players between x and y ratings, or above/blow z rating.

Currently, I'm mostly working on adding editing and detail functionality (for example, renaming tournaments/events/players, adding regions, mains, videos of matches, merging players) and increasing overall stability. If there are any features that I should make note to include, everyone can feel free to suggest them.

For permanent hosting, my plan is to put it on Google App Engine, and hopefully that transition will go smoothly. I've been working with GAE in mind, but I also don't have past experience with the platform. It should be fairly cheap, and possibly free, so it seems like the best choice.