[WIP] Elo Ratings System

[Isatis]

For permanent hosting, my plan is to put it on Google App Engine, and hopefully that transition will go smoothly. I've been working with GAE in mind, but I also don't have past experience with the platform. It should be fairly cheap, and possibly free, so it seems like the best choice.

I'd offer hosting on Smashboards as well, although I don't have access anymore and not sure if you're sticking to Java or doing it in another language.

 

[Terral]

The original SSBPD was made using C# and ASP.Net, if I'm not mistaken. The one I'm working on is using Python via mod_wsgi. I know for sure that Google App Engine supports mod_wsgi and it would be fairly easy for me to manage it there, based on what I've read. Of course, Smash Boards hosting would also be good, but as long as it works on Google App Engine without much hassle it's probably just easier to go that route.

 

[Nintendude]

Wow, that's awesome Terral. Good work.

 

[Zankoku]

Hey, I didn't realize you guys already had something going, I wrote a program to do rankings for my own purposes (ranking in the local FGC)
http://www.smashboards.com/showthread.php?t=328804
It runs on Glicko, my source is kinda messy but if you want to check it out I can upload it. It's written in C# on VS2012 Express.

 

[Zankoku]

Uh... Hello? Anyone there?

 

[Terral]

Sorry, I had thought I'd made a response to you. Since we're using Elo, source code for a Glicko implementation probably wouldn't be of any use, especially since the ratings portion is basically done and only needs to be modified to use a more complete version of the Elo algorithm. There is technically support in my code to handle more than one rating algorithm, but the benefit probably wouldn't be worth the effort. Thanks for the offer, though.

 

[Nintendude]

Apparently neal is still alive:
http://www.smashboards.com/showthread.php?p=15017834#post15017834

He confirmed that the whole custom plugin thing from way back when is actually going to be a reality, so the concept of reprogramming tio to use ratings to seed should be possible.

 

[Terral]

I know a decent amount of C# from classes, so assuming I get the time for it I should be able to make a tio plugin happen. Also, sorry about the lack of updates, I haven't allocated any time toward the player database in a while. I was actually planning on readying the website for beta this past week, but as Murphy predicted my computer ended up having some major problems and has been out of commission until I'm able to repair it.

 

[Terral]

I wanted to get some opinions on some of the tools I'm making/have made for moderators, since I don't know anything of the old site's moderator tools and wasn't sure I felt like learning about them. I'm still polishing/implementing these features, so changing them at this point isn't going to be a big problem. Even though this isn't a comprehensive list, if there are necessary features not listed here, request them.

To deal with multiple games and the potential for people to make typos, there is a way to merge or rename entire games. So if someone insists on marking their events as Melee, all events using that name can be moved over to Super Smash Bros. Melee at once by a Moderator. The site will also try to guess games if it isn't listed in an uploaded file: first, it will guess the most common game in a tournament. If none of them have games assigned, it will fill in Super Smash Bros. Melee since we're initially starting this off as Melee-focused, but I'll probably disable that later.

I'm not sure what kinds of tools the current site has for changing player names, but right now I give moderators the ability to merge or rename players (on a per-game basis, naturally) from the website. There will also be the option to change players per-event, or remove players from a tournament if they didn't actually participate.

In an attempt to make it easy on moderators processing tournaments (not that I know what all is involved in the current website), I'm allowing confirmed users (confirmed through another confirmed user or moderator) to upload tournaments and have them automatically accessible on the website, but not rated. The uploader can then request that the tournament or individual events be marked as rated. Uploaders will also be able to delete their own tournaments/events or have the tournament not appear in search results (this shouldn't be considered a privacy setting, just convenience).

My idea for making some of these systems more friendly is a flagging system. General users are able to flag a player name, character, etc. as incorrect and add a comment, then moderators can view flags per-type and respond to them. This is also the system that would be used to confirm rating requests for tournaments and anything else similar, like suggesting a YouTube link.


Also, does anyone know if we could get some of current site's database so that we can migrate region/character data over? That would probably save a lot of work. I assume that people (Ziv?) still have their .tio files, so that shouldn't be a big concern, but it wouldn't hurt to get those as well.

Finally, what should I do about multiple players having identical names? Region suffix as used on the current database is definitely easiest, and since you can rename players after tournaments are uploaded this is probably not a big deal. It would also be possible to have players with the same name in different regions if it's something that people really want to see happen.

 

[Nintendude]

To deal with multiple games and the potential for people to make typos, there is a way to merge or rename entire games. So if someone insists on marking their events as Melee, all events using that name can be moved over to Super Smash Bros. Melee at once by a Moderator. The site will also try to guess games if it isn't listed in an uploaded file: first, it will guess the most common game in a tournament. If none of them have games assigned, it will fill in Super Smash Bros. Melee since we're initially starting this off as Melee-focused, but I'll probably disable that later.

This sounds great to me.

I'm not sure what kinds of tools the current site has for changing player names, but right now I give moderators the ability to merge or rename players (on a per-game basis, naturally) from the website. There will also be the option to change players per-event, or remove players from a tournament if they didn't actually participate.

Again, this sounds fine.

In an attempt to make it easy on moderators processing tournaments (not that I know what all is involved in the current website), I'm allowing confirmed users (confirmed through another confirmed user or moderator) to upload tournaments and have them automatically accessible on the website, but not rated. The uploader can then request that the tournament or individual events be marked as rated. Uploaders will also be able to delete their own tournaments/events or have the tournament not appear in search results (this shouldn't be considered a privacy setting, just convenience).

I'm a little hesitant about giving confirmed users the ability to delete and hide tournaments. Wouldn't it seem odd if you can't see someone's full tournament history when viewing their profile?

My idea for making some of these systems more friendly is a flagging system. General users are able to flag a player name, character, etc. as incorrect and add a comment, then moderators can view flags per-type and respond to them. This is also the system that would be used to confirm rating requests for tournaments and anything else similar, like suggesting a YouTube link.

I'm pretty sure this is similar to what was done in the old SSBPD but minus the comment system, so this should be good.

Finally, what should I do about multiple players having identical names? Region suffix as used on the current database is definitely easiest, and since you can rename players after tournaments are uploaded this is probably not a big deal. It would also be possible to have players with the same name in different regions if it's something that people really want to see happen.

I think being able to have multiple names with different regions is fine (and better than using stuff like Greg (NY)). In fact, I'd imagine the list of names to look something like this:

Mango - CA - 2000
Dr. Peepee - NC - 1900
Ky - CA - 1600
Ky - NY - 1500

I used Ky as an example since that's definitely a duplicate name in the community. The way I envision it, gamertags would be hyperlinks to profiles and the region text would be a light gray.

I think that duplicate names (particularly generic names like John) won't be an issue once we properly tackle provisional ratings. Players with only 1 tournament under their belt shouldn't appear in the master rankings database. A big problem with the original SSBPD was it was clogged with a million one-and-dones that skewed the ratings, since they were all random nobodies. There needs to be a way to store data for them and then retrieve it later when they have participated in a minimum threshold of tournaments (I'd say 3 is a reasonable number). Among players that have entered multiple tournaments, there are very very few duplicate names to worry about.

btw, will player tags be able to be tied to a master profile showing their ratings and tournament history across a variety of games? It would be cool for me to have one main "Nintendude" page with stats for Melee, 64, and Brawl.

 

[Nintendude]

Oh, also, I want to make sure you are aware that TioPro has been released and replaced the old tio. I hope this doesn't break any of the functionality. On a similar note, challonge is becoming really popular these days. How hard would it be to make the database capable of processing results from challonge?

 

[Juggleguy]

I strongly prefer the old tio over the new and not-really-improved TioPro. Will the site be able to take either filetype as input?

 

[Strong Badam]

I don't see why it wouldn't.

 

[Nintendude]

I think it depends on how the old tio and new tio output to an XML file. Hopefully it's identical in which case there would be no issues presumably.

 

[Zankoku]

Oh, also, I want to make sure you are aware that TioPro has been released and replaced the old tio. I hope this doesn't break any of the functionality. On a similar note, challonge is becoming really popular these days. How hard would it be to make the database capable of processing results from challonge?

I strongly prefer the old tio over the new and not-really-improved TioPro. Will the site be able to take either filetype as input?

I think it depends on how the old tio and new tio output to an XML file. Hopefully it's identical in which case there would be no issues presumably.

The little program I wrote that imports tio files and processes them into individual matches had no issue reading older tio files and tiopro-made tio files, with no changes made to the code, so as far as things that matter I believe there are no changes in the XML.

I don't think Challonge exports results or anything in any meaningful way, the final results and the bracket module they've got on-site is all you can take from it.

 

[Isatis]

They have an API to fetch tournaments, at least from what it looked like in the examples.

 

[Terral]

I'm a little hesitant about giving confirmed users the ability to delete and hide tournaments. Wouldn't it seem odd if you can't see someone's full tournament history when viewing their profile?

Mostly, I want to give people who upload stuff control over the tournaments they've uploaded. If they made a mistake with a bracket or uploaded the wrong file, then I'd prefer that they be able to deal with that problem without needing to contact a moderator. This could be changed if it isn't a popular design choice, though.

I think being able to have multiple names with different regions is fine (and better than using stuff like Greg (NY)). In fact, I'd imagine the list of names to look something like this:

Mango - CA - 2000
Dr. Peepee - NC - 1900
Ky - CA - 1600
Ky - NY - 1500

I used Ky as an example since that's definitely a duplicate name in the community. The way I envision it, gamertags would be hyperlinks to profiles and the region text would be a light gray.

I think that duplicate names (particularly generic names like John) won't be an issue once we properly tackle provisional ratings. Players with only 1 tournament under their belt shouldn't appear in the master rankings database. A big problem with the original SSBPD was it was clogged with a million one-and-dones that skewed the ratings, since they were all random nobodies. There needs to be a way to store data for them and then retrieve it later when they have participated in a minimum threshold of tournaments (I'd say 3 is a reasonable number). Among players that have entered multiple tournaments, there are very very few duplicate names to worry about.

The main complication with duplicate names is uploading tournament files. While it's feasible to let the uploader choose a local region that the database could use to guess, it still isn't a great option. A nice (and annoying) thing about the current region suffix system is that you can add it in the tournament file without things needing to be double-checked. This could probably be addressed with a Tio plugin once we have that ability.

btw, will player tags be able to be tied to a master profile showing their ratings and tournament history across a variety of games? It would be cool for me to have one main "Nintendude" page with stats for Melee, 64, and Brawl.

In a way. For example, if you search for Terral in the database without specifying a game the search result looks like this:

Terral (Super Smash Bros. Melee (1097), Super Smash Bros. Brawl (1418), Super Street Fighter IV (1183), Super Smash Bros. (1184), Project M (1200))
Terral + Terry (Super Smash Bros. Melee - Teams (1183))
Terral + Ugvts (Super Smash Bros. Melee - Teams (1152), Super Smash Bros. Brawl - Teams (1094))

Clicking on a name sends you to a page with each game listed after the previous. I may work on making it look nicer at some point. As far as the database is concerned, the different name+game pairs are completely separate entities.


As for Challonge, it's not something I'd even heard of before a few weeks ago, but if they do have an API then it shouldn't be too difficult.

 

[Zivilyn Bane]

I believe the SSBPD had trouble when we first had uploads come from .tio files made in TioPro. FoxLisk was able to fix it pretty easily though, so I doubt Terral will have a problem with that.

As far as what Nintendude was saying about the one-and-dones. The cutoff should be for a specified number of rated games, not tournaments. I'm thinking 12.

 

[Nintendude]

The main complication with duplicate names is uploading tournament files. While it's feasible to let the uploader choose a local region that the database could use to guess, it still isn't a great option. A nice (and annoying) thing about the current region suffix system is that you can add it in the tournament file without things needing to be double-checked. This could probably be addressed with a Tio plugin once we have that ability.

I think ideally, the first instance of a duplicate name would have to be created manually, and then when a new tournament is uploaded containing that name, it prompts the user to pick the correct name/region to assign those games to. So, for example, let's say Mike is in the database. Then another Mike gets uploaded and he is manually separated into his own profile, so you have:

Mike - NY
Mike - NJ

Then from now on, any time there's an upload containing "Mike," upon uploading the file, it'll ask you if this Mike is from NJ or NY (or a new entry entirely). Does that make sense?

Everything else in your post seems good.

As far as what Nintendude was saying about the one-and-dones. The cutoff should be for a specified number of rated games, not tournaments. I'm thinking 12.

Why 12? Is there some sort of statistic behind that?

 

[Terral]

I think ideally, the first instance of a duplicate name would have to be created manually, and then when a new tournament is uploaded containing that name, it prompts the user to pick the correct name/region to assign those games to. So, for example, let's say Mike is in the database. Then another Mike gets uploaded and he is manually separated into his own profile, so you have:

Mike - NY
Mike - NJ

Then from now on, any time there's an upload containing "Mike," upon uploading the file, it'll ask you if this Mike is from NJ or NY (or a new entry entirely). Does that make sense?

This is still basically the same issue/solution, though. Ideally, there won't be many people with duplicate names, so this is probably just what I'll go with.

 

[Zivilyn Bane]

I chose 12 because it seems like a good fit. If you're awful and go 0-2 at every tournament, it would take 6 tournaments to be on the list, which shows some dedication. Or if there are pools it could only take 2-3. Also if you're insanely good, like say Javi, 12 is enough to get you're rating pretty established, but not too much so that you have to enter several tournies before you get in.

 

[Nintendude]

6 tourneys is a lot imo, especially because some regions only have tourneys once every 2 or 3 months. I'd reduce the threshold number of sets to 8. It's high enough to weed out the one-and-dones but makes it more likely that everyone who cares gets to have arating.

 

[Terral]

What list are we filtering, exactly?

 

[Strong Badam]

Erm, wouldn't the easiest way to prevent an excess of "One-and-done"s be to require at least two tournaments?

 

[Nintendude]

Well even if one and dones wasn't an issue, we still need a proper way to provisionally rate people, so it's not as straightforward as making the threshold 2 tournaments (unless we make that the provisional rating criteria).

 

[Strong Badam]

Provisional ratings and 1-and-dones can be separate thresholds. Ziv, what threshold does USCF use for provisional ratings?

 

[Zankoku]

http://archive.uschess.org/ratings/

Every USCF member will have a provisional rating published after their first four games, and an established rating published after their first 25 games.

 

[Strong Badam]

I think that'd be acceptable for us, or perhaps lower the established rating to 15ish. It's a bit hard to discern because some regions run pools more consistently, while others go with straight bracket.

 

[Terral]

So what is the problem with one-and-dones that we are worried about, in regards to the database/website? Not including provisional ratings, which is mostly a separate issue.

 

[Fortress | Sveet]

Hey hows this going?

Would you mind if I took a look at your progress so far?

 

[Terral]

It hasn't really been going much, but you can see what's done. I realized I had done one thing particularly stupidly, came up with a way to fix it, put it off for a while, and then participated in Ludum Dare. I'll PM you a link in case you want to check out the current work-in-progress looks-like-garbage version. A couple moderator interfaces are a little glitchy, but the rest of the features that are present should basically work. I would just post the link here, but it's still hosted on my home server and I don't really want more traffic going to that than needs to be.

 

[Juggleguy]

Is it possible to work with AlphaZealot to get this implemented onto Smashboards along with the new upcoming upgrades? I feel like this is a huge need for the current disorganization of the site. It seems like half of the links to results threads, bracket images, etc. are broken everywhere I look, and I usually have to resort to Google to just get results from past nationals and such. A separate site isn't bad, but for this to truly take off, it needs Smashboards IMO.

 

[Marc]

I can't speak for AZ, but it's definitely something that can be discussed. Right now there's obviously a lot going on with the upcoming server migration, but the more specific a plan you can lay out, the easier it is for the Senate to look into it.

 

[Nintendude]

A lot of the technical details of this are way beyond me, but it would be cool if smashboards could host ssbpd and then tie in the ratings for the Smash games into people's profiles. The only problem with that is that not everyone has an account and some people's accounts are way different than the tag they enter with. Also we'd be leaving out AiB in the event that Brawl wants to jump ship. Having it as an independent website is not necessarily a bad thing though. It would be easier for communities (and not just Smash communities) to interact with it. Smashboards could still pull data from the website even if it is separate.

 

[Isatis]

If Smashboards will host it, the new server won't have C# or Java as it is going to be a CentOS Linux system (iirc), so nearly everything Terrell has done so far will probably be useless. The ELO program itself will have to be written using PHP/MySQL in a forum framework.

 

[Strong Badam]

unrelated but smashboards/tio needs a plugin to auto-post a results thread from tio

 

[Fortress | Sveet]

I could be wrong, but I think terral is using php. Foxlisk was the one using c#.

I'm working on some design plans right now. when I'm done with them, I want to reach out to alphazealot to see if we can all work towards a common goal. Bionic, is there a better way to reach him than PMs? I assume he is a lot busier now

 

[Terral]

The ssbpd was written in C#. My code is Python 2.7/mod_wsgi (Apache), using sqlite as the database and the file system to store uploaded images and backups. Porting the sqlite code to another SQL server wouldn't be too difficult, it was mostly chosen for convenience.

I looked around a bit to try to find a good host, but most hosting is far over my budget. For the affordable options, Google App Engine would require a complete re-write of all of the database and file storage code, and Amazon Web Services is a bit confusing to figure pricing/features for. My home server would be an option, but I'm only on a residential connection so it's probably against some terms of service I agreed to with my ISP.

 

[Isatis]

I can see about possibly installing python on http://ssbpd.smashboards.com once we transfer to the new server. The new server is probably running nginx instead of Apache, so I'll look up how to hook nginx and Python.

I'd be greatful if you could do MySQL instead, since combining that with the forums would be 100x easier and with this you can fetch Smashboards user data to combine usernames/user ID's to the SSBPD names.

 

[Strong Badam]

So annoying that SQL is so common, because it kinda sucks