Elo rankings in Smash?

[Frames]

I've always been curious about something; many other games that have competitive aspects (Starcraft, Magic the Gathering, Halo 2), rely on a variant of some kind of the Elo ranking system, a system designed to calculate relative skill between people in a 2 player game.

Smash has always been highly competitive, so I'm just curious why either this or some variant of this ranking has never been implemented in competitive play. For the people who were with MLG when they sponsored melee, tell me, did they have any type of formal ranking system like this? I think if so many other games and areas can manage to use this effectively, then why not smash, especially with such a huge fanbase behind it.

 

[EnigmaticCam]

I actually attempted to do this a long time ago.

http://www.smashboards.com/showthread.php?t=89648

The only problem is that Elo ratings don't seem to work well with a standard double-elimination tournament. They work much better in chess because most chess tournaments are round-robin, which lends to much more accurate ratings.

 

[Dastrn]

actually most chess tournaments are Swiss system, not round robin.

 

[Frames]

@ EnigmaticCam - I could see how there could be potential hazards with elimination format, but I've been thinking about/discussing with other people about having swiss style tournaments in our region. As far as your thread goes, it looks like you had the right idea, but as I said, the format would lead to potential hazards. Good work though. If you happen to have some sort of program for swiss style paring based on ELO ranking, that would be great, but otherwise i'd like to see that program you used for your MLG examples.

 

[EnigmaticCam]

All the tournaments I've seen on chessbase.com have been round-robin, so I assumed that's what was used. But I guess it could be different for the lower-end tournaments, or anything not big enough or with high enough rated contestants.

Frames, I can email you a copy of the program, I actually still have it. PM me your email address, or something.

At some point in that thread I posted some pics with basic instructions on how to use it; you might want to refer to that as a good reference for now. It's been a long time since I've used it myself so it'll take a little while to reacquaint myself with it. As far as I can remember, most of the major bugs are fixed and it's in working condition, but there were some things I still wanted to do with it that I never was able to. And by all means, if you have any further questions on it, let me know. I might even be willing to make more changes to it if there's anything you need.

 

[AltF4]

Oh, wow. That's crazy. I was just talking with friends today about starting an ELO-esque system here in Az for Brawl.

It needs some tweaks though to adjust for the intricacies of the game. But yea, totally.

 

[Frames]

The only problem i have with ELO is that its design, by nature, involves only two parties competing at once, meaning that there would be no support initally for doubles play. Other than that, I think it could work for smash, at least for the singles element.

EDIT - While thinking about it, I realized that if we just counted teams as a 1 on 1 team battle, with 2 players on each team 2v2 could work fine. In order to keep a consistent score, players would need to have a dedicated teammate, which could be a problem for local tourneys where people just pair up with whoever, but if people were going to a ranked tournament, they would probably take advance notice.

 

[ÒÚßÉÑ]

It would be nice if:

1. there's an easy way to represent the results of a double-elimination tournament in a simple easy to read format.
2. there was a rating system designed for double-elimination, catered to smash (and not necessarily ELO)
3. somebody wrote a tool that would take files from 1 and use 2 to generate ratings

i think it's too much overhead for each tournament director to calculate ratings, so

4. tournament results should be sent to a central place to be processed
5. using both past history and current ratings, tournament directors would make fair brackets

 

[Shaman]

It would be nice if:

1. there's an easy way to represent the results of a double-elimination tournament in a simple easy to read format.
2. there was a rating system designed for double-elimination, catered to smash (and not necessarily ELO)
3. somebody wrote a tool that would take files from 1 and use 2 to generate ratings

i think it's too much overhead for each tournament director to calculate ratings, so

4. tournament results should be sent to a central place to be processed
5. using both past history and current ratings, tournament directors would make fair brackets

thats a great idea, and AIB at least in terms of Result posting is getting their, but actually processing for ratings might be a bit beyond their reach.

 

[nealdt]

I was going to put Elo rankings in All is Brawl's tournament archive but there are a few issues that make this difficult from an administrator's point of few.

The biggest concern is that Smashers are lazy and often do not upload their tournament results for a week or two after the event. Other tournaments have gone on since then involving players from the previous tournament, and if those results get uploaded before the original's then you're going to be running Elo rankings out of order and messing things up.

Another problem is that not all Smash tournaments are created equal. Elo assumes that every single match has equal weight, but this simply is not true at Smash tournaments. A player like Ken/Azen/Chu isn't going to try as hard at a $40 local tournament as they are at a $2000 event like Pound/Champ Combo/FC. They're going to lose some matches they shouldn't (Azen is known for going Pikachu/Luigi/ICs/etc. at local tournaments and losing to people he's clearly better than) and that's going to falsely affect their ranking and their opponent's.

You can argue, of course, that Elo rankings would force people to try harder, but I think you'd see the opposite: if every "didn't try" loss counts against you, the top players will just ignore the rankings completely. I can't imagine a successful ranking system where the top players didn't care about their ranks.

I'm a math geek like you guys but I just don't think Elo rankings are appropriate for Smash. Player-run discussion panels will always be better at producing rankings with the assistance of tools like All is Brawl's tournament archive.

 

[EnigmaticCam]

I was going to put Elo rankings in All is Brawl's tournament archive but there are a few issues that make this difficult from an administrator's point of few.

The biggest concern is that Smashers are lazy and often do not upload their tournament results for a week or two after the event. Other tournaments have gone on since then involving players from the previous tournament, and if those results get uploaded before the original's then you're going to be running Elo rankings out of order and messing things up.

Actually, my program already addresses this problem. It lets you reorder events in the system and it will recalculate all the ratings based on the new order. So if you get the brackets for one tournament later than others, yet the tournament happened after the others, you can just rearrange the order when you get all the brackets for the others and it will automatically adjust the ratings. As long as you have an easy means of changing the chronological order of matches and events and a function to loop through all the matches and recalculate, it should be pretty easy to implement in any system.

But yeah, everything else you mentioned would be a concern. I will point out though that sometimes things like that happen in chess, where sometimes a grandmaster will accept a draw to someone rated much lower for reasons beyond the game (like if he knows the outcome of the match won't change anything and he's too tired to put any effort into it). You also see a lot of forced draws in world-championships just for the sake of ending the game quickly and saving energy for the games that count.

 

[nealdt]

So you re-run the entire Elo ranking from scratch every time a new tournament is added? How well would that scale if you ran it on, say, 6000 accounts over 700 tournaments?

 

[EnigmaticCam]

Nah, it doesn't rerun the entire system for every new tournament. It keeps an active roster of all participants and their ratings and assumes that rating is the most recent and accurate. That way as I enter each individual match, it automatically updates the ratings based on the results of that match. The only time that active roster isn't up to to date is when, as you brought up, you're inputting a tournament that happens before other tournaments you've already put in. When that happens, I just enter the tournament, rearrange the order to the correct chronological order, then rerun the entire system with a special function I wrote just for that purpose.

Something else I was thinking about but haven't yet implemented programatically, instead of just rerunning the entire system - a means of selecting a certain tournament and rerunning everything from that tournament onward. That way if you have a massive archive of tournaments, you can just tell the system, "Well I know everything up to this point is correct, let's recalculate ratings onward from here," and it's a painless process. That of course depends on your archives, if they have enough information to tell you what everyone's rating was after each tournament.

If you're not interested in housing years worth of tournaments, I also don't see a reason why you can't create a means of taking a good chunk of history and solidifying it - in other words, having the program one-time calculate the ratings up to a point and saving the result as a capture of ratings so you never have to recalculate up to that point again, because you know there weren't any other tournaments that you didn't know about.