Official Character Ranking System

[fMaster]

Hi my name is fMaster and I am currently studying Actuarial Science (a mathematics based degree). I am currently working on a formula to rank SSB64 characters but need to collect much much more data! If there is a database, and if not currently can we form one, that contains huge quantities of saved games (specifically containing the players, their respective characters, the level and the winners). I have currently developed a program for storage and basic analysis, but I would like to expand my analysis to a generalized ranking of all characters. The formula should work with user rankings as well. Anyone willing to assist or having information about an idea similar to this already, please comment!

 

[Battlecow]

There's no such database

We had like an elo system going, but that was more for players than characters

and you'd need one of those before going into character stuff, which would be pretty shady even then...

Anyways, good luck, I don't have any data to give you

 

[ballin4life]

Hey fmaster, perhaps take a look at the match up chart here: http://www.smashboards.com/showthread.php?t=241670. Then you could maybe plug in those values into your formulas (i.e. if a matchup is blue then plug it in as 70 wins, 30 losses, if it is green plug it in as 60 wins 40 losses, something like that).

Also I thought we did a newer matchup chart than that one I linked; does anyone know where to find it?

 

[<Fruity>]

Not sure if this will help, I hope so:


http://www.smashboards.com/showthread.php?t=329221

 

[fMaster]

That is an excellent chart and probably contains lots of useful data. I am looking more for the raw data, the data being discussed in the chart would be most useful. I was wondering if the emulator stored any data from online matches? or who to ask to figure it out?

 

[fMaster]

Preliminary results: I have determined that each individual player vs player match-up with characters on a specific level is a group of bernolli trials, summated into a binomial distribution. Each binoimial distribution contains an estimate for (p*), of the true probability of one character beating another, all other variables equal. There is a second highly correlated estimate (negatively correlated) for the same player match-up, but with opposite characters. I am trying to find a way to average these two numbers so that each character-character match-up averages out the skill level of the users. (IE if player1 > player2 then player1's ratio against player2 with character1 vs character2 will be high, while player's 2 ratio against player1 with character1 vs character2 will be low) I believe there is a way to average out these two ratios to get a single more accurate measure of each character match-up. My only concern is that when trying to estimate the p* value from the distribution of all player-player match-ups, it is most convenient and accurate when the samples are independent, and each of these 2 ratios will be correlated. Also, I believe that each player's character vs character ratio against different players will be correlated as well. Another idea: Take the average of each players averages (IE take player1's estimate with each character1 vs character2 on map1 and average the estimates against each individual and use that as the sample in distribution of the estimate of p*. I apologize if this is confusing, questions/suggestions/critizisms would be greatly appreciated. And the links are great, it will help if I can find more resources and individuals with knowledge on the subject!

 

[clubbadubba]

1) Paragraphs are your friend

2) Cool that you are attempting this, maybe it will force us to retain more data. Btw if you really, really want player v player data, there are a number of tournaments that have been held online and offline that you could go through. A lot of the players you'll see are not super active, but some of them will be reoccurring.

The 64 competitive online events section has a fair amount of competitive online data. Then there are also console tournaments you could consider. Some people might ***** about combining console and online together, but you can do whatever the **** you want with it all.

3) Just as a check to make sure your probabilities are correct:

My chances of beating isai should be 0
My chances of beating superboomfan should be 0

 

[fMaster]

estimation

your probability of 0 can't theoretically be accurate, but for just about any sample size we take, isai and super's ratios would be a 1 as long as they had no losses against you. If they are ever beaten then their estimates could not be 1 for their p*.

Also, I will work on the paragraphs.

Is there any specific I should contact or talk to about tournament data. I have been asking around but currently am stuck!

 

[clubbadubba]

Okay so you want specifics?

This is an online competitive league that I ran a couple of summers ago: http://smashboards.com/showthread.php?t=305831

This wasn't exactly tournament style, but every single match was under a competitive ruleset in a competitive environment and can be taken into account for what you're doing.
Other than that, go here: http://smashboards.com/forumdisplay.php?s=&f=208&page=1&pp=30&sort=lastpost&order=desc&daysprune=-1&prefixid=

Make sure the "From The" setting at the bottom is set to "Beginning", and take a look at the threads and see which events were actually completed and record the data. Unfortunately there is no one you can talk to that has a compilation of tournament data. If you want an actual database you'll have to use data from here and assemble it yourself.

Some good ones to take a look at would be any of the "Galaxy Ladder" threads. Though sorting through all of that data would be a mess. A couple of good tourneys were hosted by MATTS!, but make sure to only use the brackets that are for SINGLES, not any of the special brackets like Fat Kong or Low Tier Only or Death Tourney, unless you want to factor those specific character match-ups into your calculations.

On the console side of things (offline), there are the Apex 2013 results up until the final 4:
http://i.imgur.com/p4pUq.jpg

For the final 4 that aren't up (yet, trying to get someone to post them):
Boom beat Ruoka
Kikoushi beat Isai
Isai beat boom
Isai beat Kikoushi
Kikoushi beat Isai

You can probably dig up the 2012 results if you want some info about yours truly

Now you has data, gogogogogogogogogogo

Also yes isai/boom both have combined for an astounding 0 losses against me.

 

[fMaster]

I will start looking through the info for usable data. I still have a couple questions:

1) If were going to start a database of accessible data, where would I want to place the database to make it accessible (here, on a shared online server, etc)

2) I still think the best way would be to set up a competitve league that anyone can play in that records all games played.

Sincerely, the best way to get the best data is to set up the system to record it now and worry about analysis and formulas for ranking later. I can sift through threads and record old tournament brackets. But manual analysis and recording of games is time consuming and will not be time effective to get accurate results. When you play online, is there a specific place/platform everyone goes through to play? or are individual games setup by individual match-ups?

 

[clubbadubba]

There is no recording anywhere of all tourney matchups ever played. The character/stage selections are almost NEVER recorded unless the video is posted on youtube. All you might be able to get is the score of a set, and even the availability of that varies from tourney to tourney.

I agree that if you want to do this you should probably just use recent and future data (like Apex 2012/2013, maybe some of the most recent tourneys). However, you should know that almost all of this data will have to be manually entered in some way. Online play is done through the Kaillera client, and almost all games take place on a specific server called The Galaxy. In the server, users set up individual rooms , other users join the rooms, and the original user will start the game when he wants to. Games are not recorded by the players or the server, and to my knowledge there is no mechanism to record exactly who one which games and who played what and where they played for each game. I suppose one of the players COULD record the match and post it, but still that would require you to manually search the video to find characters, stages, and results.

The best way to get this information would be to have the winner of each match post the results directly to some database that you set up. It would be pretty difficult to get players to do that consistently though, even in a tournament setting where they would only have to do it a couple of times over the course of a few weeks. There are just too many players in the community that are.... anti-competitive, and prefer not to be ranked or judged in any way. Maybe you could make an online tournament and have a requirement be that the kind of results you are looking for be sent directly to you by the winner for each match.

And for your own good, don't bump a bunch of really old threads with the same message lol. You don't want people to think you're spamming the forum or being a troll. POST RESPONSIBLY. Or just private message people with posts like that so you don't flood the boards with old threads.

 

[prisonchild]

Also yes isai/boom both have combined for an astounding 0 losses against me.

just curious, what is the closest you've gotten to beating either of them?

 

[clubbadubba]

I got isai's link down to 2 stocks with my fox in Apex 2012. Probably haven't done any better than that, and I haven't played him too much tbh. I think playing me is boring for him .

I've come pretty close to boom a number of times. Like I've been up 2 stocks to 1 a few times, but he just comes back anyway. All in horrible match-ups for him of course.

 

[fMaster]

SmashFest

So I've heard about these things called SmashFests. Are they just sessions where players get together and play? Also, how many people usually show up at Apex for N64? What is the biggest smash event for strictly smash (like not apex or apex regional)? Sorry if I am spamming questions, clearly a nub to the community, just got some ideas.

 

[meowmeowrainbowkitty]

lol yeah, smashfests = generic term that we use when smashers get together and play.
low-key,whatevs, for funsies. (friendlies = not [too] serious play)

I think apex was like 80-somethnig this year? (wow, i can't believe i don't remember..it's only been a month). and then a bit less last year. (i think the cutoff was 64...can someone confirm?)

biggest smash event besides Apex...? Not sure in terms of 64. Maybe Genesis? besides that, just like random events held by people. 64 community is growing, but still nowhere close to melee

 

[asianaussie]

japan has had (and will probs continue to have) massive meet-ups, but idk if you want to count them

 

[meowmeowrainbowkitty]

Oh yeah, forgot about that, good call AA.

and just a FYI for fMaster: Japan smash (J) is different than the US version (US).
a few of the characters have differences between moves/knockback/etcetc between the versions, so, like AA said,
not sure if you want to count that.

 

[Han Solo]

Apex 2013 had 95 entrants I believe.

 

[Nintendude]

It was actually 96 since we filled the vacancies last minute (Deano and Bobbo).

Biggest solo Smash events might have been Pound V in MD in 2010 (I think), which had 347 for Melee and 151 for Brawl. Genesis in 2009 in CA had 290 for Melee, 292 for Brawl, and I think 22 for SSB. Apex 2012 and 2013, of course, shattered those total attendance records for Smash alone but it had other games too.

 

[fMaster]

Smash Season

I'm thinking of holding a smash fest in the midwest region. Ideally, 100 players and a weeks worth of play data would be enough to do some conclusive analysis. Or even 2 or 3 weekends with 50+ might work. Would that be something the 64 community might be interested in attending?

 

[clubbadubba]

A weeklong, 100 person smashfest is definitely something that we are interested in. Some of us may even dream about it at night. But the realistic possibility of making that happen is pretty low. The largest "high-level" 64 tourney in NA was this past Apex, which still had only 96 entrants. Getting people to travel again for another smashfest would be difficult. Costs money ya know?

 

[Nintendude]

You could probably get decent numbers if you advertised and held it on a college campus. College kids love SSB and they all think they're amazing at the game. Scrubs aren't useful for analyzing the game though. The pool of people actually good at the game is pretty small and very spread out.

 

[fMaster]

tournaments

I already hosted one tournament and have 2 more on the calendar for the next 2 months, at Illinois State University in central Illinois. 2 hours from chicago, 3 from indi. Had a couple 50+ turnouts last year. I agree though, most are chump change.

What if there was an annual smashfest in with each regional Apex tournament and data from each region was collected. I have a basic program written in java for looking at character and player match-ups, but no strong method or parameters for collecting good raw data. What factors do you guys think are most important in determining fair character vs character match-ups?

 

[clubbadubba]

Well the thing is actual character matchups only come into play when players are decently close in skill. I think first you'd have to put together a ranking system for the players, then use that when considering character matchups. I'm sure TO's wouldn't mind collecting data for you, if you just state what data you want collected.

Btw the only data worth collecting is who won each game. #Stocks left can be influenced by people not trying hard when they have leads and stuff like that. So who's playing, who wins, what stage, that's about it.

 

[fMaster]

only data I collect is characater match-ups on specific levels with specific players. IE i have data on player1 vs player2 using char1 and char2, respectively, the map and the winner. I can extrapolate actual character relative ranks from that sample data alone (if its isn't skewed and there is enough). To set up a ranking system with players would require a ranking system of characters seeing as (if true) certain match-ups favor the user. Also. I believe I mentioned it before but for each instance (player1 wins, player2 losses, char1, char2, map) the better players stat is inflated (this is what you mean by having equitably ranked players must be used) but then the same character match-up with players using opposite characters (player1 wins, player2 losses, char2, char1, map) would be equally unweighted. Perfect example would be boom or Isai. If Isai's record is 10-0 against you when he is link and you are pikachu, then it is also likely that your record with link against his pik with also be a no loss ratio.

So say I win x10 time as much as the other player. Then a character vs character ratio of 20-1 (20) for my mario against his luigi, and a character vs character ratio of 2-20 (.1) would show that even though it is apparent that I am better than my opponent (the fact my mario-luigi ratio is higher than the inverse of his mario-luigi (ratio ^ -1) implies that even though neither total distinguishes either, the combination of the oppositely correlated ratios evens out the difference in skill levels.

Also, in my quest for usable data there are some standards that would eliminate those risks. Any useable ratios for wins and losses that are to be applied to a character as a whole must contain a minimum number of games played (i'm thinking 10 min) and have at least 1 win and loss. This is extremely important when using ratios to eliminathe the risk of having to integrated undefeated characters. Another example, if Isai is part of the data and never loses, I can't use his character statistics because I have no data that differentiates his characters and thus they provide no useful insight in the character rankings overall. Like you said, the players need to be at least reasonable close in skill (such that in any match-up there is enough probably of either player winning that the event has actually occurred).

 

[clubbadubba]

Well its tough to see exactly what you're doing here (nor do I care to go too deep into it lol), as long as you are considering both character AND player ability in some way then that should be fine. Its especially tough when there are a fair amount of players who will show up for a single tournament and never show their face again. The actual # of games between active, competitive players, is very limited. That is really the biggest issue you'll have. Plus, competitive events are so few and far between that skill gaps between players can change a large amount in between data sets, making past data quickly obsolete in terms of player skill calibration.

 

[Studstill]

Hi my name is fMaster and I am currently studying Actuarial Science (a mathematics based degree). I am currently working on a formula to rank SSB64 characters but need to collect much much more data! If there is a database, and if not currently can we form one, that contains huge quantities of saved games (specifically containing the players, their respective characters, the level and the winners). I have currently developed a program for storage and basic analysis, but I would like to expand my analysis to a generalized ranking of all characters. The formula should work with user rankings as well. Anyone willing to assist or having information about an idea similar to this already, please comment!

chris.studstill on Skype. The one from SC. Holla. I have your data.

 

[Sangoku]

Stop digging year(s) old threads seriously.