Official: SSBPD unsupported; source code released.

[ShiroiKen]

Do you include pool games in the calculation? Do you have a K factor (like in Chess)?

It would also be interesting to add a factor of "importance" of the tournament, as in you might have local tournaments and national tournaments.

 

[Fortress | Sveet]

Well just know that ssbpd.com is not being updated anymore and isn't supported. We are reworking the system to be integrated into the smashboards site. We have had discussions on the specifics, and I think the consensus is to follow chess's ELO system very closely.

 

[michael_li]

ssbpd is garbage site. failed project pffffff

 

[Fortress | Sveet]

See: my title

 

[Mahie]

Do we get a preview of the features or a due date or some kind of annoucement?

 

[Fortress | Sveet]

I'm taking bribes through my paypal if you can't wait

 

[Nos]

Any update on this?

 

[Fortress | Sveet]

I have the coding done for processing TIO files already. I have design diagrams for the scope of the project. I am just waiting for server access for the time being. To do for me is 1) output pages (player.php and tournament.php) and 2) ELO calculations. Im probably missing something, but thats pretty much the gist of it.

 

[Terral]

Also, I appear to not be involved in this anymore, if anyone had been following my progress. I'm not too upset, since they decided that my mostly-complete (if not hastily-coded) Python site should be scrapped in favor of rewriting it in PHP, which I'm not very fond of. On the plus side, the new-new SSBPD may have an easier time integrating with the forums since it will have been built to do that from the beginning, instead of just as an after-thought.

 

[Fortress | Sveet]

Terral, your advice will always be welcome with me. Honestly, I haven't seen you post since the forum switch, where you been man?

 

[Terral]

I made a few posts in the backroom thread post-switch, but I couldn't tell if it was just an archived forum. It was the only backroom-related area I could access at the time, though since then I was removed from it. I don't generally post on the forums much, what's left of the local crew moved to Facebook to talk about things.

 

[Fortress | Sveet]

Which backroom thread? I haven't had access to anything MBR since the switch

 

[Terral]

Just the same one that we had been posting in prior to the switch, in the Melee Backroom. I had it bookmarked so I just went back to it, and it still existed.

 

[Fortress | Sveet]

Oh ****. I don't have access to it or anything.

 

[FoxLisk]

Sup homies.

First off, since I do have some pride, I outright reject the idea that it was a "failed project."

Secondly, since in other people's eyes perhaps "no longer supported or even reachable" might be considered a "failure," I've decided to release the source code.*

Finally, I'm going to go back to not paying attention to the site or community now, just had some time to throw on putting this up online after meaning to for a while. Enough of you have ways to contact me that I'm sure I'll hear from anyone who needs anything.

Good luck with whatever follow projects are intended to replace this!

*I make no guarantees about code quality: this was all done in a short time period with no prior experience in C#. It worked, though!

 

[Fortress | Sveet]

FoxLisk you da man! You have a guaranteed high five from me no matter what, where or when (please redeem before my arms are amputated from video game related arthritis)

 

[FoxLisk]

Haha thanks! Let me know if I can be of any tangential use - I don't have time to really work on it but I'm happy to answer questions.

 

[Bones0]

Anyone who considers this a failed project is dumb. It'll definitely go down as a footnote in the history of Smash for being the first working objective ranking+database system.

 

[Hax]

an objective ranking system is by definition a failed project

 

[Ziodyne]

It fails to accurately gauge who the better players are, but did a decent job in monitoring at least who was winning tournaments. It could've been a fine project for just that purpose alone (at least, I see no reason why it had to be considered a failure).

 

[Anand]

an objective ranking system is by definition a failed project

What does this even mean?

 

[Krynxe]

^ That you can't objectively rank players, and thus anything attempting to do so will ultimately not be successful.

It was still very nice for seeing tournament history and such. Your work was very commendable FoxLisk, I really hope you don't feel under-appreciated because I know a lot of people enjoyed your contributions to the community even if they didn't directly say it here. Including myself.

Thanks for your time in this project dude

 

[Zivilyn Bane]

Obviously Hax has zero understanding of what an Elo system is supposed to do. Please wikipedia the term before posting what you think it was designed to do.

 

[Krynxe]

Obviously you have zero understanding that Hax was referring to what Bones had said. Please accuse the correct person before posting that they're wrong.

 

[Fortress | Sveet]

an objective ranking system is by definition a failed project

^ That you can't objectively rank players, and thus anything attempting to do so will ultimately not be successful.

An objective ranking system is just a count of the facts. Who beat who beat who, etc. What is wrong with that?

 

[Hax]

I don't see what good it does to widely publicize a list where I'm 18th, Kels is 7th (no disrespect), the top 5 isn't even ranked correctly, etc. an ELO system works when you have a larger sample size of tournaments where everyone gets to play each other (or, ideally, when people can play each other online at any time). in SSBM, these tournaments happen once or twice a year. to use ELO in this situation is asinine. speculation/subjective assessment of how good people are is the only way to properly rank people with a sample size this small, sadly. SSBPD was bound to fall victim to regional inflation/other factors, which is why I wasn't a fan of it from the start.

I respect the amount of work you guys put in, and I think compiling a database of that many tio files is great in itself, but IMO it doesn't do any good to create an extremely inaccurate list and call it our premier ranking system. sorry

 

[Juggleguy]

I can't speak for everyone else, but I never considered SSBPD to be an objective "ranking system" by any means. I always thought of it as just a database that was extremely useful for tournament documentation. Whoever makes the next SSBPD should probably stick to that kind of label, because yeah, it's easy for an attempt at ranking players to be distorted by regional inflation or lack of data.

 

[phish-it]

The best thing to do would be to polish this system as you guys are doing, and by the time Smash 4 tournaments can utilize it from the games birth and actually have respectable sample size.

 

[Bones0]

lol Smash 4


As far as the system being inaccurate, that's just because there wasn't enough data (which has only been said a billion times by now). Over time, and with the proper adjustments, it definitely would have been an accurate representation of skill, or at least close enough that we wouldn't need region-specific power rankings. It seems naive to claim that it didn't work when the source of the problem was stuff like TOs in certain regions not uploading their Tio files. It wasn't like the system just didn't work...

 

[Zivilyn Bane]

If that is your reasoning Hax, then I can agree that the project wasn't complete. But not complete does not mean failure. It needed more time and more data.

Also the main goal of the project was to compile a database of players and tournaments. The Elo was a side project. Elo is not meant to be a ranking system by any means. An Elo rating is a relative number that can be generally used to gauge a players performance based solely off the data it contains.

Elo usage for online games actually creates much more inflation than regional density. This is proven again with the model of Chess. Online chess always uses different formulas and systems because you'll typically see people playing each other several games in a row (especially with fast games like blitz or bullet chess where it's game in 5 or less).

Some day hopefully we can come up with a fair and accurate means to account for regional congestion. However once again, just because somebody lives in a smaller or less active community doesn't always mean that person is worse, which is why Elo has to be very black and white.

 

[Fortress | Sveet]

Elo is just one way to interpret the data and it isn't even accurate most of the time, but it is still an interesting metric to look at. I also want to look into character usage lists; its hard to get all of the old tournaments, but I'd like to keep track of it moving forward.

 

[Massive]

Without character and stage selection information and an mathematically sound method of determining coefficients for character/stage matchups, I feel like the only way of really salvaging the current Elo data would be to introduce regional weights (AND THEN WATCH MY SCORE PLUMMET INTO OBLIVION).

I have a feeling determining such values would prove to be beyond controversial though.

 

[Dan Salvato]

I agree with regional weights. It's easy to skyrocket your Elo simply by winning tournaments against locals. For the formula to be modestly accurate, we'd have to make some adjustments based on region, or at least the potence of those with bad or no records.

 

[phish-it]

Are there diminishing returns on beating the same person multiple times?

 

[Mahie]

Wasn't it a matter of top players not being rated high enough, rather than good players being rated too high?

IIRC EC had little to no files uploaded, so that didn't help either.

 

[FoxLisk]

You guys don't understand Elo systems at all. You can't put in a priori weights for characters or stages, those drop out of the rest of the data, at best.

If you come into trying to rank players assuming you already know better than the system, then of course you're going to be disappointed - one of you has to be wrong, and you know better, so it's the system.

@hax is of course right: there's not enough data, and there never will be. it's just a curiousity, and not even the primary point of the project when it was begun.

 

[Zivilyn Bane]

Mahie hit the nail on the head. Everybody complaining that Kels was rated too high or whatever other player was too high: the issue was that there was just not enough data from the other regions, specifically the East Coast. Their best players only had 4-10 tournaments on file opposed to Kels having like 50. This happened because many of the East Coast TO's didn't have .tio files or just didn't want to upload them. Some of them were just too cool for school and though that the Elo ratings were dumb so they didn't want to help. Whatever that's fine. But don't hate the system because it lacked data.

Also, we'll implement regional weight as soon as somebody proves to me that just because a player lives in the Midwest automatically means they're a worse player.

FYI the highest rated chess player in the United States lives in Saint Louis Missouri. He's also the 9th highest rated player in the world, is the current US Chess Champion, and has broken nearly every record for an American citizen there is. Perhaps we should take 200-300 points off his Elo because he lives in Missouri huh?

 

[Dan Salvato]

I think you could probably establish a regional weight not based on an estimate of how good each region is, but instead based on how well the top players of one region fare against the top players of another. You don't make it a manual guess, but you make it a formula. The regional weight would update whenever that region shows up at a national.

The point of regional weights is that we don't have enough nationals to accurate rank top players in relation to one another. However, there are tons of locals to accurately determine who is best in the region. However, the best player in the region needs to be "capped" based on his skill compared to other regions, or the locals will give him inflated Elo. In the end, we're making up for not having enough national data by using a formula to estimate it.

 

[Massive]

Also, we'll implement regional weight as soon as somebody proves to me that just because a player lives in the Midwest automatically means they're a worse player.

There are some very good players in the Midwest, but I defy you to point out even one that has won a national/international in the last 6 years. Our average skill level lags behind most other parts of the country owing at least partially the sparsely populated nature of our area. We definitely have our outliers, but there is not even a comparison to the density of very good players on the coasts. It was far too easy for us to dump loads of local tourney tio files into the system and watch our scores grow, since we're so far from the coasts that there was rarely any contamination.

FYI the highest rated chess player in the United States lives in Saint Louis Missouri. He's also the 9th highest rated player in the world, is the current US Chess Champion, and has broken nearly every record for an American citizen there is. Perhaps we should take 200-300 points off his Elo because he lives in Missouri huh?

I don't know, how good is he at melee?

 

[Fortress | Sveet]

I think you could probably establish a regional weight not based on an estimate of how good each region is, but instead based on how well the top players of one region fare against the top players of another. You don't make it a manual guess, but you make it a formula. The regional weight would update whenever that region shows up at a national.

The point of regional weights is that we don't have enough nationals to accurate rank top players in relation to one another. However, there are tons of locals to accurately determine who is best in the region. However, the best player in the region needs to be "capped" based on his skill compared to other regions, or the locals will give him inflated Elo. In the end, we're making up for not having enough national data by using a formula to estimate it.

The way to make the system more accurate is to give it more accurate information. Theres no tweaking the data without ruining its objectivity.