According to my calculations, I would give the chances of at least five of the top ten as somewhere around 63% if I'm pessimistic and around 81% if I'm somewhat optimistic on the characters. I'm inclined to lean towards the more optimistic.
So chances of 5+ out of 10? I'll say 76.54%, after going back and forth on scores.
I actually want about 6 of them. I'm mostly indifferent to Pac-Man and Takamaru. I want Mii and Chrom
not to make it. So in most scenarios, I like the outcome. I thought about weighting by their likelihood, but I don't weight my want scores that way any other time. And in any case, it didn't make a lot of difference.
So for want, I calculated my want for each scenario (basically my average want score for each character to be in or not in), but gave extra emphasis to Ridley and K Rool. If Ridley and K Rool both get in, that scenario is basically one that I want a lot (>90%), even if the rest were my least favorite of the ten. Then I just averaged across the scenarios.
Half RTC Top Ten chance: 75.91%
This is likely to happen. There are just a lot of ways to get to 5 out of 10, especially since Palutena and Pac-Man are very, very likely, you only need three more... And Chrom is also very likely, and several others have decent chances.
Half RTC Top Ten want: 68.21%
Most of the scenarios I'll be getting at least two or three characters I want, and a decent number will have either K Rool or Ridley... and the ones where I get both basically mean I'll be satisfied with the roster. So overall, this will generally please me.
NOTE! These are for our
current top ten. One of our top ten
already made it.
Grade for our predictions:
We have to grade given what we know. We're not going to be perfect without insider information. So an A should be predicting around the right roster size, beating naive baseline predictions, and having solid performance in ranking of characters. And accounting for all the possibilities.
Well, the predictions for Brawl were not very good. They wildly overpredicted the number of slots. Beyond that, they also had some really bad misses. Mathematically speaking, they were worse than just predicting the Melee roster would repeat itself. So I'll cut them some slack, and look at it from a ranking point of view.
7 out of their top ten made it. But their job was a little easier back then, with some low-hanging fruit like Sonic, Diddy and King Dedede. Of the 23 newcomers they rated over 50%, only 8 made it. They predicted 15 of them to make it. Of their top hundred, they predicted about 38 would make it... but only 10 did.
All in all, they should have adjusted their scores downward by a lot, because a roster over 40 or so for Brawl was very unlikely, and they were predicting it would be like 80, meaning they were either delusional or really bad at math.
We're a
lot better this time around. We may have underestimated Rosalina's chances, but she's still ranked a respectable 30 on our list (1/6 of the way down our list). ROB was ranked 110th on the GameFAQs Brawl RTC (1/4 of the way down their list). They gave him higher chances, but their numbers were also way inflated, as I've already pointed out. We got it wrong, but we shouldn't feel
too bad about it - if we think our scores are accurate, we should get at least a couple of characters from the lower end of our distribution.
So,
GameFAQs Brawl RTC grade:
Quantitative evaluation (Brier score): F
Roster size: F- (this was just awful)
Top Ten: B+
Ranking: C
Big whiffs: C (Pokemon Trainer)
Overall: F
They get credit for good performance in their top ten... But that's the only thing they did well. Subtract a lot for their ridiculously high scores for everyone else, and the fact that according to Brier score, they were worse than just predicting the Melee roster. They missed Pokemon Trainer, but in their defense, Pokemon is probably the hardest series to predict for. On the other hand, PT was from Gen 1, so they had plenty of time to think of him. And they rated 400 characters...
Right now, I'd say...
SmashBoards Smash4 RTC:
Brier score: B
Roster size: B
Top Ten: C+
Ranking: C+
Big whiffs: B- (Greninja)
SmashBoards Smash4 RTC grade: B-
We've got a solid top ten, and our scores appropriately drop off quickly after that. I think we can beat simply predicting the Brawl cast to return in Brier score. But I think our scores are still a bit too high (most of the characters out of our top 50 or so should be in the 1% or less range, and our initial scores were even worse... with more info we're better, but duh). But our roster isn't
too inflated. And I think the order we have the characters ranked is not bad.
But we did miss Greninja altogether. Although, again, Pokemon has the largest cast and so is the hardest to predict. Compared to GameFAQs, we only rated about half as many characters, and Greninja didn't even debut until after we started the game, so it's a
lot more excusable.
All in all, I think we're doing decently - better than most.
If we're grading on a curve, judging by Miiverse and other places I've seen predictions, I think we're doing alright.
Because of the combinatorics, people will be surprised at how high the chances of this are. Some of you are really far off. For example...
RTC's Top 5 predictions: 20%
Let's see, Palutena and Pac-Man are kind of given at this point. K. Rool is likely, and so are Takamaru and Mii. Isaac is a 50-50. I don't have quite as much faith in Shulk, Dixie or Chrom's chances (40-ish). Ridley's chances are dead in the water.
If I assume you give 90% to Palutena and Pac-Man, "likely" means 60%, 35% to the "40-ish" and 1% to Ridley... You should be saying: ~69%
If you think their chances are a bit higher, the overall would correspondingly be a bit higher (maybe around 80%?).
You need to work out each individual scenario's chances and then add them together... there really is no easier way to do it with varying chances like this. If you really want to I'd advise a spreadsheet with true and false on if they're in the game in said configuration and then put the chances to the power of that value (1 or 0), multiplying all the resulting numbers to make a total chance for that particular configuration.
Do this around 1500 times, making sure you don't double up on any of the scenarios (the hard bit), then add them together and you'll have your answer!
You beat me to giving out some correct numbers.
But I instead wrote a little Python script. The thing is, there are 1024 scenarios (each of ten characters has two possibilities, 2^10 = 1024). So, the easiest way to generate scenarios? The numbers 0 to 1024 in binary!
Then I picked out the combinations with >=5 characters, multiplied their chances accordingly and added the scenarios all up.