As a fan of your ratings, I must express I am at a loss now that instead of pure commentary you're talking about some statistical model that I admit I find tremendously confusing.
Mostly because Sakurai doesn't follow math. We're talking about a chaotic being, so given that I miss your less math-y commentary.
First of all, I'll say that since Palutena was a re-rate, I didn't bother writing as much. I might include links to my previous ratings in the future... but you can find my original post about Palutena back there.
Anyway... if Sakurai didn't "follow math", then you wouldn't be able to get an 81% correlation by plugging these numbers into the model. Now, obviously I don't think he plugs information into a calculator to decide who gets in. But he knows which series are the money makers. Sales are also quite strongly correlated with "popularity" as well. If people like Mario, they'll buy his games, and vice versa, to some extent (I don't think people loving Zelda games has helped Tingle that much). And I'm sure he gets at least some pressure from Nintendo to promote popular/money-making characters. So, his behavior is not going to be random with respect to sales figures.
There's also the matter that I do not have access to WiiWare and Virtual Console sales figures, which means I may be significantly underestimating the appeal of some retro characters. It might be that VC sales have helped Earthbound's stature relative to some of the larger series. I also do not have access to sales figures in money, rather than copies sold, nor their profit margins. Nor do I know about money made from other merchandise (which probably helps Pokemon a large amount). So the series may be ranked differently in terms of their overall profitability and popularity (and fanbase strength rather than just size may matter as well) than the numbers I have.
Nonetheless, the correlation is pretty robust and although I haven't used the most rigorous methodology, 81%-83% correlation is
far too strong of a performance to be dismissed as "chance", to say that Sakurai is a "chaotic being" and thus cannot be predicted on the basis of more objective figures. Of course, he does have some freedom to pick characters he just likes, and he may respond to characters that are more popular than their sales would suggest. Then there's the matter of the cast of characters in a series - it doesn't matter how much sales Game & Watch or Wii Sports/Mii games get, it's very safe to assume they won't get more than one character. Other series, like Pokemon, have no shortage of characters.
I also don't think that the clones in Melee followed these rules as much, and in fact, I get slightly better results if I pretend the Melee clones didn't exist.
I can break down what I said a little bit more though...
I said "Using sales (logarithm of total sales since the relation is not linear), previous slots and movesets, whether the game is a typical genre (so Zelda, Mario, Golden Sun = typical, Wii Fit, Nintendogs = atypical), or has any playable characters (in Duck Hunt and Wii Fit there isn't really any playable character), I can get about an 81% correlation with a 10-fold cross validation on slots and 83% for move sets."
- I ran predictions for both slots and move sets. Some of the models I tried performed better on move sets than slots, which is not surprising given Sakurai's comments about how he thinks of it more in those terms. More move sets are more work for that series, so a series should "merit" that additional work. But the performance is pretty similar.
- 10-fold cross validation means that the model is run 10 times, each time training the model on 90% of the data. Then that model makes predictions for the remaining 10%. The performance of the model is then based on those predictions. This is meant to approximate how well the model will perform on new data. If you train the data on all of the data and test on that, then you would tend to perform worse than expected on new data due to overfitting.
- 81% correlation means that about 81% of the variation is explained by the model.
- So, I took total sales (in millions). You can see a clear relation between a franchise's sales and the number of slots/movesets they get if you plot it on a graph. It is, however, not linear. It appears to be approximately logarithmic. This explains why the Mario and Pokemon series do not have many, many, many times more representatives than say, Fire Emblem, despite having ~60x or ~25x as many sales, respectively. A logarithmic scale instead predicts the influence of Mario's/Pokemon's sales relative to Fire Emblem's sales to only be a factor of 2.5-3x as many slots. Obviously this is still a bit off, which is why I incorporated other factors.
- Previous slots and move sets are just that. For Brawl, these numbers would be how many slots and move sets the Mario series had in Melee. For Smash 64, this is 0 for all franchises since there was no previous iteration. This decision could be criticized. I do get slightly better performance if I only consider Melee and Brawl, but it may be slightly less robust due to fewer data points.
- Typical genre deals with Sakurai's comments about what sorts of characters are considered. Obviously being from an atypical genre is not a total barrier, as seen with Animal Crossing and Wii Fit. I took this to mean series that do not involve fighting, destroying things, platforming, etc. AND which do not have plots that involve such things. This means that F-Zero, which would otherwise be atypical as a racing game, is considered "typical" since the story line involves bounty hunters and such. Meanwhile, Wave Race and Excite as pure racers are atypical. I also counted Game & Watch due to how individual Game & Watch games would mostly not qualify as typical. Brain Age is obviously atypical.
- Playable character means that there is a visible playable character in the game or in its story. F-Zero has no visible player character, but we know Captain Falcon and Goose and Samurai Goroh are "in the cars". Duck Hunt, on the other hand, is first-person. There is no hunter ever seen or anything. This is why Duck Hunt fans always suggest the dog, who is not playable. Same goes for Wii Fit, Brain Age, etc.
How much each individual factor matters depends somewhat. Running the same data through but leaving out features can tell you this. You can note that some features are correlated with each other. Since franchise sales are an influence in how many slots a series got in the previous iteration, they are correlated. But it does seem that inertia does have a strong effect, and so Earthbound continues to get slots despite not keeping up with other represented series, and most series do not have a change of more than 1 slot from one game to the next.
Correlation in various configurations:
- Removing previous slots/move sets: 73% for slots (8% decline), 72% for move sets (11% decline)
- Removing sales: 78% for slots (3% decline), 79% for move sets (4% decline)
- Previous slots/move sets and sales only: 79% for slots (2% decline), 80% for move sets (3% decline)
- Previous slots/move sets only: 76% for slots (5% decline), 79% for move sets (4% decline)
- Sales variables only (including previous sales figures): 66% for slots, 66% for move sets
So you can notice that sales and previous slots do pretty well on their own, but previous slots does quite a bit better. So, series inertia seems to be the dominant factor. Nevertheless, adding you can get a significant improvement by incorporating sales into it.
But like I said, while this explains a lot of the variation, it obviously doesn't explain all of it. But I think it is a good guideline from which to start. So when I look at, say, Fire Emblem, I can see that it doesn't have the most amazing sales, so the model doesn't predict slots to increase much, predicting about 2.16 slots and 2.43 move sets. But I can also know some other things about the series, like that it has rotating protagonists (so plenty of good candidates), that they're very well-suited for being in the game, that it had one of its best selling iterations lately, but that the series almost ended due to previous games selling poorly. These are all going to effect how I'd rate them.
But since roster inertia is the largest factor, it's pretty safe to assume that Fire Emblem will not lose slots. That already gets you pretty close to the right answer for Fire Emblem - I'm pretty willing to predict that there will be at least 2 slots for FE. So there's quite a bit of wiggle room around that final slot. But it also tells you that expecting four slots for Fire Emblem is probably going to result in disappointment, as the model only made errors that large three times (they were underestimating how many slots Mario, Zelda and Fire Emblem would get in Melee... perhaps not coincidentally, those series all got clones).
Bottom line: sales and roster inertia explain a lot of what we see.
Series with less than 1.3 million in sales have never gotten a representative. No series has gotten 2 representatives with less than 2 million in sales. No series has gotten 2
unique representatives with less than 5.5 million in sales (Lucas and Roy both being at least semi-clones). No series with less than 10 million in sales has gotten 3 representatives. No series has gotten 3
unique representatives with less than 25 million in sales. No series with less than 30 million in sales has gotten 4 representatives. No series has gotten 4 reasonably
unique representatives with less than 55 million in sales. This doesn't make those things happening impossible by any means, but it certainly doesn't
look like Sakurai doesn't take sales into account in some fashion (although clearly not a linear fashion).
So looking at some of the franchises under consideration for Smash 4:
- Custom Robo has 1.3 million in sales, so it would be on the low end for a rep. Chibi-Robo, Xenoblade, Ouendan/Elite Beat Agents, Sin & Punishment and The Last Story all are less than a million in sales, so them getting slots would be unusual. Of those, only Xenoblade has an announced sequel, and it looks relatively high-profile and big budget, so if one was going to buck that trend, I'd guess it would be Xenoblade.
- Kid Icarus has 3 million in sales, and Pikmin has 3.8 million. Additional representatives would seem nowhere near guaranteed, but certainly possible.
- Earthbound still only has about 2 million in sales. Thinking that Ness and Lucas will stay but that Claus or Porky will get added seems a bit delusional, considering that it took 10 million to get Star Fox a couple clones, and Kirby had 25 million for actual unique characters. Fire Emblem with 8 million in sales would be on the low side, but looks a lot more plausible.
- A fourth rep for Star Fox, with only 11.5 million in sales seems pretty unlikely. Kirby, at 33.5 million, might be able to do it... maybe. But Kirby already has its major characters. Donkey Kong doesn't even have 3 reps, but yet has almost 70 million in sales and a high-profile DKC game on the way. It has by far the most sales of any series with less than four reps. Not only could they justify giving it a third rep, but they could even give it a fourth. DK getting a 3rd rep seems very likely.
- Zelda is also at about 70 million in sales, Pokemon at 220 million and Mario at 500 million. Mario and Pokemon can easily justify getting additional reps. Zelda maybe not.