Don't get why people would in particular be upset over a 20 roster. In fact I'd say it could actually be more diverse then a roster containing 50, because there would be a stronger emphasis on creating unique characters and create better balance. Games like Guilty Gear and BlazBlue benefit form this particular type of layout.
Ideally maybe, but realistically I think it's a very different situation since smash is a mascot fighting game. It's a completely different situation from something like Guilty Gear or Blazblue. Even ignoring the fact that lots of people want to play as their their favorite characters, there are a lot of necessary characters that would have to be in the game, like most of the 64 cast minus Jigglypuff and maybe Luigi, as well as a few other characters that have become very noteworthy since then, and since Sakurai tries to stay true to each character, there's only so much diversity he can give a cast with a given set of characters. Having a large cast gives Sakurai more leeway to add characters that he wants to add for gameplay purposes. A game like Blazblue is completely different because they can think up a gameplay idea and just make a character that fits that idea. Sakurai can do something similar by picking a character from the Nintendo catalog that fits the style he wants, but only if the essential characters are out of the way. It also lets him incorporate characters he's really passionate about, you can tell from his interviews that there have always been character he's though would work great in Smash, but he couldn't add them because there were more important characters to add first. I honestly think roster diversity is something that wouldn't be as much of a focus in Smash game with a small roster, because other things would take precedent, but in a larger game they can afford to do more diversification, if that makes sense. Whereas in a non-mascot fighting game like Guilty Gear or whatever, the opposite is generally true.
The game WOULD be more balanced though if that's what you're into. It'd probably be balanced the way Smash games usually are, though, taking stuff like items and free for all into account, and not taking into account the sort of metagame the game would develop once it's out in the wild, so it's anyone's guess how many of the characters would end up tournament viable.