Some might disagree with you, if they are competitive. Losing isn't fun. A casual might not care, but those like me won't find enjoyment in a game, especially one where certain characters are better than others. I find the idea of a truely balanced game to be more idealistic anyway. Still, I'd prefer to have a character who will be able to deal with most characters, rather than few.
I know some will disagree. Regardless, this makes me glad that these people aren't the main focus. There are "hyper competitive" players that will only play the game a certain way, see the characters as mere functions and wish for the series to become more like that.
And if you commit yourself to playing a fighting game, you have to accept that you are going to lose sometimes. Failure is a part of life; a lot of us will fail many times before we succeed. Losing doesn't always have to be unfun. You can still use the experience as a learning opportunity. It's about the disposition.
Also, there will always be some unbalances in a roster filled with characters, Smash isn't the only one guilty of this as it goes to every fighting game on the market. This is another thing that one has to accept if they commit themselves to playing Smash, Street Fighter, Mortal Kombat, etc. The ideal of having a character being able to deal with most of the other fighters is noble, but it is extremely improbable.
Balance patches will still issued throughout Ultimate's history, and that gives it an advantage over its predecessors. Regardless, the purpose of buffing and nerfing isn't really to balance, but to make a game that is fun to play and watch. Focusing too much on balance can hurt a game in the same way that focusing too much on money can make you unhappy.
If you want to play a fighting game that is 100% balanced and fair, go play the first Street Fighter. During a match, both players have access to the exact same options, making it the ultimate balanced competitive game. But it is super boring because you can only play as two characters who fight exactly the same, but hey! At least its balanced! So, yay, I guess?
Having diversity of movesets is better than focusing on creating some 5:5 fighting game utopia.
I fail to see why having a professional mindset for SSB would be harmful. Apparently, four top players from Japan had their influence in SSBU, and for whatever reason, SSBM fans are drawn to this game. Is it so they stay relevant? Is this the next best SSB since SSBM? Do they realize SSBM won't last forever? I don't have epistemic access to answer those questions. Nevertheless, that four top Japanese players had any part with working with Sakurai shows the direction in which fans want SSB to go.
Because it would cater to a particular set of needs that can stunt the creative process of the game, its players' creativity, lower the excitement factor and overshadow its crossover appeal. Your previous posts are very indicative of this; caring exclusively about the numbers and balance focus when they don't exactly guarantee for a game to be fun. To clarify, I don't think that listening to this "professional" mindset is harmful per se, but letting it bleed unfettered over the creative process is.
And yes, 9B, AMSA, Ranai and Earth participated in the development on this game, as Test & Quality Assurance if the credits are to be believed. This means that they helped test the game but didn't exactly influence the decision making in terms of roster. It is very unlikely that they were whispering into Sakurai's ear the whole time, telling him to cut down the roster in the name of balance. Even if they had that kind of power, I doubt they would have done it. Nevertheless, I'm glad they let them help and I'm sure their input was valuable.
I've seen some people complain about how few people want to goof off in Quickplay, or perhaps that includes Public Arena. Perhaps the direction of SSB is to cater toward a more competitive style. As for removing characters, I just don't see how logistically or economically wise it'd be to push past, say, 100 characters. We're already getting 80+, and if more characters are really what makes SSB so great, why not play other fighting games with massive rosters?
There will always be people who go online to troll or grief other players. And this isn't exclusive to Smash either. You can't really use such an example of some isolated behavior like this as a measuring stick.
And to answer, your last question: Because beloved Nintendo franchises and other icons face against each other, that's why. I hope I don't have to explain the crossover appeal.
Quite honestly, I think SSBB should probably have been the last. I recall an interview where Sakurai wondered what else could be added after SSBB. Nothing, as far as I can tell. It'll be the same old content all prettied up, perhaps with some additional features that are too mundane to awe anyone. I've never been a fan of a huge roster anyway, which is why I've promoted by protagonist-antagonist concept here and there on this site. Heck, I even thought maybe a new SSB game should come out called Super Smash Bros. Dark, where there are only villains to play as.
This seems to be a personal problem rather than a flaw from the series.
If you think that the series should have ended at Brawl, then you have clearly lost your drive or passion to play it's successors. The crossover factor is also lost on you and you seem to be personally detached from Smash at this point. Why are you bothering to go through the motions of playing Ultimate then?
If I were in your place at a point where I can no longer get any enjoyment out of the game I would have dropped the series already and rethink some of my decisions.
You may have thought that there was nothing more to be done after Brawl but that is simply not true. There will always be room for improvement, that's how we progress. Aside from the obvious new characters and stage additions; refined gameplay, new mechanics, polish, quality of life features, music tracks, prettier graphics, etc. Personally, I see Ultimate as the height of the series, and if that were to be the last game then the series would end on a high note.
Imagine if Nintendo thought to themselves that they peaked at the N64 and they didn't need to anything else, or if certain series were discontinued because the devs thought they couldn't do better.
Regarding Ridley, the history between him and Samus is too intertwined to make removing him any sense. I could see those like ZSS and Dark Samus being removed, although the latter had her own story arc. Anyway, you can perceive what I want as selfish, and that's fine. Even if it was for selfish reasons, I fail to see why my motive would in any way make my argument invalid. I see your point of view, but I don't think we'll be getting anywhere. I appreciate your time, however.
I was only using Ridley as an example. Point still stands in what I meant to say. Some characters, including plenty of favorites, would be cut for the sake of a selfish, detached desire.
You said that the crossover factor doesn't matter to you, all the more reason for me to believe that your approach would be harmful for these series. Smash means a lot of things to a lot of fans, and a project like it needs to be worked on by passionate people who care for it.
This makes me feel that Smash simply isn't for you anymore, and it may be time to move on to something you can actually enjoy.