Melee has faults but they are usually in specific matchups or bugs. Like, the difference between top tier and bottom tier in Melee is a lot less than in Brawl. And tripping isn't the only thing. Auto-ledge grabs, ridiculously easy chain grabs that usually work on most of the cast, lack of combos (it just feels like your trading hits more than taking advantage of the physics to do combos), slow speed (again, every time a developer slows the game down players hate it and it ruins the game), and a bunch of other stuff. Point being, Brawl is not a good competitive game. It's decent enough for it to work, that's about it. Again, you can take any game with multiplayer and find a way to make it competitive but that doesn't equate to a good competitive game.
Skill gaps are important because they reward players that put time into the game. Imagine how frustrating it would be to play a game for months or years, you know everything there is to know about the game, yet when you play against a newbie they beat you (or come very close to it). It would make you not want to play anymore. Mario Party, for example, is a game that has virtually no skill gap. In terms of being a party game, this is fine because it's meant for people to have fun and just enjoy the game. But now take Mario Party and try to make tournaments for it, with money on the line. You will see the game fail very fast. Why? Because there is no being good at the game. Maybe to a very small extent, but no one will enjoy it. This is what I felt they did to Brawl. There's certainly more to Brawl than Mario Party, but when you compare to Melee or even 64 it is just no where near that level.
Sakuri purposely changed the game to be like this, when it wasn't at all necessary. There was nothing wrong with Melee that negatively affected casuals. Brawl could have been made to be like Project M (aka. Melee) and EVERYTHING would have been the same except that the game would have been superior in a competitive setting. Reviews would have still been great, casuals would have still loved it, SSE would still be awesome (btw SSE was the only part of Brawl I thought was excellent), and best of all the competitive community would have GROWN and EXPANDED and been overall BETTER. Instead we got Brawl, and the competitive community imploded on itself, and limped along ever since. This is why I don't like Brawl.
I loved the SSE as well. Ironically, a lot of people seem to hate it and blame it for Brawl's problems (saying that they should have spent more time on balancing the game, etc.), but I'm glad that you liked it. It's kinda a bummer we won't see an SSE2, but we'll have some kind of story mode at least.
And it sounds like your main issue really is the whole competitive aspect and the community splitting. And I'll admit, that is pretty much Brawl's fault. It's obvious that the game was divisive among the community, so you can't really place blame on anything else. And your Mario Party analogy was spot on (though I have thoroughly enjoyed the series since my introduction to Mario Party 4). I had forgotten about the money in tournaments. Yes, people get riled up when money's on the line.
I think the main thing to remember is that Smash Bros. was never INTENDED to be a serious competitive game. The original game, including its origin concept as Dragon King, was specifically designed away from the typical 2D fighter like Street Fighter. For one thing, four player free for all. Another thing, items. Smash Bros. is by definition a fighting game, but it's designed and has been referred to as a party game. Melee and Brawl keep this same spirit and I expect both versions of Smash 4 to follow suit.
You're right that any game can be made competitive, but I think the problem was that Melee made it EASY. As far as I know, the whole Smash as a competitive fighting game never happened with the original 64 until after Melee had already been doing it. In that transition, Sakurai added/left in advanced techniques that made people realize that it could be a deep game. In any case, it was not INTENDED to happen, but the fanbase made it happen.
At this point, we can only speculate what happened between Melee and Brawl. Sakurai may have regretted Melee because fans turned it into another generic fighter. He also had just quit HAL a few years ago because of people expecting him to make sequels. It's possible he could have used that to justify making changes from Melee. Or maybe Sakurai was okay with the fans playing how they wanted, but Nintendo forced him to make changes. There was an article in the aftermath of EVO about Nintendo considering Smash Bros. "a dangerous brand". Nintendo might not have been comfortable about having Smash viewed along the same lines as Street Fighter or Mortal Kombat.
A third possibility is that Game Arts ****ed things up. They supposedly put hundreds of hours into Melee, so you'd think that they would be suited for the job, right? Well, apparently not. Recent evidence indicates that Brawl was poorly programmed and someone in this topic already pointed out a programming error/glitch used as an advanced technique. We'll never know how the game would have turned out had the programming been just as polished as Melee's.
No matter what the case, Sakurai ended up taking the blame. Now, while that could mean he is at fault, I think it could tie in with the Japanese pride you mentioned earlier. Even if Nintendo or Game Arts caused these things, it's Sakurai's game, through and through. He was ultimately the final say and holds himself responsible for each game's achievements and faults.
As for recent events... EVO, in my opinion, was due to some severe oversight. It sounds like no one tried to inform Nintendo that they were going to have Melee at the event and stream it until the last minute. At that point, Nintendo's PR department was caught offguard and gave a knee jerk reaction: "Wait, what? No! Absolutely not! We never gave authorization for this! You get Melee out of there now!" Of course, the EVO rep was able to persuade them to allow the event and just not the streaming, but the reaction was justifiable. Someone from EVO should have contacted Nintendo's PR when it was clear that Melee was going to win to ensure that they had permission. As you can imagine, this type of request would have to go through various channels so it can be approved. In hindsight, a flat-out denial may not have been the best word choice. Something along the lines of "This has to go through the proper channels and it may not get through on such short notice, but we can try" may have been a better response. Again, though, it was a knee jerk reaction, so perfectly understandable.
Of course, the fans didn't think so and they made their voice heard. Loud enough that a higher up heard them and allowed complete approval for the event and the stream. In the end, this miscommunication may be a blessing in disguise. Nintendo now clearly realizes that the fans WANT Smash to be recognized at these events. The fact that they're talking with MLG proactively about Smash 4 being there is a sign that they may be willing to step up their support.
I had more to say regarding your other thoughts, but it's late so I'll stop here and follow up in the morning.