You kind of contradicted yourself here. Let's take a look:
"There's a reason quite a few companies would love to have their character in Smash. Companies tweeted to ask for players to vote for their own IP's in the Smash Ballot..... It's the ultimate way to promote their IP's"
Which has nothing to do with how Directs are done. Nindies have nothing to do with Smash itself, which is a regular Nintendo product. So they keep it out of it because it's such a massively known series that people would barely care about the rest. It's not contradictory. They're just different situations. Seeing Smash alongside your game will not promote it. Seeing your game content in Smash will. You're acting like they are the same thing. Smash overshadows tons of games due to how huge it is in reality.
Why? Why do companies want their characters in Smash? Because it makes people aware and promotes their game, as you stated. But this goes completely against the point you just tried to make in the second paragraph; if, say, a DQ character had been confirmed for Smash in this direct, that quote I snipped out says this would draw more people to DQ, but the rest of what you said suggests the opposite.
Except it wouldn't really do much for the actual games within the Direct. Because those were the important part, not just a Smash character. They're trying to draw people to the game in itself. How can they do that if Smash overshadows the rest of the DQ content? Actual DQ fans(not just people who might be interested after seeing good information) are going to buy it anyway. But if you want new viewers, you don't let anything overshadow your current video content. The reality is, major Smash information overshadows any Direct because of its massive influence. This'll continue to happen as is as long as the series exists.
A character getting in Smash will promote their game, but only after people actually play Smash. That's where all the stuff is. The trailer's key point is to appeal to Smash fans first and foremost. Or those who are already fans of the IP the character is from. Cloud's trailer wasn't to promote FFVII. It was to promote Smash Bros. itself by making people hyped for the character in question. Imagine if you aren't already a FF fan. The trailer won't do anything for you. And for good reasons. It's not designed to do so. At best it might explain a thing or two to the FF fans about how Cloud works(in which case it did), but it would not really make anyone care about FFVII either. They're focused on Smash. This is why Smash overshadows other games with its announcements as well. It's because it's not designed to really promote other games, even within the trailers. Appealing to the fans of a franchise is not necessarily promoting the franchise as is. Now, it can be done that way. But it has to be intentional, like Isabelle's reveal was tied into the reveal for AC Switch. But that's pretty much... the only time where they used Smash's own trailer to promote a game without overshadowing it. However, this is also DQ, which is niche in the US but also godly in Japan. It's too risky to try and overshadow it. Even if Erdrick(or Slime, etc.) were revealed, the DQ stuff alone wouldn't be talked about nearly as much. Even the casual Smash fanbase doesn't talk much about stuff other than Smash during directs. It's pretty much always the biggest highlight of a direct as long as any gameplay or a trailer is shown(hence why few cared about the Smash information in the last direct, but it's still heavily talked about as is anyway).
Smash is certainly a huge influencer, but the only people who forget everything else are the ones who only really care for Smash in the first place...which doesn't even make a whole lot of sense in itself, because the most hardcore Smash fans are the ones who really care about the Nintendo cast and, therefore, are big fans of at least some other Nintendo franchises.
The hardcore Smash fans aren't necessarily Nintendo fans. Some just care about the 3rd party options alone, which is the reason they even got interested in it. That's why some are willing to buy a Nintendo system just for Smash. Smash fans are not inherently Nintendo fans anyway. Not when it appeals to more than just Nintendo fans(which is part of why 3rd parties exist, to gather multiple players who don't own Nintendo systems. It's meant to be a gaming crossover at this point, as said by the man himself. A lot changed).
Smash overtook E3 simply because that's how Nintendo planned it. Half of their presentation was Smash (more than any game has ever gotten in a general direct) and their show floor was clearly tailored for Smash. It doesn't have to be that way, though; Isabelle's announcement certainly didn't overtake that direct, and the Smash 4 reveals didn't overtake their directs. In fact, after E3 there were memes and complaints of how Nintendo's presentation was only good if you're a Smash fan (which I, as a huge FE fan, disagree with, but the opinions were there).
Even if Smash wasn't almost the entire focus of E3 it would've still overshadowed it heavily. It had Ridley, for cripes' sake. That was enough alone. It didn't need to have 'Everyone is here". It already had a huge reveal that immediately changed everything. The rest was a bonus at that point, really. Smash Character announcements are pretty much always going to be the biggest deal of any direct, but it's only going to be a big deal in the context of Smash(as in, to be clear, it will only actually promote Smash in itself with people not caring nearly as much what games the character comes from... if they aren't already fans of that series), nothing else. As I said above, it isn't going to help people care about the franchise a character is from just by having an announcement or trailer alone. Only if the game is talked about as well, or promoted alongside of it in a similar manner(like with AC Switch) will it not entirely overshadow it. Though really, Isabelle's reveal easily overshadowed AC Switch's announcement anyway. Which was inevitable.