Bold proves my point. Either way, its not the characters driving sales is it (remember that software pushes hardware). Also, Smash Wii U sold less than NSMBU, Mario 3D Land and Mario Kart 8. Animal Crossing and Pokemon outsold Smash on the 3DS by a lot.
The DLC did better than Nintendo had done in the past but it was overshowed in 2017. Compare download revenue in 2015 and 2017. Nintendo made more money off the Breath of the Wild and Xenoblade 2 Passes than they did with Smash and Mario Kart 8.
There is no good business reason for guest characters in Smash.
These seem like flawed comparisons, particularly the part about Nintendo making more money off of Breath of the Wild and Xenoblade season passes compared to Smash and Mario Kart 8 DLC. The season passes are about four or five times more expensive than individual Smash characters or stages (it reminds me of the claim made in the early days of the Gamecube that
Luigi's Mansion made more money in its first day than the first Harry Potter movie made on opening weekend while ignoring the fact that Harry Potter tickets didn't cost $50 each), and they're on a console that is SIGNIFICANTLY more successful. (the Switch surpassed the Wii U's lifetime sales in its first year)
Launch titles like New Super Mario Bros. U make for an unfair comparison (many of them sell extremely well because they're almost no competition at the time - it's a big reason why games like Super Bomberman R and Ultra Street Fighter II sold as well as they did), and assuming you didn't make a typo, you're comparing a Wii U game's sales to the sales of Super Mario 3D Land, a 3DS game, where there's a massive difference in terms of install base. (plus Pokemon and Animal Crossing are much more casual games than Smash, so they're going to sell much better - it's part of what made Wii Fit, Tetris, the Sims, and Minecraft some of the best-selling franchises of all time)
---
As for your last point, there are plenty of good reasons. If a Venn Diagram was made of Smash fans and fans of specific Nintendo properties (Mario, Metroid, Animal Crossing, etc.), there would be plenty of overlap - anyone who's interested in one of those franchises would have likely played Smash Bros. at some point. The same couldn't be said for fans of franchises like Final Fantasy or Castlevania - they're franchises on classic Nintendo consoles, but nowadays, they're on just about everything. There wouldn't be as much overlap, so fans who moved to other consoles would see those characters in Smash, and that would get them interested in the game. (especially if it's a series like Mega Man that had been neglected for a few years)
The Switch's popularity also means that a lot of third party titles (every series in Smash aside from the Konami ones) have made their way to the system, so if someone likes Final Fantasy, plays Smash, and wants to learn more about Ryu, then they could get the Street Fighter anniversary collection.
It seems like a win-win - both Nintendo and the third party companies that are involved attract a broader audience. (plus, it's not like there are a ton of third party characters in the game - there's only nine of them, and that's spread out between eight different series)