Lots of competitive players and Japan in general really wanted Roy back in Smash. That's a huge part of the Smash market right there.
I'm not saying there wasn't reason to think Roy would be successful. However, you would have to concede that Roy won't be very close to being as successful as Ryu or Mewtwo. There are a lot of people, like myself, who don't think Roy is interesting enough for $5. Nintendo had to have known this, but went for it anyway.
My point is, Banjo and Kazooie would be WAY more successful than Roy, and yet Nintendo still made Roy.
Because they'd have to renegotiate the contract with Microsoft with every successive game. We already saw Snake get dropped, you think they would bother with Banjo?
You're thinking with far too of a western mindset. Banjo has fairly lukewarm popularity in Japan. He wouldn't make waves in that market.
Are you seriously suggesting that Sonic being in Brawl made it difficult to get him in Smash 4? That Pac-Man and Mega Man being in MSash 4 will make it hard for them to get into smash 5?
Firstly, the contract will only be difficult the first time. The second time, everyone knows pretty much how things are going to turn out, so they just model the contract after the first one.
Secondly your reasoning leads to an infinite loop of never including 3rd party characters because that somehow makes them easier to include in the next game. At which you put it off till the next game for the same reasons. And so on.
There are way more westerners than Japanese people.