Actually, Mother Brain did appear in Metroid Prime 3, sorta. I mean, Mother Brain is just another Aurora Unit like the ones Samus communicates with in the game.
The reason why there's been a lot more focus on Ridley is because he's been established as the leader of the space pirates (who just happend to be the main foes for 90% of the Metroid games). I mean yeah, in the first few games he and Kraid were just bosses, but as the Metroid series went on, new stories and lore were developed that just happened to make Ridley more important. He just happens to be a truly major character now, while Kraid and Mother Brain specifically had already served their general purposes. Admittedly, it's interesting to note that they both appeared at the beginning and near the end of the Metroid timeline of games (considering that Metroid is the first and Super Metroid is the second to last).
As far as Smash is concerned, all three big enemies have been used in the backgrounds of stages. Ridley in Zebes (SSB 64), Mother Brain in Brinstar (SSBM), and Kraid in Brinstar Depths (SSBM). Out of all those, Ridley just seems like the logical choice because a) I could actually imagine him as playable and b) he hasn't appeared in Smash since SSB 64.
Who would you rather have be a playable character in Brawl? Dark Samus? That's a reasonable choice considering that Dark Samus is Metroid Prime and Metroid Prime is like the Mother Brain of the MP trilogy. However, Ridley just seems more like a shoo-in for Smash because he's been in more games, has been there since the very beginning, and has been established as Samus' true nemesis.