My current theory is that the Space Pirate heirarchy is based on the strongest ruling over the weak. Ridley was originally just a particularly powerful mercenary/serial killer that the Space Pirates hired/recruited, and his natural talent and prowess at killing things and raiding planets elevated a simple mercenary into the leader of the most terrible force in the galaxy. Ridley never fought tooth or nail for the job, he was just naturally suited for it.
So when the more powerful Mother Brain came into the picture, Ridley's lack of ambition for leadership made it easy for him to simply follow orders once more. Ridley's a murderous, bloodthirsty beast first, and a cunning, intelligent leader second.
I think there is a difference between Mother Brain and Dark Samus and why they took control.
The Space Pirates presumably found Mother Brain on Zebes and decided to use it because of its superior intellect to plan and organize the Zebes operation. it WAS a super-powerful supercomputer created by the Chozo, so no doubt they wanted to use it while they could. Dark Samus, on the other hand, more or less absorbed all of the pirates' phazon and bullied them into submission through phazon corruption.
But moral of the story is, pretty sure Mother Brain was seated as the leader because it was more intelligent by design and was able to control Metroids.
EDIT: Fact checking. Unsure if Mother Brain is able to explicitly control the actions of Ridley or Kraid or if it just hightens their aggression, but Space Pirates are definitely susceptible to Mother Brain mind controlling them. Which is presumably why they didn't take Mother Brain with them when they left Zebes, and also why they rebuilt Tourian when they returned.
