Sakurai can see more than you think. He could tell that Mega Man, Ridley (mainly a Western request), and Geno were popular without a ballot, and I don't think he needed a ballot to see that K. Rool and Banjo were popular either. A lot of characters who he claims to have gotten in because of the ballot should've been obvious.
I think he had a ballot substantiate all those characters. Maybe not Geno, but the timing of being "considered" for Brawl would line up with receiving the results from the Brawl poll. However, Mega Man's inclusion was owing to his placement as second-most requested third-party (behind Sonic) on the (Japan-only) Brawl poll. That much has been admitted by Sakurai.
Then after several games of nothing, almost immediately after the ballot K. Rool receives a costume then a character in the game shaped largely by the ballot (and that's if we're assuming the NoA source that claimed they were taken aback by K. Rool's popularity is untrue). You would certainly be able to make a compelling case that the ballot was the impetus.
Then with Banjo, maybe a ballot wasn't necessary, but given Sakurai used the ballot for Ultimate, then included Banjo, then compared Banjo to K. Rool and Ridley, two characters who also have ties to the ballot, I'm sure it played a part as well. There was no time period post-buyout in which Sakurai could observe Banjo's popularity in which he didn't already have ballot data, given it only reignited when the ballot began.
Personally I'm not sure if Sakurai sees all we think he does. It's hard to tell. Sometimes he seems blindsided until presented with tangible evidence, but then there are cases like Mewtwo where despite the lack of some census, Sakurai still knew the character was popular. Though perhaps not until mid-way through developing Smash 4, even though he had been popular ever since being cut.