You won't be charging up any dsmashes when there's a trip sapling directly under you while you're being bombarded with lloyds and pellets and all your projectiles are being pocketed. At which point did everyone start ignoring 3/4 of what makes this strat so good? Exploding balloons are hardly a problem and we all know this, it's the rest of Villager's customs and moves in conjunction with it that makes his ledgestalling so difficult to deal with.
A player who isn't just a test dummy will think and actually do things and he certainly won't brainlessly let you charge up moves on the ledge. He'll either time his stalling properly and shoot you off, cover himself with lloyd / trip sapling / both, or explode the balloons earlier so that only you take damage. The only ways to beat him are to retain the percent lead forcing him to approach, outcamp him (good luck with that one though) or have a dangerous enough offstage game to take his stock in one move, although even that is risky and requires a mistake on Villager's part.
I'm assuming you're paying attention while breaking the stall. When does the trip sapling get planted and why didn't you contest it? When it wears off, how is Villager re-planting it? Villager has to stand on the stage and commit to a long animation to plant a sapling; stalling on the ledge does nothing to enable this. When Villager is starting EBT from below the plane of the stage, how is he using Lloid or the slingshot? If Villager jumps above the plane of the stage, how does the opponent not perfectly predict that either Lloid or slingshot is coming and just powershield or do anything else that stuffs the telegraphed option? Villager has zero other options if he moves into position to slingshot or Lloid.
Also at this point you're basically arguing that if Villager plays a full, dynamic keep-away game he can succeed. Uh, sure. What's the problem? Villager is designed from the ground up to be a defensive character. There's nothing wrong with this; gameplay diversity is about all types of characters. He happens to be the best of the defensive characters (this is true even if you ban customs, incidentally). If he is doing a variety of skillful defensive zoning actions, many of which involve substantial on-stage gameplay, it just doesn't count as ledge stalling.
Honestly it's really hard to win with him since for all of his strengths his weaknesses are quite large too (can easily get stuck with poorly allocated resources, has substantial kill move trouble). Villager's whole game involves being a step ahead of his opponent, having the sapling ready and well places, being at the right height to use the right zoning move, doing the below the stage EBTs when the opponent isn't in position to just charge down smash. There is no algorithmic tactic that produces easy wins for Villager; he has to outwit his opponent to stand a chance.
Let's do a walk-through of the following common situation. Villager has just grabbed the ledge and is running 1322 as a custom set. What do you do?
1. Is a Timber Counter sapling down and near the ledge he's holding? If yes, stay somewhat near the ledge, powershield any projectiles (from the ledge, the timing is really predictable so this is not hard), and wait out the lifetime of the sapling. Stay close enough to fsmash on reaction the planting of a new one if Villager tries to be in position to plant a new one right as the current one expires. If no or if you've now waited it out and Villager is still on the ledge move to step two.
2. Watch how Villager moves off the ledge.
a. If Villager drops low, he has no options but to use EBT to recover to the ledge. Simply walk over to the ledge and start charging dsmash for easy money. Do watch if previous balloons are in the way, but their trajectories are random so eventually they won't be well positioned. Villager exploding the balloons early prevents ledge grabs so this won't protect him.
b. If Villager pops high, he's looking to fire a projectile. His only two options are Pushy Lloid or the slingshot. The slingshot will come at a super predictable timing and is easily powershielded. Lloid is more imposing but also more committed; you can generally roll past it on reaction. Even if your reactions are poor, you input a powershield timing shield for the slingshot and hit toward the ledge to roll when you see the rocket so no guessing required. Now Villager has no defenses between you and him and is falling to the ledge which he can grab with zero invincibility. Do some kind of offensive mix-up here (or punish the grab if you have time which will sometimes happen), and you're massively favored.
If you follow this, you see that waiting on the ledge only pays off for Villager if you either stand too far back (and thus let him plant saplings uncontested which you should never do) or if you get impatient and run into his stuff. He has no way to actually do damage to you and is stuck in a super option limited position. Smarter Villager players will only use the ledge as necessary to escape other undesirable pressure situations and will mostly try to control the stage and play a more dynamic zoning game... which is "playing the game" and is not abusive. Trying to outcamp him is mostly a poor strategy anyway; sure if you have a lead don't approach without good reason. In general though, Villager suffocates under strong offensive pressure; he needs time and space to set up his game.
I do want to be clear that I don't disagree with the assertion "Villager is a generally good character who has a very defensive playstyle" since that's true. I disagree with assertions that he's inherently abusive since to me an abusive character must play in a simple or algorithmic way which Villager just can't do and still win games. When we start talking about a character who spends a lot of the time on the stage and who uses a wide variety of moves to appropriately respond to situations or to dynamically set up complex spatial control, sure, this is a good character, but it's not what I'm arguing against since by definition a Villager who does all of this is doing a lot more than ledge stalling.