Yeah, but then you would lose the match if you make that mistake.
So you can't stall and you can't make the match unfinishable. If you think about it, your opponent's ability to Smash DI means that if you start hitting them while they're in the glitch, they can maneuver themselves out of range of grabbing and FORCE you to lose the entire match. So basically, you can freeze glitch somebody, but you can't hit them while they're inside of it, and you can't NOT hit them because that would constitute stalling. The only logical course of action would be to grab them again and combo them from that, which would remove them from the freeze glitch.
So... here's the line we walk: if we have an infinite that can't be escaped, and it's permitted as long as it doesn't stall, we should be permitted to use the glitch provided it isn't used to stall. However, if using the glitch makes the match unfinishable, the glitcher loses the match. Since the only way to obey both those stipulations is to re-grab and remove them from the glitch, what we're left with is a very high risk, very low reward technique that's really hard to do and doesn't benefit you any more than the various other combos at your disposal.
All of that is fine by me because, again, the freeze glitch has very little use in competitive play. It jeopardizes your chance to win even when it isn't banned. In my game, I prefer to make every opening count, and wasting it on an attempted glitch that would have very low payout is silly.
There is one exception, however; if you grab somebody out of the freeze glitch, you can infinite them without any time wasted at start up, meaning that if I could somehow glitch you at zero you couldn't escape the infinite... AT ZERO PERCENT. So if the freeze glitch wasn't banned, I might use it as a risky set-up for the infinite at very very low percents.
We don't need to ban the freeze glitch to maintain variety in the game. We can place decent restrictions on it to keep it from being broken, and if it doesn't ruin the competitive nature of the game, a ban simply isn't warranted. No matter how stupid or "gay" (why is the Smash community so obsessed with that **** word to describe things we don't like?) something is, banning it is not called for unless it ruins the game.
Again though, like I said... with a rare exception, freeze glitching is pointless, dangerous, and difficult. The restrictions, which are completely reasonable, make it pretty much useless.