Technically, aren't games just a waste of time at the end? Anyway that's something I personally disagree with as I never felt that my time was being wasted. The swapping never bothered me, there were always enough barrels so that swapping isn't an issue, except for maybe the toy factory.
Here's the thing though, a lot of people dislike the the choice, fair enough, but I don't think it's not enough to call something outright bad. Maybe it's not for you, that's fine, but that doesn't mean there aren't people who like it. I think it's just being blown way out of proportion. The game still has the usual Rare polish. It has a great soundtrack, controls well enough, not a glitchy mess, graphics are alright, the personality is there, etc.
Stuff like Sonic 06 is bad. Most movie tie-ins are bad. DK64, at worst, is mediocre. It does too many things right for it to be just dismissed as a bad game.
That's my take on it.