While his scrawny look didn't fit, to me, with the way he was carrying that huge sword of his around, I feel I can empathize to a certain extent with the people calling him a 'gorilla'.
I just feel his muscles are too much. They're pretty huge. He should have more muscles than Marth, but that build is over the top, I think.
Also, my view may be tainted by the fact that I didn't play any of the games he was in, but I still tried to look at his official art, and while way manlier than his Brawl incarnation, it didn't look like he was on steroids.