I don't think Yoshi is technically difficult, unless you want to really try implementing some niche stuff into your gameplay.
What Yoshi is, is unconventional. His double jump, his recovery, the way he moves, heavy yet floatyish, it all adds up to be a package that doesn't quite mesh well with the rest of the cast. Meaning that basics, which carry over really well between most characters, don't carry quite as hard with Yoshi -- he demands more familiarity for success. Characters in a similar boat are Rosy and Olimar.
Yoshi has been normalized a bit, as has the entire cast, with the introduction of this game. But he's still kinda "out there" as far as character feels go. Which means people can't just pick up and play him quite as easily as they could anyone else.