The limit (at least with cBliss, I can't speak for ACE though I imagine it's similar) comes from the way the cosmetics are named, specifically, the battle portraits. Mario's first portrait is numbered 000, his next one is 001, all the way up to 009 (or, like, 005 or 006 in vBrawl). Donkey Kong's starts at 010, so if they gave Mario an eleventh costume, it would overlap with Donkey Kong's battle portrait. If you gave Donkey Kong another costume, it would intersect with the next character, who starts at 020 (Link, I think.)
Wario is special, because he had more allotted battle portraits from the beginning since he had 12 costumes anyway, but every other character follows the naming pattern above. PMBR could, theoretically, change it, but I doubt they would bother renaming and writing code to repoint at 460+ files.