You ever gone to a party, or even a hang out, or any event, really, where someone you didn't like was there? They probably didn't ruin the party, that'd be a bit extreme, but if they weren't there, you'd just prefer it. Is that selfish? Idk, maybe; it's human nature, and it's not like the party is ruined either way. More importantly, it's not like you wish unhappiness on those that like that one person.
People can dislike what they dislike, y'all need to be strong enough not to make everything a sweeping personal attack when you hear a dissenting opinion over a character's presence. This site really shames people for not liking stuff, it's insane. Just pretend it's a stage or game mode or something, people don't deride each other over not liking those, even though it's the same principle.