Well here's the thing... An easy mode doesn't affect me, but that's because easier difficulty modes are not the draws of any game. If I see a game is too daunting for my liking, I just wont play it until I feel I'm up for it. It's why I don't go into horror games like Resident Evil. I'm a chicken **** when it comes for horror and I'm not gonna demand that they add a sunshine and rainbows option. What you really want to be looking at is good difficulty curves, because that's where these games can fall apart through their gameplay.
Take Super Meat Boy, in which you're given short hard levels. You're gonna die over a 1000 times in that game, but it'll never feel like that because the levels are short and they're designed to gradually build up to the harder set pieces.
Also what do you define as fun? There are multiple genres of games, multiple consoles to choose from, multiple developers that suit different niches, etc. Some times I need a game that teaches me more about my abilities as a gamer and sometimes I like a casual stroll through my Animal Crossing village. Again, it's why I don't go into horror. It's some peoples cup of tea, but it ain't mine, and that's fine.