Nah, this is a bad idea. The dice aren't meant to be a sign of consistent accomplishment, but a sign of accomplishment in a game. By saying they can be removed for bad performance, it's changing them from a sign of winning an MVP into some kind of rating system. The dice are fine as they are: A signifier of an outstanding performance in a given game. No matter what they do in the future, they will always hold onto their mvp from the game and/or games they earned them from, and as such the dice will always hold the same meaning: that at some point in their Dgames career, they won an MVP.
Not to mention, how exactly does one define a "horrendous game"? Inactivity? Most everyone has, or will, run into real life problems that cause their activity to falter or fail, and this isn't a sign of a bad player but of the fact that this is a game and real life takes precedence when it comes to a choice. Being completely wrong about scum? Everyone's done it before. CC'ing dayvig? Admittedly it's stupid, but everyone makes a stupid mistake sometimes. There is no way to define a "bad" game like there is to define an MVP performance. Quite frankly, a "bad" game comes down to things people don't like. This idea has nothing going for it, and causes more problems than it solves.