When I painted my buttons, I sanded mine down. I did nothing to the controller's walls for it.
For my first controller, it was perfectly fine because I actually followed directions on the can in terms of distances sprayed.
I went through about 2 sets of buttons for the second one because I didn't follow the rules. If I learned anything from spraying buttons, it's this:
- Sand the buttons. Don't do anything serious though.
- Spray a LIGHT coat.
- Spray at the distance it says on the can.
- If you care about the rubbers, immediately clean it off after the time to be handled has passed. Since it's paint on rubber, it should come off fairly easily.
- Reassemble the controller about 3 days after, making sure the clear coat has dried.