Thanks boom! Wow, how embarassing. Back to the drawing board...
I've decided to test all joysticks using this program to make sure they give full range values in all directions:
http://s9.zetaboards.com/Nintendo_64_Forever/topic/7349580/1/#new
It's an extremely useful program, and tonight is the first time I've used it.
The following images were made using the part of the program that maps the max distance of the 8 corners of the octagonal gate:
Here's a graph of one of my sticks that I've been working on tonight (notice the shortness in the diagonals, especially the top right... this one still needs work):
(EDIT: I'm going to intall a special configuration tool for MESS for my controller, because this reading may be off. I seem to get full range in the top right diagonal on the console... the edge jump works by mashing NE.)
here is a hori mini pad. (EDIT: this may be a defective controller. not my pic)
here is a brand new original controller: (not my pic)
here is a gc/wii stick wired into a controller with an ATMEGA8: (not my pic)
EDIT: of the 50 or 60+ controllers I have, this is the best OEM joystick with < 1/8" deadzone at the top of the stick:
I thought it would have gotten a much better reading than that.
Check out the n64 Forever thread above for more details on the program, and for other joystick examples. Special thanks to Sanni from N64 Forever for writing that program. If you'd like to use this program yourself, pm me and I'll walk you through it (if you have Windows). You need to use the MESS emulator. Or a flashcart...
I'm going to put orders on hold for now, maybe about a week, until I can jig these up for max range. I'm waiting on wii controllers in the mail anyway. Btw, if superboom were a paying customer, he could return that controller for a full refund or exchange. He would have been out only the $5.xx dollars to ship it back because I would have refunded the original shipping cost too. the other controllers I tested tonight do happen to pass the edge jump test. But that's irrelevant, because quality control is everything. I don't want any more range deficits passing unnoticed.
Thanks for the patience, and I'll be sure to keep you guys posted. Yes, I am going to send boom another stick once I get it all worked out.