I finally got everything working. When I had been using the gentoo ebuilds they somehow hadn't emerge the latest nvidia-glx package (which was masked for some reason). I forced that to emerge and then the nvidia-kernel and everything is working now. I can only assume the older glx package was giving me problems.
My module is bigger than yours, weird, but now it's working.
blackbox root # modinfo nvidia
vermagic: 2.6.0-test5 SMP preempt PENTIUM4 gcc-3.2
For those who have problems on gentoo:
ACCEPT_KEYWORDS="~x86" emerge /usr/portage/media-video/nvidia-glx/ nvidia-glx-1.0.4496.ebuild
That will also pull the 4496 nvidia-kernel and patch automatically (gentoo rocks).
Thanks everyone for the help.