the problem will modt likely have nothing to do with the nvidia drivers, perhaps your system has to many different GLX packages.
try uninstalling nvidia, uninstalling all gl rpm stuff and then installing nvidia again.
remember if you uninstall nvidia, you have to do a ldconfig, and better do that after uninstalling all gl rpm's as well (installing nvidia does that automatically).
if you're stuck in runlevel 3:
you can do
rpm -qa | grep gl
rpm -e <package-name>
for all those mesa and other gl rpms.
you didn't have any gl stuff sources built, did you?
try building rpms instead:
rpm -tb <tarball.gz> --clean --sign