I had the exact same problem when trying to use the source rpm's, so instead, I used the tarballs. They worked the way they are supposed to. I have X up and running now, with only one minor problem with my mouse (it doesn't work). Get the tarball for the display driver from this link:
Then just install according to the instructions NVIDIA supplies.
Hope this helps!