The nvidia driver is not compatible with rt kernels by default. I believe the nvidia driver in Ubuntu already has a patch to make this work properly. If you want to use the latest driver, you could try out the one in my Launchpad PPA
. It should also have the patch, because it is directly based off the driver in Jaunty.