I've been having problems with the combination of rt kernels and nvidia drivers for years, both on 64bit and 32bit. When I need to do rt stuff on my box with an NVidia gpu, I have to use the vesa drivers, as the nvidia driver locks the system, and the nouveau driver just gives me a blank screen. I seem to have a fairly cantankerous NVidia chip - it's a GeForce 6150 LE.
The 2.6.31-rt and 2.6.33-rt kernels actually worked for me with the NVidia binary driver, but the problem re-surfaced with 3.0-rt. We are by no means the only ones who have this problem, as is obvious here:
The problem remains with 3.4-rt, though thankfully it is no longer necessary to patch the NVidia driver to get it to build. Undoubtedly, the rt-preempt patch is changing something that breaks the driver for a significant minority of systems. But, it's hard to say what is really going on.