I've tried your solution (keeps my vidcard cool ;-), but problems are remaining.
However I did some research on my own and discovered that this phenomenon is the result of at least 2 different problems:
If your X crashes in use of GL: overclocking problem (nvclock fixes it)
Lockup in Firefox (opening new tab) and/or kcontrol: causes nvrm (nvidia agp driver) to hang and because this driver has high priority everything hangs except the mouse cursor, which has the same priority.
(In my case dmesg reveales: NVRM: Xid: 13, 0000 02005600 00000056 00000c28 00230195 00000080)
Solution: Disable "RenderAccel" in your xorg.conf (#Option "RenderAccel" "True") and everything works just fine!