I encountered an error with the nvidia drivers (currently 256.35 but also with the previous version after a downgrade).
As far as I could discern this, X will freeze, if the card has a little bit too much to do. For example starting a game through Wine, starting VirtualBox or just switching between Metacity and Compiz (although, I can use Compiz as default window manager without freezing).
The log files will fill with something according to the following (more to see in the bug report)
Jul 16 17:35:06 antares kernel: [ 1180.628088] NVRM: Xid (0002:00): 8, Channel 00000004
Jul 17 16:09:32 antares kernel: [13589.026019] NVRM: RmInitAdapter failed! 0x12:0x2b:1671)
Jul 17 16:09:32 antares kernel: [13589.026029] NVRM: rm_init_adapter(0) failed
Thanks in advance for any hints, help or pointers in the right direction to get things running again.