Alright! Fixed it!
First though, my vid card temp never got very high. It was warm, but nothing unusual.
The problem was something I should have tried a looong time ago, and I feel a little dumb that I didn't. For some reason I didn't think it would have any reason to break things. I was using TwinView with two monitors. Here is what my XF86Config "Device" section:
Identifier "NVIDIA GeForce 4 (generic)"
VendorName "NVIDIA GeForce 4 (generic)"
BoardName "NVIDIA GeForce 4 (generic)"
Option "NvAGP" "1"
Option "UseEdidFreqs" "true"
Option "TwinView" "true"
Option "MetaModes" "1600x1200, 1600x1200; 1280x1024, 1280x1024; 1024x768, 1024x768; 1024x768, NULL; 800x600, NULL; 640x480, NULL"
I commented out the TwinView and MetaModes options and I play forever. The GFlux screensaver crashes still, but just drops to a window saying the window was destroyed unexpectedly, so I'm presuming theres a bug with GFlux too.
Thanks for the help to everyone who kicked in with ideas...