I think those freezes are related to power management, yesterday I was playing ut2k4 and Xorg memory usage didn't increase a bit, so it is not 3d related, even if I leave computer without any programs running (just desktop) nothing bad happens either. So the only thing left might be because of monitor power management (standby / suspend / power off feature), because this increases starts over night. The only thing different is that graphic card pushes monitor into standby after 40 min (40 min standby, 50 min suspend, 60 min power off).
Can I test this with:
xset dpms force off
or it doesn't do the same thing as leaving computer to automatically power off display?