I've got the same problems as you have but I cannot test if mine is hardware related or not
(I don't have windows and don't have the opportunity to install it).
My config :
Chaintech GeForce 4 Ti 4200 64M
Athlon XP 1800
768M DDR SDRAM
I compiled two kernel
no mtrr, no agp, no rtc
mtrr, no agp, rtc
My bios settings are the following :
Everything default except :
Video Memory Cache Mode : UC
XFree settings :
Default + Use of Nvidia AGP
For booting I append this line to lilo : pci=biosirq but doesn't have any effect. As you can see :
nicoe@smarties ~ % grep nvidia /proc/interrupts
11: 222046 XT-PIC usb-uhci, nvidia
The strange thing is that with the kernel 2.4.19, AGP is disabled but I can play (bzflag) for hours without problems.
With the 2.4.20 kernel, AGP is enabled but as soon as I play a little (20-30s or so) -> Freeze.
But I can launch glxgears without any problems.