I have the same problem
I'm seeing the same problem with my GeForce 3. I think that
it might be AMD related, although I'm not certain.
I happen to dual boot this system, Mandrake 9.1 and Windows
XP and the problem occurs in both, so it is not linux specific.
I have found that I can only use the 3123 Linux drivers.
Otherwise either X takes forever to start or the colors are
garbled. Unfortunately, it was a real pain in the ass to back
out NVidia's latest drivers because I couldn't find an uninstall
option with their fancy new installer. I did manage to get
it working again, but GLX doesn't work anymore because the
installer messed up some libraries. I'm sure there is any
easy fix, though.
My Config is thus:
Athlon XP 1700+
18.1" digital flat panel
Also, you need to make sure you have a pretty recent linux
kernel as there are AMD/AGP issues in older ones. Redhat 9
and Mandrake 9.1 contain these fixes and there are RPM updates
for older versions of both. There are probably updates for all
the other flavors as well.
In closing, I would try and back out the latest drivers and go
with 3123. These are the last drivers from NVidia that have
worked for me.