I have a GF3 Ti 200 running in a machine with a Jetway V266A motherboard and Athlon XP 1800+, 256MB DDR2100 RAM, running a Linus Linux 2.4.20 kernel.
I too get the slow start-up. I've diagnosed the following:
Driver 1.0-3123: System halts for 30 seconds on X start-up.
Driver 1.0-4191: System halts twice during X start-up, each halt is for approximately 2 minutes.
During both these halt periods the system is TOTALLY unresponsive (no ping etc.).
As you can imagine, the behavour of 3123 is just about acceptable as the machine isn't being used as a server and doesn't have other people using it interactively. However, the behavour of the 4191 driver isn't. I've tried using both the kernel agpgart and the Nvidia built-in version and it make no difference.
If necessary I can upload the XFree86 log file (with annotation showing where the halts appear) and the XF86Config-4 file.