I am giving up. After trying the latest drivers, the problem has just worsened. If you can't beat 'em, join 'em. My fiance just left to pick me up a GeForce-4 MX440. Please tell me that this card is not affected by this "slow X startup" crap. I have read all of the posts in this thread and did not see any GeForce-4 users complaining.
I'll let you know if it works.
P.S. In Columbus, Ohio the GeForce-4 MX420's are going for $79 and the MX440's are $119. I hope it's worth it.