I've had that problems on the computer I had before this one.
I used Nvidia Geoforce 4200 Titan and Tv-out.
I also had blinking blocks instead of text when logging out of X, among other problems.
My solution on that computer was to use nvidia driver prior to v4363 (April 2003), and kernel 2.4.x. Then everything worked fine.
Now however, on my new computer, the problem is back.
Now my card is a Gainward Geoforce 5700 LE, and X won't even start with the v4363 nvidia driver, but works with newest drivers, with the graphic bugs back.
Maybe time that Nvidia make something about these conflicts with Vesa drivers?
Ps. The Xconf I use and have been using is... http://188.8.131.52/xorg.conf
on Slackware 10.0.