I've also had this error every once and a while. I have a Geforce 4 4200 Ti and a ViewSonic VG191 (1280x1024, 60hz) and am using the DVI interface. The error didn't occur very often, however, so I've since forgotten why they were there and how I made them go away. More recently, though, I've been having problems where some pixels seem to be leaking to the right and left... that is, scattered around in the video are rows of about 10 or 20 pixels of the same color, centered on a pixel which really is that color. For example, if the video was all black with a thin vertical white line running down the center there would be (seemingly randomly placed) white horizontal lines of about 20 pixels centered on that white line.
Anyway, I'm not sure why I'm getting these kind of jaggies instead of the old kind of jaggies that I used to get, but the conditions are exactly the same... that is, it only happens with hardware video overlay (and not in 3d or anything) and only once I size the video up to about full screen. The only difference between the situations that you guys are having and that I am having is that my jaggies have always occurred at 60 Hz (why would you care about higher refresh rates on an LCD, anyway?).
It was easily repeatable and, in fact, I couldn't figure out how to get rid of them. I booted into windows and noticed THE EXACT SAME PROBLEM with windows mediaplayer at full screen. So, there's nothing linux-specific about it.
I recently hooked up my analog monitor and started using twinview on clone to see if my analog monitor would show the jaggies too (I wasn't sure what the problem was at that point), and, for some reason, that seems to have fixed the problem. Neither display exhibits either kind of jaggies! It may be that the twinview mode forces that aspect scaling mode... who knows.
Also, because you guys were mentioning the problems having to do with refresh rate, I just checked, and my LCD is now running at 75 Hz. So, maybe that's what did it.
I'm sure NVidia will start looking into this sooner or later (since it's a problem on windows, too), but at the moment it's such a minor issue that I don't really care too much -- the way I have it set up now works fine without any jaggies... and it sounds like that aspect-scaling fix works well, too. It *would* be useful for them to put that in the FAQ somewhere, though.