JBC: Thanks for the very thoughtful suggestions.
I am not sure that it is response time, since problem only occurs within the video frame and does not affect rest of the screen. If you have multiple video streams playing, then problem only occurs in the first one (probably since it uses hardware scaling, while other streams use software scaling).
You might want to check out the original thread to see the extensive collection of troubleshooting that has been done, including some very interesting workarounds which seem to point in the direction of some oddball hardware scalling issues.
The original thread also includes screenshots, so you can get a better picture of what the jaggies look like.
Fedora Core6/Linux 2.6.x
nVidea driver: 9631
Asus P4PE motherboard
Intel P4 2.8GHz
1.5GB PC2700 DDR Kingston
Leadtek Ultra A250 GF4 4600 128MB with VIVO/TVO
WinFast TV2000XP Deluxe
Dell 1901FP 19" LCD
Antec 400W Power Supply