One thing to note though, in terms of mplayer I can reproduce the problem with -vo x11 and caca output. So I imagine this is not an Xv specific issue.
My wild guess is a mix of X + Composite issue.
Perhaps if possible I'll see if i can reproduce the same issue with a mininal X app.
I'm also curious if any of the NVIDIA guys has any comments with this issue. (Which so far they haven't replied