See the thread titled 'Anyone get smooth video playback with beryl? Is it possible?' that I started earlier.
Sorry for the double post, but I thaught this one was dead as it hadn't been responded to in a month, then the second I create the new thread you replied
In any case user piotrq__ replied with this info:
My config is MUCH slower (in theory ;-) ) than yours, I've got aDuron o/ced to 1800, GF 6800XT and 512 RAM. Video playback is super-smooth for me, I'm using MPlayer from svn and an svn Beryl with latest nvidia driver, and I'm using default -vo xv as my video output driver. I can even apply post-procssing effects via Beryl such as blur, transparency, inverted colours and stuff, wobble the window around, minimize/maximize and the playback will remain just as smooth as without it.
So it seems that our problem is a configuration issue and can be solved.