Re: Why can't we have composite with overlay?
I'm assuming you're talking about VDPAU.
What Dragoran says is true; overlay and redirected (i.e. composited) windows simply don't make logical sense together.
VDPAU currently prohibits overlay use whenever the X composite extension is enabled, irrespective of whether the VDPAU presentation window is actually redirected. This is because windows can be dynamically redirected/unredirected, and VDPAU can't dynamically switch between using the overlay and blitting. Having the X composite extension enabled is the default, I believe, in recent times. Consequently, one has to manually configure systems to disable the X composite extension to get the overlay presentation queue. We're putting some thought into removing this restriction, so that overlay can be used when the X composite extension is enabled, but the window is not currently redirected (i.e. compiz/desktop effects are not enabled). That said, I'm not sure exactly how useful that'll be, since I think full compiz is also the default in many cases, which would still prevent overlay use.
Also, the way X protocols are currently defined, there's no way for an application to say "I'm done rendering a frame", and synchronize with the composite manager before continuing to render. This means that sometimes the composite manager may only start reading the window's content after the application has started drawing the next frame, which will lead to tearing. Depending on your exact HW/SW configuration, I believe it's possible for NVIDIA GL's sync-to-vblank (i.e. compiz's rendering) to tear too.