Re: VDPAU video surface destroy takes too long
If your window is going to be the same size each time, there's no need to destroy the window or presentation queue at all, and probably not the output surfaces either. This should avoid the green flash.
If your video streams are the same size, there's no need to destroy the video mixer or video surfaces.
Still, I can see how it might be simpler to tear everything down again and start from scratch.
In that case, setting the presentation queue background color to approximately black will make the flash black instead of green. The background color alpha isn't used. What's happening is that the background color is actually the overlay chroma key color, so you may just be seeing parts of the overlay showing through, when something the same color is drawn over the top of the window.
If that's not the issue, could you post a screenshot or photo of the issue? A small repro case might be useful too, if the overlay chroma key explanation isn't the problem.