VDPAU referenceFrams vs DPB

[jusst]

Hi,

we are just working on a xine integration of vdpau with a h264 parser independent from ffmpeg. Actually we ran into one point we are not sure about: The VdpPictureInfoH264 doc says that referenceFrames should contain the DPB (Decoded Picture Buffer) as referenced in H264 specs. From my understanding the DPB does not only contain reference frames, but also frames which have already been decoded but not displayed, due to the decoding/display order. Now should these frames also be copied to the referenceFrames array or would just the real referenceFrams from DPB be copied into it?
And related to this another question I am not exactly sure about is: will the decoder output the frames in decoder order (this is what I assume) or in display order (so doing the reordering internally)?

Regards,
Julian

[Stephen Warren]

Originally Posted by jusst

The VdpPictureInfoH264 doc says that referenceFrames should contain the DPB (Decoded Picture Buffer) as referenced in H264 specs. From my understanding the DPB does not only contain reference frames, but also frames which have already been decoded but not displayed, due to the decoding/display order. Now should these frames also be copied to the referenceFrames array or would just the real referenceFrams from DPB be copied into it?

I'll double-check this.

And related to this another question I am not exactly sure about is: will the decoder output the frames in decoder order (this is what I assume) or in display order (so doing the reordering internally)?

The decoder does no re-ordering. It decodes the specified data into the specified surface. The same is true of the mixer. Display re-ordering is solely implemented by the order of calls to the presentation queue's display API, which probably interacts with the order you pass surfaces into mixer render in your application.

[jusst]

Originally Posted by Stephen Warren

I'll double-check this.



The decoder does no re-ordering. It decodes the specified data into the specified surface. The same is true of the mixer. Display re-ordering is solely implemented by the order of calls to the presentation queue's display API, which probably interacts with the order you pass surfaces into mixer render in your application.

Ok, thank you so far. Knowing that the decoder does no reordering at all makes some things a bit more clear (c: - I was used to CoreAVC windows dlls which do the reordering internally...

[Stephen Warren]

The DPB table might be spec'd as also including non-reference pictures that haven't been displayed yet, but we only need the reference pictures. Any frame marked as a reference must go into the DPB table and stay there until it is no longer a reference. It will often stay there for long after it's displayed. In ffmpeg I think this is the union of the long-term and short-term reference lists.

Hope that helps.

[jusst]

Another question:
What can cause VdpDecoderRender to return VDP_STATUS_INVALID_HANDLE (3)?
Might this be caused by a bad buffer or PictureInfo?

[Stephen Warren]

Originally Posted by jusst

What can cause VdpDecoderRender to return VDP_STATUS_INVALID_HANDLE (3)?

This happens when any VDPAU object handle passed to the API is invalid (i.e. is not a value returned from some previous object creation API, and that has not since been deleted, and the error callback has not occurred).

This could occur for:

* The decoder object itself.
* The surface being rendered to (which must be a video surface.)
* The reference frames listed in the picture_info (all handles in the structure must be valid even if they aren't logically used; pass in VDP_INVALID_HANDLE for unused values.)

[jusst]

Thanks, we missed the VDP_INVALID_HANDLE for unused reference frames...

[jusst]

Stephen,

I have one more question about referenceFrames. Is there any exact description what to set frame_idx to?
For long-term refs I set it to long_term_frame_idx now, for short-term refs to currpicnum. Either one seems not to be ok. Can you give some details please?

Thanks

[Stephen Warren]

The MPlayer patches at least fill frame_idx with the frame_num field from the slice header, for all types of reference frames.

[jusst]

We're making good progress, but regarding h264 interlaced streams I have another question:
In the VDPAU doc you state:

Depending on the exact encoding structure of the compressed video stream, the application may need to call VdpDecoderRender twice to fill a single VdpVideoSurface. When the stream contains an encoded progressive frame, or a "frame coded" interlaced field-pair, a single VdpDecoderRender call will fill the entire surface. When the stream contains separately encoded interlaced fields, two VdpDecoderRender calls will be required; one for the top field, and one for the bottom field.

Talking about frame-coded interlaced field pair do you mean complementary field pairs? Or is there anything I missed from the h264 datasheet which matches the frame-coded field pair?

[Stephen Warren]

I forget the exact terminology off the top of my head right now.

The basic idea is that anything that's encoded as separate pictures (not just slices within pictures) needs to be a separate decode call to VDPAU.