Another GT-240 Audio/video Issue

[kennelm]

I am running MythTV on fedora 12, and I use the HDMI out on the GT-240 for both audio and video. When I playback content captured from my HDHR, everything is great. I get awesome video, and 5.1 surround sound. I am using VDPAU for this, and it seems to work no problem. My CPU is practically idling when I play back HD.

Recently, I picked up an HD-PVR, and I am capturing 720P from the STB and audio over RCA cables (S/PDIF works, but only on certain programming, so I had to go with analog sound). When I playback this H.264 content, the video was mostly watchable, but it was intermittently jerky. I enabled verbose logging in mythfrontend, and it turns out I was getting lots of dropped frames, in order to keep the audio and video in sync. I checked all my settings and, among many other things, I disabled the setting to upconvert stereo to 5.1. After that, the dropped frames went away, at least I thought, since the video became quite watchable. But, under scrutiny, I noticed a very slight sync issue with the audio and video. Here is what appears in the myth log now:

2010-07-30 19:18:42.078 NVP(3): Video is 3.04058 frames behind audio (too slow), dropping frame to catch up.
2010-07-30 19:18:42.178 NVP(3): Video is 3.02236 frames behind audio (too slow), dropping frame to catch up.
2010-07-30 19:18:42.278 NVP(3): Video is 3.01912 frames behind audio (too slow), dropping frame to catch up.
2010-07-30 19:18:42.378 NVP(3): Video is 3.00731 frames behind audio (too slow), dropping frame to catch up.
2010-07-30 19:18:42.498 NVP(3): Video is 3.11125 frames behind audio (too slow), dropping frame to catch up.
2010-07-30 19:18:42.578 NVP(3): Video is 3.0564 frames behind audio (too slow), dropping frame to catch up.
2010-07-30 19:18:42.698 NVP(3): Video is 3.12905 frames behind audio (too slow), dropping frame to catch up.
2010-07-30 19:18:42.778 NVP(3): Video is 3.07217 frames behind audio (too slow), dropping frame to catch up.
2010-07-30 19:18:42.878 NVP(3): Video is 3.00563 frames behind audio (too slow), dropping frame to catch up.
2010-07-30 19:18:42.998 NVP(3): Video is 3.08638 frames behind audio (too slow), dropping frame to catch up.

So, in the span of 1 second, my system dropped something like 30 frames. Does that even make sense?

If I go into mythtv and set the default audio to NULL, I get no sound, and the dropped frames disappear. I assume this means the audio is the problem?

Within mythtv settings, I get sound from both alsa: DEFAULT and alsa:HDMI. Don't think I tried all the choices, tho.

If I switch from VDPAU to SLIM or some other playback profile, I still get the dropped frames. I assume this means the issue is not VDPAU-related?

Here is my driver version, which I installed from the rpmfusion repos:

rpm -qa | grep nvidia
xorg-x11-drv-nvidia-libs-195.36.31-2.fc12.i686
xorg-x11-drv-nvidia-195.36.31-2.fc12.i686
kmod-nvidia-2.6.32.14-127.fc12.i686-195.36.24-1.fc12.3.i686
kmod-nvidia-195.36.31-1.fc12.1.i686
nvidia-xconfig-1.0-2.fc12.i686
kmod-nvidia-2.6.32.16-141.fc12.i686-195.36.31-1.fc12.1.i686
nvidia-settings-1.0-4.fc12.i686

This I got from atrpms:

rpm -qa | grep alsa
alsa-kmdl-2.6.32.14-127.fc12-1.0.23-84.fc12.i686
alsa-lib-devel-1.0.23-1.fc12.i686
alsa-kmdl-2.6.32.16-141.fc12-1.0.23-84.fc12.i686
alsa-driver-1.0.23-84.fc12.i686
report-plugin-localsave-0.14-1.fc12.i686
alsa-utils-1.0.22-1.fc12.i686
report-config-localsave-0.14-1.fc12.i686
alsa-lib-1.0.23-1.fc12.i686

This also from atrpms:

rpm -qa | grep mythtv
mythtv-0.23-235.fc12.i686

Sound card devices:

aplay -L
null
Discard all samples (playback) or generate zero samples (capture)
hdmi:CARD=NVidia
HDA NVidia, NVIDIA HDMI
HDMI Audio Output

aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: NVidia [HDA NVidia], device 3: NVIDIA HDMI [NVIDIA HDMI]
Subdevices: 0/1
Subdevice #0: subdevice #0
card 0: NVidia [HDA NVidia], device 7: NVIDIA HDMI [NVIDIA HDMI]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: NVidia [HDA NVidia], device 8: NVIDIA HDMI [NVIDIA HDMI]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: NVidia [HDA NVidia], device 9: NVIDIA HDMI [NVIDIA HDMI]
Subdevices: 1/1
Subdevice #0: subdevice #0

I can post more of my log, but I didn't want to overload you guys with too much info until we at least try to frame this problem a little better.

Can anyone give me some pointers on how to address this?

Thanks!
Larry

[Stephen Warren]

When capturing audio and video, each frame of video and packet of audio is marked with a capture timestamp. During playback, SW attempts to have video and audio frames/packets with the same timestamp play back at the same time to maintain AV sync. However, this only works if the clock source used to create those timestamps at recording time are the same, and the clock source used for both video and audio during playback are the same.

I imagine the HDHR has a HW design that uses the same clock for recording both video and audio, and hence the timestamp streams correctly align in the encoded file.

With your new HD-PVR setup, I'm not sure whether the HD-PVR is a box that encode both the audio and video, or whether it's a PCI/PCI-E card that just captures the video, while you're using a PC sound-card to capture the audio? In the latter case, different clocks are used, so the timestamps will drift apart, causing the kind of issue you're seeing. Even if the HD-PVR is a single all-in-one device, it may have been incorrectly designed and hence produce a timestamp drift that the HDHR doesn't.

Finally, in the PC architecture, the PC's system clock, the video scanout clock, and the audio clock are all completely different clocks, and will suffer some drift. This in itself can produce the kind of issue you're seeing too.

Summary: Getting AV sync correct on a PC without dropping/duplicating either video frames or short segments of audio is impossible.

That said, the apparent rate of frame drops in your log seems excessive. To find the real cause, some significant analysis of the video files would be required. I'd suggest asking in a MythTV forum about this though.

[kennelm]

Originally Posted by Stephen Warren

When capturing audio and video, each frame of video and packet of audio is marked with a capture timestamp. During playback, SW attempts to have video and audio frames/packets with the same timestamp play back at the same time to maintain AV sync. However, this only works if the clock source used to create those timestamps at recording time are the same, and the clock source used for both video and audio during playback are the same.

I imagine the HDHR has a HW design that uses the same clock for recording both video and audio, and hence the timestamp streams correctly align in the encoded file.

With your new HD-PVR setup, I'm not sure whether the HD-PVR is a box that encode both the audio and video, or whether it's a PCI/PCI-E card that just captures the video, while you're using a PC sound-card to capture the audio? In the latter case, different clocks are used, so the timestamps will drift apart, causing the kind of issue you're seeing. Even if the HD-PVR is a single all-in-one device, it may have been incorrectly designed and hence produce a timestamp drift that the HDHR doesn't.

Finally, in the PC architecture, the PC's system clock, the video scanout clock, and the audio clock are all completely different clocks, and will suffer some drift. This in itself can produce the kind of issue you're seeing too.

Summary: Getting AV sync correct on a PC without dropping/duplicating either video frames or short segments of audio is impossible.

That said, the apparent rate of frame drops in your log seems excessive. To find the real cause, some significant analysis of the video files would be required. I'd suggest asking in a MythTV forum about this though.

I need to make an important correction. Namely, I am seeing the same behavior (log messages) when I playback content from both the SiliconDust HDHomeRun AND the Hauppauge HD-PVR. Originally, I thought this was strictly related to the content captured by the HD-PVR (H.264). However, it is also observed for the HDHR (mpeg-2) as well. Maybe this is a good thing since it points to something in my configuration that is equally affecting playback of content from both devices.

To clarify, the HD-PVR is an H.264 encoder that captures both audio and video from a STB (over a USB connection).

Again, if I completely disable audio in the myth setup, the messages stop (and so does the sound).

I'll keep looking. Tips appreciated.


Thanks!

[mlord]

What output path/device are you using for the audio playback?

Eg. audio over HDMI, or SPDIF-out from the PC, or 5.1 analog out, or simple stereo out, etc. I don't mean the software settings, but rather which hardware pathway is being used?

I have a GT-240 card here, plus an HDHR, and life is generally very good on playback. I'll enable logging and look for the dropped frame messages later, just for interest sake.

Edit: Mmm.. nothing about dropped frames in the default mythfrontend.log output; did you have to enable any special logging flags to get those messages?

[kennelm]

Originally Posted by mlord

What output path/device are you using for the audio playback?

Eg. audio over HDMI, or SPDIF-out from the PC, or 5.1 analog out, or simple stereo out, etc. I don't mean the software settings, but rather which hardware pathway is being used?

I have a GT-240 card here, plus an HDHR, and life is generally very good on playback. I'll enable logging and look for the dropped frame messages later, just for interest sake.

Edit: Mmm.. nothing about dropped frames in the default mythfrontend.log output; did you have to enable any special logging flags to get those messages?

I am using the HDMI-out on the GT 240 to carry both audio and video to my receiver.

To enable verbose logging, I think it was mythfrontend -v audio,playback. You can run mythfrontend -v help or something like that to see the valid switches.

Without these verbose switches you will not see anything about dropped frames in the log.