Go Back   nV News Forums > Linux Support Forums > NVIDIA Linux

Newegg Daily Deals

Reply
 
Thread Tools
Old 01-08-09, 09:43 AM   #1
crisalide
Registered User
 
Join Date: Dec 2008
Posts: 173
Default VDPAU: multithreading issues

Hi,

As you may know, xine-lib is highly multithreaded.
And VDPAU is meant to be thread safe.
Unfortunately, in some environments, some VDPAU calls lock.
We have to XLock/UnlockDisplay around these calls to get rid of the locks.
The affected functions are:

VdpVideoSurfaceCreate
VdpDecoderCreate
VdpDecoderDestroy
VdpDecoderRender

Since we also have to lock the display at display_frame, this could eventually cause some frames being decoded too late and thus dropped by the engine, especially on lower gpu.

Have you created any fixes in this matter, or is this unknown to you?
crisalide is offline   Reply With Quote
Old 01-08-09, 10:52 AM   #2
Stephen Warren
Moderator
 
Stephen Warren's Avatar
 
Join Date: Aug 2005
Posts: 1,327
Default Re: VDPAU: multithreading issues

crisalide,

When you say "Unfortunately, in some environments, some VDPAU calls lock", do you mean VDPAU "locks up" or hangs? If so, can you attach gdb and get a backtrace in this situation.

Is xine-lib using pthreads or some other threading system? Every VDPAU call is protected by a pthread_mutex_lock/unlock around the entire code...
Stephen Warren is offline   Reply With Quote
Old 01-08-09, 02:57 PM   #3
rnissl
Registered User
 
Join Date: Jun 2005
Posts: 36
Default Re: VDPAU: multithreading issues

Hi,

Quote:
Originally Posted by Stephen Warren View Post
crisalide,

When you say "Unfortunately, in some environments, some VDPAU calls lock", do you mean VDPAU "locks up" or hangs? If so, can you attach gdb and get a backtrace in this situation.
Well, on some environments means most likely all environments which have a xcb based X system, like openSUSE 10.3 and up, current ubuntu, ...

Please have a look at this backtrace: http://jusst.de/vdpau/backtrace3.txt

Thread 19 is one of xine-ui's threads which is about to update the window title. To do this, it has locked the display and called the appropriate function. The xcb based Xlib is now waiting to retrieve the X servers reply before it returns.

The offending thread is number 14. It's xine-libs video decoder thread. It is just about to create a decoder for the stream to watch. As you can see, vdpau is waiting for a X server's reply too. But despite Xlib programming practise, vdpau hasn't locked the display before calling the Xlib function and now it depends on the Xlib implementation, whether a deadlock happens or not.

As you can see, an xcb based Xlib locks the display while processing _XReply(). And now a deadlock happens because thread 19 has locked the display already. But thread 19 cannot go on in _XReply() because it's reply was queued after the reply for thread 14.

You can find a functions named guarded_vdp_video_surface_create (see thread 17) or guarded_vdp_decoder_create (see thread 14) on the backtrace. We had to guard all the previously mentioned vdpau functions with XLockDisplay() / XUnlockDisplay() to avoid this kind of deadlocks. But the drawback is, that for example VdpDecoderRender may either take some time to process while no screen update is possible or -- on the other hand -- it cannot start to decode due to the display beeing locked for screen updates for example.

To lock the display for the shortest time possible, it would require the vdpau function to do this internally. Another solution would be to avoid Xlib calls in VDPAU where possible and to document which functions require a locked display. Among the previously mentioned functions, VdpDecoderRender is the only one which is called regularly. At least this one should be modified as mentioned before.

Quote:
Originally Posted by Stephen Warren View Post
Is xine-lib using pthreads or some other threading system? Every VDPAU call is protected by a pthread_mutex_lock/unlock around the entire code...
xine-lib uses pthreads and so does xine-ui. But xine-ui takes also care to guard every Xlib call with XLockDisplay() / XUnlockDisplay() as it should be done in a multithreaded environment where more than one thread access the display.

Some notes about my system:
Host/Kernel/OS "corei7" running Linux 2.6.27.7-9.1_rnissl-default x86_64 [ openSUSE 11.1 (x86_64) VERSION = 11.1 ]
CPU Info (1) Intel Core i7 965 @ 8192 KB cache flags( sse sse2 nx lm pni vmx ) clocked at [ 1600.000 MHz ]
(2) Intel Core i7 965 @ 8192 KB cache flags( sse sse2 nx lm pni vmx ) clocked at [ 1600.000 MHz ]
(3) Intel Core i7 965 @ 8192 KB cache flags( sse sse2 nx lm pni vmx ) clocked at [ 1600.000 MHz ]
(4) Intel Core i7 965 @ 8192 KB cache flags( sse sse2 nx lm pni vmx ) clocked at [ 1600.000 MHz ]
(5) Intel Core i7 965 @ 8192 KB cache flags( sse sse2 nx lm pni vmx ) clocked at [ 1600.000 MHz ]
(6) Intel Core i7 965 @ 8192 KB cache flags( sse sse2 nx lm pni vmx ) clocked at [ 1600.000 MHz ]
(7) Intel Core i7 965 @ 8192 KB cache flags( sse sse2 nx lm pni vmx ) clocked at [ 1600.000 MHz ]
(8) Intel Core i7 965 @ 8192 KB cache flags( sse sse2 nx lm pni vmx ) clocked at [ 1600.000 MHz ]
Videocard nVidia GeForce 8600 GTS X.Org 1.5.2 [ 1920x1080 @50hz ]
Network cards 2x Realtek RTL8111/8168B PCI Express Gigabit Ethernet controller, at ports: d800 e800
Processes 197 | Uptime 2:20 | Memory 517.4/5964.7MB | HDD SEAGATE ST3450856SS Size 450GB (24%used) | GLX Renderer GeForce 8600 GTS/PCI/SSE2 | GLX Version 2.1.2 NVIDIA 180.22 | Client Shell | Infobash v2.67.1

(II) NVIDIA(0): NVIDIA GPU GeForce 8600 GTS (G84) at PCI:4:0:0 (GPU-0)


Bye.
rnissl is offline   Reply With Quote
Old 01-08-09, 03:29 PM   #4
Stephen Warren
Moderator
 
Stephen Warren's Avatar
 
Join Date: Aug 2005
Posts: 1,327
Default Re: VDPAU: multithreading issues

I see what you mean. We'll look into fixing this.
Stephen Warren is offline   Reply With Quote
Old 01-08-09, 03:47 PM   #5
Stephen Warren
Moderator
 
Stephen Warren's Avatar
 
Join Date: Aug 2005
Posts: 1,327
Default Re: VDPAU: multithreading issues

A couple more questions on this:

1) Does xine call XInitThreads befiore any other X API?

2) In your backtrace in thread 14, the following stack entry:

Code:
#4 0x00007f87aaebc21e in ?? () from /usr/lib64/libvdpau_nvidia.so
... does call LockDisplay/UnlockDisplay (this function is part of the NV-GLX X extension client implementation, so it uses LockDisplay not XLockDisplay, but this should work fine.)

So, I'm confused why XCBLockDisplay/XDisplayLockWait in that thread think that the display is not already locked by that thread...

Aside from that, some functions in VDPAU do/may call regular Xlib APIs without locking the display. These are: presentation_queue_create, presentation_queue_destroy, presentation_queue_display, presentation_queue_set_backgroun_color. However, you should be able to avoid all these by "export VDPAU_NVIDIA_NO_OVERLAY=1" before running your application.

3) How many X display connections does xine use?

Last edited by Stephen Warren; 01-08-09 at 03:49 PM. Reason: Added question 3
Stephen Warren is offline   Reply With Quote
Old 01-08-09, 04:40 PM   #6
rnissl
Registered User
 
Join Date: Jun 2005
Posts: 36
Default Re: VDPAU: multithreading issues

Hi,

Quote:
Originally Posted by Stephen Warren View Post
A couple more questions on this:

1) Does xine call XInitThreads befiore any other X API?
yes
Quote:
Originally Posted by Stephen Warren View Post
2) In your backtrace in thread 14, the following stack entry:

Code:
#4 0x00007f87aaebc21e in ?? () from /usr/lib64/libvdpau_nvidia.so
... does call LockDisplay/UnlockDisplay (this function is part of the NV-GLX X extension client implementation, so it uses LockDisplay not XLockDisplay, but this should work fine.)

So, I'm confused why XCBLockDisplay/XDisplayLockWait in that thread think that the display is not already locked by that thread...

Aside from that, some functions in VDPAU do/may call regular Xlib APIs without locking the display. These are: presentation_queue_create, presentation_queue_destroy, presentation_queue_display, presentation_queue_set_backgroun_color. However, you should be able to avoid all these by "export VDPAU_NVIDIA_NO_OVERLAY=1" before running your application.
Hhm, as far as I recall, the xcb based Xlib doesn't use LockDisplay due to xcb. xcb has it's own mutexes and variables which are used for their XLockDisplay implementation. Maybe they don't use LockDisplay as xcb is "thread-safe" and calling LockDisplay would block threads unnecessarily.

Quote:
Originally Posted by Stephen Warren View Post
3) How many X display connections does xine use?
Can you explain what I should look for? I'm not the auther of xine-ui nor xine-lib and I need to dig into the source to answer your question.

Bye.
rnissl is offline   Reply With Quote
Old 01-13-09, 01:10 PM   #7
Stephen Warren
Moderator
 
Stephen Warren's Avatar
 
Join Date: Aug 2005
Posts: 1,327
Default Re: VDPAU: multithreading issues

Just as an FYI, I've been testing with xine/xine-ui (SVN/CVS from mid-yesterday) and found some missing error handling in VdpVideoMixerRender, specifically related to data validation of the layers parameter.

I've fixed this bug, and it shows up some issue in xine where some of the fields in the input layers data structure are invalid (I think it's the surface handle) some of the time. It seems that if it's ever OK, it'll be OK for the entire duration of the clip, and if it's ever bad, it'll be bad for the entire duration of the clip. Each time I press play on the UI it's "random" whether it'll be OK/bad.

In existing driver releases, this will either cause a segfault within VDPAU, or the layer entry to be ignored. In a future driver release, this will start returning an error for this function call.

On the topic of the hangs: I believe I've identified the issues. So, this should be solved in a future next driver release.
Stephen Warren is offline   Reply With Quote
Old 01-13-09, 03:37 PM   #8
jusst
Registered User
 
Join Date: Mar 2006
Posts: 99
Default Re: VDPAU: multithreading issues

Could you post the fixes you made for xine-vdpau, Stephen?
Either attach it here or mail it to the xine-vdpau list.
jusst is offline   Reply With Quote

Old 01-13-09, 03:38 PM   #9
Stephen Warren
Moderator
 
Stephen Warren's Avatar
 
Join Date: Aug 2005
Posts: 1,327
Default Re: VDPAU: multithreading issues

I didn't fix the xine issue; I just ignored that problem during my testing.
Stephen Warren is offline   Reply With Quote
Old 01-13-09, 03:53 PM   #10
jusst
Registered User
 
Join Date: Mar 2006
Posts: 99
Default Re: VDPAU: multithreading issues

Ah ok, I misunderstood you then. Actually we can not reproduce what you describe... So if you can guide us to reproduce it'd be handy...
jusst is offline   Reply With Quote
Old 01-13-09, 03:59 PM   #11
Stephen Warren
Moderator
 
Stephen Warren's Avatar
 
Join Date: Aug 2005
Posts: 1,327
Default Re: VDPAU: multithreading issues

Right now (i.e. in released drivers), the VDPAU function doesn't have correct error checking. So, you're probably just getting lucky and it's ignoring the entry in layers rather than crashing. Certainly, some of the time this happened to me too.

Also, this seemed like it was probably a race condition between when the mode set happened and trashed all the VDPAU objects, v.s. when they were used, so it's possible it depends on many other factors such as system performance etc.

I'd say ignore this issue until the NVIDIA driver is released with the error checking, then it should be pretty obvious if you're seeing it.
Stephen Warren is offline   Reply With Quote
Old 01-15-09, 10:22 PM   #12
Stephen Warren
Moderator
 
Stephen Warren's Avatar
 
Join Date: Aug 2005
Posts: 1,327
Default Re: VDPAU: multithreading issues

Further investigation implies the deadlock in libX11 is not a bug in VDPAU, but rather a bug in libX11 or libX11-xcb. I believe the following links are probably the same bug:

https://bugs.launchpad.net/ubuntu/+s...11/+bug/232476
https://bugs.freedesktop.org/show_bug.cgi?id=16617

Also, some of the changes at the link below are possibly related to fixing it or similar issues:

http://lists.debian.org/debian-x/2008/12/msg00012.html

Some other information: libX11 and libX11-xcb clearly are intended to operate correctly in a multi-threaded environment without the application/... calling XLockDisplay/XUnlockDisplay around all X APIs; there is a *considerable* amount of locking code in libX11/libX11-xcb that simply would not be present if it could instead be implemented by a simple mutex in XLockDisplay.

Note: As I think I mentioned above, as a side-effect of testing with xine, we did find another hang in the blit-based presentation queue, which has been fixed for the next driver release.
Stephen Warren is offline   Reply With Quote
Reply


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


Similar Threads
Thread Thread Starter Forum Replies Last Post
xorg locks-up with newest nvidia drivers w/ vdpau. theroot NVIDIA Linux 1 06-24-12 11:04 AM
Strange RedHat 7.3 issues with A7N266-C nforceuser NVIDIA Linux 8 09-27-02 03:16 PM
Overclocking issues... mcortz_2000 CPUs, Motherboards And Memory 5 09-16-02 03:41 PM
Is there a Microsoft patch for the refresh rate issues yet? Zarich NVIDIA GeForce 7, 8, And 9 Series 22 08-24-02 01:02 PM

All times are GMT -5. The time now is 03:44 AM.


Powered by vBulletin® Version 3.7.1
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Copyright 1998 - 2014, nV News.