Go Back   nV News Forums > Linux Support Forums > NVIDIA Linux

Newegg Daily Deals

Reply
 
Thread Tools
Old 07-22-08, 09:05 AM   #1
twbos
Registered User
 
Join Date: Jul 2008
Posts: 14
Default glXSwapBuffers latency/dropped frames

Hello,

I have a technical problem with OpenGL on Nvidia cards (I don't have any other vendors, so I don't know whether the problem occurs there as well). I have turned VSYNC on, and I have a very simple test application, which is basically:

while(1) {
glXSwapBuffers();
clock_gettime();
}

Most of the time, the the time between iterations of the loop is 16.9ms-17.9ms, but sometimes its like 60ms or even > 500ms. I normally check about 10000 iterations, and every time I start my application, the pattern of delays almost exactly repeats, i.e. the exact same frames have a long delay. I have turned off everything on my machine, even ran this on a bare X server without window manager, no swapping, no network, no USB. Nothing helps. I did notice when using this with top in parallel that the long delays seem to correlate with X taking over the the CPU for a while. Also my harddrive led seems to flash exactly as many times as their are these long lags (5 times in the 10000 iterations), and blktrace showed kjournald to be doing something.
I have reproduced this on three different computers, each running a different Linux (redhat, ubuntu and gentoo), and I'm not sure where to continue next. Has anyone ever observed this behaviour or knows any solutions to this ? I'm thinking of now reinstalling one machine with a non-journaling file system, but even if that was the case, somehow I feel that the journal demon should not mess with my display.
I have btw, also used the newest nvidia driver on a 2.6.25.8-rt7 kernel (with the semaphore patch) and ran my application in RT priority, which makes the timing more robust to mouse movements and other stuff going on, but the 5 long delays are still there and exactly the same, so I'm pretty convinced that whatever is causing this is inside the glXSwapBuffers call, and not in my application.

Thanks so much for your help in advance,
Thomas

Update: I found out that the extreme latencies are caused by PowerMizer switching clock frequency, so whenever the clock frequency changes a long delay occurs. I turned powermizer off, and now I'm not loosing any frames, however I get some very long delays in perfectly regular intervals, every 1200 frames or so, where swapbuffers doesn't return till 2 ms before the next refresh is due. __GL_YIELD settings have no effect whatsoever on this, is __GL_YIELD still supported in the current drivers ?

Last edited by twbos; 07-23-08 at 09:07 AM. Reason: updated information
twbos is offline   Reply With Quote
Old 07-29-08, 01:37 PM   #2
twbos
Registered User
 
Join Date: Jul 2008
Posts: 14
Default Re: glXSwapBuffers latency/dropped frames

Hello,

I have tried many things over the last week and still have not resolved the issue of why glXSwapBuffers is sometimes lagging. I'm now using SwapControl to either swap every 5 syncs or every 10 syncs, and it turns out that in every ten syncs config the first long lag occurs after 170 swaps (1700 frames), wereas in the every 5 sync mode the first lag occurs only after 340 swaps (also 1700 frames), so it appears that the delays occur at constant intervals relative to the loop start, irregardless of the frame rate. This behaviour is the exact same with and without scene rendered, with and without glFlush, for kernels 2.6.24,2.6.25 and 2.6.26, with swap on and swap off and across different days. I tried driver versions 173 and 177, its a 8600 series mobile card. I also tried on two different machines, they have similar problems, but the lags occur at different times.
Can someone please help, or please let me know who to talk to ? I know this is not a deterministic real time system, but some reliability within 3-5 ms should be doable, no ?

Thanks Thomas
twbos is offline   Reply With Quote
Old 07-29-08, 02:37 PM   #3
Plagman
NVIDIA Corporation
 
Plagman's Avatar
 
Join Date: Sep 2007
Posts: 254
Default Re: glXSwapBuffers latency/dropped frames

Can you please attach an nvidia-bug-report.log (as per the sticky thread about reporting problems) and a copy of a test case reproducing the problem?
Plagman is offline   Reply With Quote
Old 07-29-08, 02:58 PM   #4
twbos
Registered User
 
Join Date: Jul 2008
Posts: 14
Default Re: glXSwapBuffers latency/dropped frames

Quote:
Originally Posted by Plagman View Post
Can you please attach an nvidia-bug-report.log (as per the sticky thread about reporting problems) and a copy of a test case reproducing the problem?
Thanks will do. Just need to clean up the code a little bit.
twbos is offline   Reply With Quote
Old 07-30-08, 08:53 AM   #5
savali
Registered User
 
Join Date: Apr 2005
Posts: 9
Default Re: glXSwapBuffers latency/dropped frames

I'm also cursed by similar problem. It seems that sometimes glXSwapBuffers() experiences odd latency.

Systems tested:
- Sun Ultra 24 + 2 x GeForce 8800GT
- Sun Ultra 24 + 1 x GeForce 8800GT
- Sun Ultra 40m2 + 2 x GeForce 9600GT

all running RHEL 5.1 (x86_64)

nvidia-bug-report included.
Please tell us if there is something else that could be done to help resolve this?
Attached Files
File Type: gz nvidia-bug-report.log.gz (30.2 KB, 125 views)
savali is offline   Reply With Quote
Old 07-31-08, 04:38 AM   #6
twbos
Registered User
 
Join Date: Jul 2008
Posts: 14
Default Re: glXSwapBuffers latency/dropped frames

Thats interesting, as you have a different architecture. I have now installed my system with 2.6.27-rc1 and patched the nvidia driver to run on it. This allows me to use ftrace. I haven't done too much, but I can see the same latency drops in the sched_switch ftrace, i.e. my test application is simply not scheduled when it should be. I'm doing all this, because I'm not 100% sure that this is the nvidia drivers fault at all. It could just be the scheduler of the kernel and/or the X server. Further analysis will show more.

BTW: With powermizer out of the way the first irregularity occurs exactly 50 seconds after the start of the test application. Also photo-diode measurements on the screen show that the frame was switched on time, but my process was not woken up on time after the swapbuffers had completed.
twbos is offline   Reply With Quote
Old 07-31-08, 06:03 AM   #7
savali
Registered User
 
Join Date: Apr 2005
Posts: 9
Default Re: glXSwapBuffers latency/dropped frames

It's true that it can be scheduler issue. My last debugging session of this issue ended few weeks ago with conclusion that it's either scheduler or nvidia drivers issue. But I'm certainly going to continue when more pressing tasks are finished.

Did you get simple test case done? I would like to test that to verify that we are talking about same problem...


FYI: I'm currently running 2.6.18-92.1.6.el5, though I tested this also with kernel-rt 2.6.24.3-1.rt3.2.el5.ccrma from ccrma distribution. It didn't affect the problem.
savali is offline   Reply With Quote
Old 07-31-08, 03:18 PM   #8
twbos
Registered User
 
Join Date: Jul 2008
Posts: 14
Default Re: glXSwapBuffers latency/dropped frames

My test code is attached. On the current schedule and my refresh rate of 59.7Hz (Dell Laptop) the first irregularity occurs at frame 582.

It needs to be compiled with gcc glx_example.c -o glx_example -lGL -lrt
The timing error detection is hardcoded for my refresh rate and would need to be adapted for yours.

Thomas
Attached Files
File Type: zip gec.zip (2.6 KB, 168 views)
twbos is offline   Reply With Quote

Old 08-01-08, 12:16 AM   #9
twbos
Registered User
 
Join Date: Jul 2008
Posts: 14
Default Re: glXSwapBuffers latency/dropped frames

Here are some more attachments. First of all my bug-report.log and secondly the ftrace files for a normal non-delayed swapbuffers and one for a delayed one. The delayed one shows that the driver is doing some extensive memory mapping and page configuration, that doesn't occur during normal swapbuffer calls. Driver developers, please have a look.
The ftrace was generated using the ftrace extension in 2.6.27-rc1, and was turned on immediately before glxswapbuffers and turned off immediately after the gettime.
Since the ftrace files are too big for attachment, please find them
here
http://www.martinos.org/~twitzel/swapbuffers
Thank you so much,
Thomas

PS: The kernel build log in the bug-report is shot, because I generated the report under 2.6.25, but the module code is patched for 2.6.27.
Attached Files
File Type: log nvidia-bug-report.log (116.2 KB, 134 views)
twbos is offline   Reply With Quote
Old 08-02-08, 01:14 PM   #10
twbos
Registered User
 
Join Date: Jul 2008
Posts: 14
Default Re: glXSwapBuffers latency/dropped frames/ nv_vm_malloc_pages eating extra 20 ms

I went through all these traces and it appears that when a delayed frame occurs, the nvidia driver is calling the nv_vm_malloc_pages function a few tousand times, which in essence eating 20 extra milliseconds. Its not clear to me, why it is necessary to allocate more memory in the middle of my program, but the calls are charged to my process. Can someone from nvidia please help me with this ?

Thank you very much,
Thomas

I detail: It seems that in case of my application the nvidia driver allocates 512 pages about every 50 seconds. The pages are not freed till the application ends.

Last edited by twbos; 08-04-08 at 08:42 AM.
twbos is offline   Reply With Quote
Old 08-04-08, 08:37 AM   #11
twbos
Registered User
 
Join Date: Jul 2008
Posts: 14
Default Re: glXSwapBuffers latency/dropped frames

Can anyone please help me ? I have now verified this problem with 8600M,8400M,Quadro NV280, Quadro FX4600 and Geforce 8800GT. Is there anything I can do, buy a different card GTX 280 or anything ? I really need to get this problem solved and if its impossible with a nvidia card I need to know.

Thank you,
Thomas
twbos is offline   Reply With Quote
Old 08-04-08, 08:58 AM   #12
savali
Registered User
 
Join Date: Apr 2005
Posts: 9
Default Re: glXSwapBuffers latency/dropped frames

I tried your test code, but surprisingly it seems that I cannot replicate the problem.

I modified the code by setting it to swap at every refresh and changed output to my liking. I even tried to run with normal scheduling settings and without cpu affinity & mlockall, but it even then it seems to work acceptably. Modified code is included as attachment.

It seems that once in about 4-5000 frames there will be two frame abnormality in swap times which is repeated once in close proximity to first occurance. These can be seen from log clip below. First frame is ~2ms late and second is ~2ms early.

I'm running twinview 2 x 1600x1200 @ 85Hz, so my normal frame time should be 11.76ms.

When I have time, I'll test my actual application code to see, if it still has this swap problem... Sadly it might take few weeks before I can return to that code...

-- Samuli

---------- log clip: cat ./glx_timing.txt.3 |egrep -3 '1[456789]\.[0-9]+ms$' -------------------
frame_num frame_clock frame_length

104075 1224461.454ms 11.990ms
104076 1224473.067ms 11.613ms
104077 1224484.756ms 11.689ms
104078 1224499.068ms 14.312ms
104079 1224508.629ms 9.561ms
104080 1224520.050ms 11.421ms
104081 1224531.819ms 11.769ms
--
104098 1224731.900ms 11.841ms
104099 1224743.583ms 11.683ms
104100 1224755.602ms 12.019ms
104101 1224769.715ms 14.113ms
104102 1224778.999ms 9.284ms
104103 1224790.663ms 11.664ms
104104 1224802.646ms 11.983ms
--
108805 1280111.059ms 11.990ms
108806 1280122.601ms 11.542ms
108807 1280134.586ms 11.985ms
108808 1280148.699ms 14.113ms
108809 1280158.158ms 9.459ms
108810 1280169.746ms 11.588ms
108811 1280181.424ms 11.678ms
--
108828 1280381.433ms 11.775ms
108829 1280393.192ms 11.759ms
108830 1280405.199ms 12.007ms
108831 1280419.385ms 14.186ms
108832 1280428.540ms 9.155ms
108833 1280440.253ms 11.713ms
108834 1280452.246ms 11.993ms
--
Attached Files
File Type: txt glx_example.c.txt (7.3 KB, 184 views)
savali is offline   Reply With Quote
Reply


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


Similar Threads
Thread Thread Starter Forum Replies Last Post
Counter-Strike skipping frames on a GeForce4 demonized NVIDIA GeForce 7, 8, And 9 Series 8 10-10-02 05:02 PM

All times are GMT -5. The time now is 09:22 PM.


Powered by vBulletin® Version 3.7.1
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Copyright 1998 - 2014, nV News.