X freezes with glx applications

[Merath]

Hi, I just started having an issue with my glx applications (compiz, glxinfo, glxgears, the opengl/glx information tab in nvidia-settings) since updatign my computer. Every time I start one my X hard freezes. I can ssh into my system but X is unresponsive. X is taking up 100pct of the proc according to top, and I cannot kill it, I have to hard reset my comp. Also a note, it was several days after I updated my system before I rebooted and the issues started.

First I removed compiz and its other packages(I'm using gentoo) and then I checked over my Xorg.conf to make sure GLcore and dir were not enabled. I also disabled composite and rebooted.

Also I have tried older versions of nvidia drivers, seeing as my problem started after a world update. Finally I recompiled xorg-X11 and still nothing.

I have searched all over and found nothing but this:

http://gentoo-wiki.com/HOWTO_nVidia_...or_OpenGL_apps

which describes my problem exactly (I get the erronous messages by tailing /var/log/messages from ssh) but my NX/XD bit is off and I am not using hardened sources for my kernel. Also I can find nothing on these forums relating to this problem.

# tail /var/log/messages

Oct 21 06:27:01 sakaru [<ffffffff8020931e>] system_call+0x7e/0x83
Oct 21 06:27:01 sakaru 
Oct 21 06:27:01 sakaru 
Oct 21 06:27:01 sakaru Code: 48 c7 80 00 02 00 00 ff ff ff ff 65 48 8b 04 25 00 00 00 00 
Oct 21 06:27:01 sakaru RIP  [<ffffffff883935cb>] :nvidia:os_set_mlock_capability+0x10/0x34
Oct 21 06:27:01 sakaru RSP <ffff81006ce55df0>
Oct 21 06:27:01 sakaru CR2: 0000000000000200
Oct 21 06:27:11 sakaru sshd[6100]: Accepted keyboard-interactive/pam for william from 192.168.1.104 port 49858 ssh2
Oct 21 06:27:11 sakaru sshd(pam_unix)[6103]: session opened for user william by (uid=0)
Oct 21 06:27:16 sakaru sudo:  william : TTY=pts/1 ; PWD=/home/william ; USER=root ; COMMAND=/bin/bash

I would appreciate any advice/insight/or links relating to this problem.

~Merath

[masterzen]

Originally Posted by Merath

Hi, I just started having an issue with my glx applications (compiz, glxinfo, glxgears, the opengl/glx information tab in nvidia-settings) since updatign my computer. Every time I start one my X hard freezes. I can ssh into my system but X is unresponsive. X is taking up 100pct of the proc according to top, and I cannot kill it, I have to hard reset my comp. Also a note, it was several days after I updated my system before I rebooted and the issues started.

I see exactly the same issue on a freshly updated Debian testing.
Each time an Open GL application is launched, the nvidia kernel module is oopsing badly in os_set_mlock_capability

BUG: unable to handle kernel NULL pointer dereference at virtual address 0000011c
 printing eip:
f93cc6f5
*pde = 00000000
Oops: 0002 [#1]
SMP
Modules linked in: loop usb_storage nvidia(P) agpgart 8021q snd_intel8x0 snd_ac97
CPU:    1
EIP:    0060:[<f93cc6f5>]    Tainted: P       VLI
EFLAGS: 00210246   (2.6.22.1-cfs-v19-cfs-zen-zen #1)
EIP is at os_set_mlock_capability+0xf/0x2d [nvidia]
eax: 00000000   ebx: 00000000   ecx: 00000050   edx: 00200296
esi: f410b800   edi: f62a8400   ebp: c6d19ea0   esp: c6d19ea0
ds: 007b   es: 007b   fs: 00d8  gs: 0033  ss: 0068
Process java (pid: 2522, ti=c6d18000 task=d823e0f0 task.ti=c6d18000)
Stack: e8c01ff0 f90f4d35 f90f6d92 f96ab3c0 e5092740 df730ba0 f90f8c26 e8c00000
       0000005b c6d19f34 f90f5a32 f96ab3c0 e5092740 0000005b df730ba0 df730ba0
       0000005b f96ab3c0 f93c8b44 e8c00000 f96ab3c0 e5092740 0000005b df730ba0
Call Trace:
 [<c010371b>] show_trace_log_lvl+0x1a/0x30
 [<c01037d6>] show_stack_log_lvl+0xa5/0xca
 [<c01039de>] show_registers+0x1e3/0x2dd
 [<c0103bec>] die+0x114/0x22e
 [<c01123cb>] do_page_fault+0x2c4/0x5c9
 [<c03d4642>] error_code+0x72/0x78
 [<f90f4d35>] _nv002553rm+0x5/0x10 [nvidia]
 =======================
Code: e5 c6 83 c3 01 39 f3 75 ef eb af 55 89 e5 64 a1 00 d0 54 c0 8b 80 a8 00 00 $
EIP: [<f93cc6f5>] os_set_mlock_capability+0xf/0x2d [nvidia] SS:ESP 0068:c6d19ea0

I reverted to an older nvidia kernel/glx but it didn't change anything. I reverted xorg and it still doesn't help.
I'm wondering what I could revert now...
Oh, and this is on kernel 2.6.22.1 on which I added CFS...
It was working fine with the exact same kernel...
Any idea what could be wrong ?

[slide_rule]

What version of nvidia drivers are you guys using? I'm having the same problem (my kernel oops looks almost exactly like masterzen's) with linux 2.6.23.9-1 and nividia-100.14.19. Has there been any improvement since October? (i'll post a proper bugreport when I get home from work)

[edit]
I uploaded a tar.bz2 file that includes the nvidia bug report and the dmesg that I captured of the kernel oops.