no X display, kernel OOPS: nvidia-180 drivers, ubuntu 9.04 (2.6.28-11-generic kernel)

[eradicator]

I'm trying to use NVAGP instead of AGPGART in an attempt to get S3 (suspend to ram) to work on my system. In the process, I disabled agpgart when booting the kernel using 'agp=off'. When X starts, I get the following oops, and my console is not recoverable:

[ 107.735423] NVRM: not using NVAGP, kernel was compiled with GART_IOMMU support!!
[ 107.735442] BUG: unable to handle kernel paging request at ffffe3a400000000
[ 107.735445] IP: [<ffffffff802e380b>] kmem_cache_free+0x4b/0xe0
[ 107.735454] PGD 0
[ 107.735456] Oops: 0000 [#1] SMP
[ 107.735458] last sysfs file: /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/resource
[ 107.735462] Dumping ftrace buffer:
[ 107.735464] (ftrace buffer empty)
[ 107.735465] CPU 0
[ 107.735467] Modules linked in: binfmt_misc ppdev bridge stp bnep video output input_polldev lp parport snd_via82xx gameport snd_via82xx_modem snd_ac97_codec snd_mpu401_uart ac97_bus snd_pcm_oss snd_mixer_oss snd_seq_dummy snd_seq_oss snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device k8temp nvidia(P) pcspkr psmouse snd snd_page_alloc soundcore usbhid i2c_viapro shpchp serio_raw skge ohci1394 ieee1394 floppy fbcon tileblit font bitblit softcursor
[ 107.735489] Pid: 3730, comm: Xorg Tainted: P 2.6.28-11-generic #42-Ubuntu
[ 107.735491] RIP: 0010:[<ffffffff802e380b>] [<ffffffff802e380b>] kmem_cache_free+0x4b/0xe0
[ 107.735495] RSP: 0018:ffff88003edb5a98 EFLAGS: 00010206
[ 107.735497] RAX: 000001a400000000 RBX: ffffe3a400000000 RCX: 0000000000008ce2
[ 107.735499] RDX: ffffe20000000000 RSI: 0000000000000000 RDI: 0000000000000000
[ 107.735501] RBP: ffff88003edb5ac8 R08: 0000000000000006 R09: 0000000000000000
[ 107.735503] R10: ffff88003edb5896 R11: ffff88003edb5888 R12: 00000000ffffffff
[ 107.735505] R13: 0000000000000000 R14: ffff88003d87a000 R15: ffff88003d56c900
[ 107.735507] FS: 00007f8fa442a700(0000) GS:ffffffff80aa3000(0000) knlGS:0000000000000000
[ 107.735509] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 107.735511] CR2: ffffe3a400000000 CR3: 00000000365aa000 CR4: 00000000000006a0
[ 107.735513] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 107.735515] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 107.735518] Process Xorg (pid: 3730, threadinfo ffff88003edb4000, task ffff88003e4bc320)
[ 107.735519] Stack:
[ 107.735521] ffff88003e1d0000 ffffffffa08b0d60 00000000ffffffff 0000000000000001
[ 107.735524] ffff88003d87a000 ffff88003d87ac20 ffff88003edb5b08 ffffffffa04f3ccc
[ 107.735527] ffff88003d87ac10 ffffffffa08b0d60 ffff88003d87a800 0000000000000001
[ 107.735530] Call Trace:
[ 107.735533] [<ffffffffa04f3ccc>] nv_agp_init+0x1ec/0x240 [nvidia]
[ 107.735856] [<ffffffffa0435684>] _nv003981rm+0xf4/0x13c [nvidia]
[ 107.736001] [<ffffffffa043f286>] ? _nv006422rm+0x110/0x13c [nvidia]
[ 107.736001] [<ffffffffa043f15b>] ? _nv006371rm+0x10b/0x126 [nvidia]
[ 107.736001] [<ffffffffa03fc945>] ? _nv005837rm+0x130/0x14e [nvidia]
[ 107.736001] [<ffffffffa03ef12c>] ? _nv008527rm+0x3f/0x2c1 [nvidia]
[ 107.736001] [<ffffffffa03eed9f>] ? _nv008211rm+0x62/0xa3 [nvidia]
[ 107.736001] [<ffffffffa02fa90b>] ? _nv008394rm+0xd/0x12 [nvidia]
[ 107.736001] [<ffffffffa043ef40>] ? _nv002987rm+0x11d/0x1c9 [nvidia]
[ 107.736001] [<ffffffffa043fbc0>] ? _nv002993rm+0x339/0x4d1 [nvidia]
[ 107.736001] [<ffffffffa043a2d8>] ? rm_init_adapter+0x8f/0xe5 [nvidia]
[ 107.736001] [<ffffffffa04f618c>] ? nv_kern_open+0x35c/0x5d0 [nvidia]
[ 107.736001] [<ffffffff802ead8a>] ? chrdev_open+0x10a/0x1c0
[ 107.736001] [<ffffffff802e5908>] ? __dentry_open+0xd8/0x2d0
[ 107.736001] [<ffffffff802eac80>] ? chrdev_open+0x0/0x1c0
[ 107.736001] [<ffffffff802e5be4>] ? nameidata_to_filp+0x44/0x60
[ 107.736001] [<ffffffff802f4c8e>] ? do_filp_open+0x20e/0x930
[ 107.736001] [<ffffffff8041fe1c>] ? strncpy_from_user+0x4c/0x80
[ 107.736001] [<ffffffff8069e569>] ? _spin_lock+0x9/0x10
[ 107.736001] [<ffffffff802ff0b1>] ? alloc_fd+0x121/0x150
[ 107.736001] [<ffffffff802e5711>] ? do_sys_open+0x81/0x150
[ 107.736001] [<ffffffff802e580b>] ? sys_open+0x1b/0x20
[ 107.736001] [<ffffffff8021253a>] ? system_call_fastpath+0x16/0x1b
[ 107.736001] Code: e0 4c 89 75 f0 e8 66 2d f5 ff 48 c1 e8 0c 48 8d 14 c5 00 00 00 00 48 c1 e0 06 48 29 d0 48 ba 00 00 00 00 00 e2 ff ff 48 8d 1c 10 <48> 8b 03 f6 c4 40 74 04 48 8b 5b 10 4c 8b 75 08 9c 58 66 66 90
[ 107.736001] RIP [<ffffffff802e380b>] kmem_cache_free+0x4b/0xe0
[ 107.736001] RSP <ffff88003edb5a98>
[ 107.736001] CR2: ffffe3a400000000
[ 107.736001] ---[ end trace cc6eb618106e332a ]---

[zander]

Thank you for reporting this problem. In order to be able to use NvAGP, you will need to rebuild your kernel without CONFIG_GART_IOMMU. This will also avoid the Oops on your system. I will fix the CONFIG_GART_IOMMU=y case for future NVIDIA Linux graphics driver releases.

[eradicator]

Originally Posted by zander

Thank you for reporting this problem. In order to be able to use NvAGP, you will need to rebuild your kernel without CONFIG_GART_IOMMU. This will also avoid the Oops on your system. I will fix the CONFIG_GART_IOMMU=y case for future NVIDIA Linux graphics driver releases.

I passed 'iommu=off' to the kernel at boot, but the driver still failed to use NVAGP. Why is that?

Unfortunately CONFIG_GART_IOMMU is not trivially disabled on recent kernels. It is enabled by default on all x86_64 builds. I had to manually edit arch/x86/Kconfig to remove the 'if EMBEDDED' to give myself the option to disable it. I am compiling the new kernel now and will report back...

[eradicator]

building with GART_IOMMU disabled produces the same failure oops. Interestingly the nvidia modue still complains about GART_IOMMU even though it is not in the kernel:

root@sephiroth:~# gunzip -dc /proc/config.gz | grep GART
# CONFIG_GART_IOMMU is not set
root@sephiroth:~# dmesg | grep GART
[ 308.675385] NVRM: not using NVAGP, kernel was compiled with GART_IOMMU support!!

[zander]

Did you rebuild the NVIDIA kernel module after changing your kernel configuration?

[eradicator]

Originally Posted by zander

Did you rebuild the NVIDIA kernel module after changing your kernel configuration?

Yes, I rebuilt 180.44 after building the new kernel.

On as side note, if i interrupt the kernel while its in this "no display state while X is starting" by pressing the power button, I see X and then I'm back to the console as the system shuts down.

[zander]

The CONFIG_GART_IOMMU check in the driver is a build-time, rather than a runtime check (i.e. there's an #ifdef). If you still get the same error message after disabling the option, then the kernel development you're building against are likely out of sync with your kernel.

[eradicator]

Ah, you're right... ubuntu handles things in an odd way... when I fix the /lib/modules/.../build directory to point to the right source tree, it builds correctly, and the kernel doesn't oops.

Unfortunately this didn't fix my resume from S3 problem...

[zander]

Are you sure your problems are AGP related? Does the system resume properly if you disable it altogether (i.e. set the NvAGP X driver option to 0)?

[eradicator]

I have not once said that it *is* AGP related (in fact, I don't believe that to be the case at all).

It happens even when X11 is not running, or when X11 is running (with either AGPGART, NVAGP, or no AGP).

It happens with the nvidia kernel module loaded and with the nvidia kernel module not loaded.

This is why I think it might be a video card BIOS issue... especially since it doesn't respond well to the POST request.