[SOLVED] 1.0-6111 was causing lock-ups with custom 2.6.8 kernel

[brblueser]

Hi,

I hope this saves some people some headaches (and reboots! ).

I have been using nvidia graphics drivers with FC2 since 1.0-6106, no problems so far. I also have the habit of compilinig my own kernel, just to get rid of some stuff I don't use and include some other stuff I do use.

When I did this with latest kernel from RH (2.6.8-1.521), I found out my system was freezing hard everytime (yes, 100%) I left X -- either to shutdown or even when I logged off back into GDM.

After many attempts (see this thread on Fedora ML), I found out what was the problem: I was disabling high mem support (since I only have 512MB), and this was in turn disabling "Allocate 3rd-level pagetables from highmem (HIGHPTE)". After leaving these options untouched, I was able to do all the tweaking I wanted with my custom kernel (more details on the Fedora ML thread).

So, to nvidia staff: IMHO this should be documented somewhere on the README...

HTH

Andre

[zander]

X locks hard on exit unless the kernel was built with HIGHMEM support? This is a common configuration that's not generally broken. Does the lockup reproduce with vanilla kernel sources?

[brblueser]

No, unmodified 2.6.8-1.521 kernel works just fine (if you mean real vanilla, from kernel.org, I haven't tested). I actually suspect it's not the HIGHMEM settings, but the HIGHPTE one. I found this out by doing it this way:

1. copied kernel-2.6.8-i686.config from configs dir
2. changed processor type to K7 (BTW: still didn't understand why there's no athlon-specific configuration file)
3. changed a couple of stuff (including HIGHMEM and HIGHPTE)

Problem still happened. Since all the other stuff I changed was not core-related (I just turned off some unneeded modules), I turned my attention to HIGHMEM. Re-enabling it still didn't make me happy (lock-ups still happened), and when I diff'ed my .config file with original kernel-2.6.8-i686.config I realized HIGHPTE was turned off on my configuration.

I tried turning it back on and now I can normally get in and out of X, with a heavily customized kernel. Actually, right now my kernel has both HIGHMEM and HIGHPTE enabled -- I didn't test the version with HIGHPTE but without HIGHMEM (I had rebooted so many times I was happy I could stop doing it; I will try it out one more time later).

[zander]

Please test vanilla (as in kernel.org) Linux 2.6.8.1 with the failing configuration file; lowmem configurations are common and expected to work correctly with the driver.

[brblueser]

Ok, will do that. I just confirmed: at least with RH kernel, it would not work, because when I set HIGHMEM to off, HIGHPTE configuration is unavailable, and this leads to the lock-up problem.