FreeBSD 8.0 amd64 with 8GB RAM (deadlock)

[miwi]

Howdy,

I played a bit with the 64bit driver under FreeBD-8.0 (RELENG_8).
RELEASE works perfect, after updating to RELENG_8 I get a hang, which
means, i can't start xorg with nvidia driver. Hang means that machine
stops responding completely, not reacting to keyboard input or network
pings.

I have added all kernel debug stuff to my kernel [1] in hope to get
any more information. As that didn't really help I then downgraded
day by day to find out which src/sys commit breaks it seems I found
it:
http://svn.freebsd.org/viewvc/base?v...evision=200443

I wrote a mail to Konstantin (kib@) and explained him my problem,
he told me that he need more infomation.
And he also said that he thinks that this commit is not related to
this problem but he liked to know more about this problem.

We attempted to debug the hung machine using kgdb over firewire. We
were able to successfully attach over firewire with kgdb before
starting X, but firewire host lost connection to the target upon start
of the X. It looks like whole machine become dead.

Upon detailed reading of the dmesg, the nvidia card appeared to share
the interrupt with on-board network controller. Disabling the
controller from the BIOS temporary (until next reboot) enabled the X
to start. Unfortunately, after the reboot, one of the USB controllers
was assigned IRQ18, used by the nvidia card, attempt to start X killed
the machine. We disabled USB in BIOS, and again, were able to start X
once until the reboot.

We were unable to found the way to force card to use MSI to confirm
the natural theory of interrupt sharing causing the hang.

Another test was to disable smp support. i've set kern.smp.disabled=1
in loader.conf nvidia started here and works fine.

About my Hardware:
I've a Asus M4A78T-E with 8 GB ram, the onboard gfx card
is disabled via BIOS.
Nvidia card is a: GeForce 7300 GS

[1]

# Debugging for use in -current
options KDB # Enable kernel debugger support.
options DDB # Support DDB.
options GDB # Support remote GDB.
options INVARIANTS # Enable calls of extra sanity checking
options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS
options WITNESS # Enable checks to detect deadlocks and cycles
options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed
options DEBUG_LOCKS
options DEBUG_VFS_LOCKS
options DIAGNOSTIC
options SW_WATCHDOG
#options MP_WATCHDOG
options KSTACK_PAGES=7
options KDB_UNATTENDED
#options BREAK_TO_DEBUGGER
options ALT_BREAK_TO_DEBUGGER

here my dmesg where nvidia failed: [2]
http://nopaste.unixfreunde.de/1130

here my dmesg where nvidia works:
http://unixfreunde.de/~miwi/nvidia/d...1.12_11_22.txt
http://unixfreunde.de/~miwi/nvidia/v...1_12_11_22.txt

pciconf -lv
http://nopaste.unixfreunde.de/1131

nvidia debug script where nvidia was working:
http://unixfreunde.de/~miwi/nvidia/n...-report.log.gz


I found an other people with the same mainboard also 8 GB ram, but a different
gfx card (GeForce 8400 GS), for him works RELENG_8/HEAD fine.

Any Ideas?

[miwi]

sadly my problem is still present with 195.36.15 ;(

[zander]

Unfortunately, I haven't been able to reproduce this problem so far. Is the problem really tied to the 8GB memory configuration? Or does it also reproduce with less RAM?

[miwi]

Originally Posted by zander

Unfortunately, I haven't been able to reproduce this problem so far. Is the problem really tied to the 8GB memory configuration? Or does it also reproduce with less RAM?

it also reproducible with 2, 4 ,6 GB.

[miwi]

zander,

I got this to work now with 2 and 4 gb ram,

i need to added machdep.disable_mtrrs=1 and hw.nvidia.registry.EnableMSI=1
this works fine with 195.36.15. With 6 and 8 GB i still get the Deadlock.

[zander]

Interesting, thanks for the update. The 195.36.15 driver release does not yet support MSI (it ignores the sysctl knob), nor should it make use of MTRRs; so there's a chance that the problem you're seeing is a system-level one. I believe we've seen one hang on PC-BSD with an 8.0-RELEASE-p2 kernel that may be consistent with yours; when I find some time, I'll take a closer look at that.

In the meantime, can you dump your system's static/dynamic MTRR configuration with 6GB/8GB just before starting X?

[zander]

BTW, it's been a while and I don't recall if there's a standard tool for dumping MTRRs. If not, the attached tool should still work. You an build/run it via:

# mv /path/to/show-mtrr.c.txt show-mtrr.c
# gcc -o show-mtrr show-mtrr.c
# ./show-mtrr

The program needs to be run as root.

[Skalla]

I guess you mean

memcontrol list