Kernel module has bad memory allocation on FreeBSD 8.0 amd64 with 8GB RAM

[ChuckAtkins]

System specs:

Dell D630 notebook
NVIDIA Quadro NVS 135M
8GB RAM
FreeBSD 8.0 Release

The nvidia.ko kernel module loads but fails to attach to my graphics card:

nvidia0: NVRM: NVIDIA MEM resource alloc failed, BAR1 @ 0x14
nvidia0: NVRM: NVIDIA hardware alloc failed

I suspect this is due to the 8GB of RAM but then again, if lots of RAM isn't supported then why all the trouble to go 64 bit?

[zander]

vgapci0: <VGA-compatible display> port 0xef00-0xef7f mem 0xf5000000-0xf5ffffff,0xf2000000-0xf3ffffff irq 16 at device 0.0 on pci1
nvidia0: <Quadro NVS 135M> on vgapci0
vgapci0: child nvidia0 requested pci_enable_busmaster
vgapci0: child nvidia0 requested pci_enable_io
vgapci0: 0x10000000 bytes of rid 0x14 res 3 failed (0, 0xffffffffffffffff).
nvidia0: NVRM: NVIDIA MEM resource alloc failed, BAR1 @ 0x14.
nvidia0: NVRM: NVIDIA hardware alloc failed.
device_attach: nvidia0 attach returned 6

The kernel fails to set up the resources for the GPU's BAR1. It would be good to check that the SBIOS assigned reasonable memory ranges to all of the GPU's BARS, and set up the upstream bridges correctly. For what it's worth, 0x10000000 bytes (256MB) is reasonable for BAR1, but the resources printed in the first line of the above output seem to include only BAR0 (16MB) and BAR2 (32MB).

[ChuckAtkins]

From the other posts, it seems like this is a compatibility issue with the 8GB of memory I have in the system. Perhaps related to the "memory hole" problem? Chances are likely that If I downgrade to 2GB that this would start working. That's not really an option though given the workload this machine does. Is it perhaps possible through the sysctl interface to enforce the memory ranges being used?

How can I get more detailed information to help?

[ChuckAtkins]

Also, before you ask, the BIOS doesn't have any options regarding memory remapping.

[zander]

I haven't looked at the kernel code path in question, so I don't know off-hand what it's trying to patch up. I suspect it's trying to allocate resources for a BAR not setup by the SBIOS, but I don't have enough information.

If you can get me the equivalent of the output of `lspci -t` and `lspci -v` after a fresh boot, I'll be able to at least tell if there is an SBIOS setup problem.

[ChuckAtkins]

Here's the output from lspci -t and lspci -v as well as the freebsd native pciconf -lv

[ChuckAtkins]

I'm assuming the sispicious line in lspci -v is:

01:00.0 VGA compatible controller: nVidia Corporation Quadro NVS 135M (rev a1) (prog-if 00 [VGA controller])
Subsystem: Dell Device 01f9
Flags: bus master, fast devsel, latency 0, IRQ 16
Memory at f5000000 (32-bit, non-prefetchable)
Memory at <unassigned> (64-bit, prefetchable)
Memory at f2000000 (64-bit, non-prefetchable)
I/O ports at ef00
Capabilities: [60] Power Management version 2
Capabilities: [68] MSI: Mask- 64bit+ Count=1/1 Enable-
Capabilities: [78] Express Endpoint, MSI 00

[ChuckAtkins]

If it helps any, here's the output of lspci -v from the same machine booted into linux:

01:00.0 VGA compatible controller: nVidia Corporation Quadro NVS 135M (rev a1) (prog-if 00 [VGA controller])
        Subsystem: Dell Device 01f9
        Flags: bus master, fast devsel, latency 0, IRQ 16
        Memory at f5000000 (32-bit, non-prefetchable) [size=16M]
        Memory at e0000000 (64-bit, prefetchable) [size=256M]
        Memory at f2000000 (64-bit, non-prefetchable) [size=32M]
        I/O ports at ef00 [size=128]
        [virtual] Expansion ROM at f4000000 [disabled] [size=128K]
        Capabilities: <access denied>
        Kernel driver in use: nvidia
        Kernel modules: nvidia, nvidiafb

[ChuckAtkins]

This appears to be the same issue they were having with the nouveau driver and an NVS 140M (missing BAR1):

http://www.mail-archive.com/freebsd-...msg102573.html

[zander]

The interesting bits are:

-[0000:00]-+-00.0
+-01.0-[0000:01]----00.0

01:00.0 VGA compatible controller: nVidia Corporation Quadro NVS 135M (rev a1) (prog-if 00 [VGA controller])
Subsystem: Dell Device 01f9
Flags: bus master, fast devsel, latency 0, IRQ 16
Memory at f5000000 (32-bit, non-prefetchable)
Memory at <unassigned> (64-bit, prefetchable)
Memory at f2000000 (64-bit, non-prefetchable)
I/O ports at ef00
Capabilities: [60] Power Management version 2
Capabilities: [68] MSI: Mask- 64bit+ Count=1/1 Enable-
Capabilities: [78] Express Endpoint, MSI 00

00:01.0 PCI bridge: Intel Corporation Mobile PM965/GM965/GL960 PCI Express Root Port (rev 0c) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
I/O behind bridge: 0000e000-0000efff
Memory behind bridge: f2000000-f6efffff
Prefetchable memory behind bridge: 00000000e0000000-00000000efffffff
Capabilities: [88] Subsystem: Dell Device 01f9
Capabilities: [80] Power Management version 3
Capabilities: [90] MSI: Mask- 64bit- Count=1/1 Enable-
Capabilities: [a0] Express Root Port (Slot+), MSI 00

As suspected (and as you pointed out), the GPU's BAR1 does not appear to have been assigned. However, the bridge immediately upstream of the GPU has a non-prefetchable-memory-behind-bridge window sized to accommodate the GPU's 256MB BAR1.

This means that the SBIOS either failed to assign the BAR, despite allocating resources for it, or that the kernel interfered. Since the correct range is reported on Linux, the latter seems more likely to me.

[gianz]

same problem here.

Dell XPS 1330 4GB RAM with GeForce 8400M GS

nvidia-bug-report.log.gz

lspci_v.gz

lspci_t.gz

[Jonathan Chen]

On a thread on freebsd-stable@, John Baldwin has diagnosed this as a conflict between the ACPI resource list and the BIOS.

http://www.mail-archive.com/freebsd-...msg107020.html

I can get the driver to work somewhat if the ACPI resource handling is disabled from loader.conf:
debug.acpi.disabled="sysres"

This works if the internal monitor is used; however, any external monitors are ignored.