Kernel oops with two NVS-400 cards

[chrispayne]

I'm attempting to get dual NVS-400 video cards (with the goal of 8 monitors total) working in a system, and can reproducibly cause a kernel oops. This system is running Scientific Linux 4.2 (equivalent to RedHat Enterprise 4), running a 2.6.9-22.0.2.EL kernel. I have been running a single NVS-400 using the 1.0-8178 driver with 4 monitors attached without too much difficulty, but now am attempting to add the second card and an additional 4 monitors.

I have tried using both a single processor, as well as SMP kernel, but both result in an oops. The reason I even attempted an SMP kernel on this single Pentium 4 system this is due to some other strange behavior I have seen lately, see http://www.nvnews.net/vbulletin/showthread.php?t=63946

Grasping at straws, I tried various versions of pci=biosirq, pci=noacpi etc, but I always get the oops.

Unfortunately, since the kernel oops' before X starts, I cannot get a nvidia-bug-report with the system in a bad state. I have included the kernel oops (from /var/log/messages) as well as an nvidia-bug-report after the hard reboot, so as to include the basic config, versions, logs etc, for both a SMP and non-SMP case.

Has anyone managed to get multiple NVS-400's running in a single system? Is this even possible?

Cheers
Chris

[netllama]

I've not personally come across anyone trying to use this configuration, and I don't believe that its something that has been tested before.

A few things to check on:
0) Do you have the latest BIOS for the motherboard?
1) Does a newer kernel help at all (such as 2.6.14.6)?
2) Does a 64bit distribution help at all?

Thanks,
Lonni

[chrispayne]

Originally Posted by netllama

I've not personally come across anyone trying to use this configuration, and I don't believe that its something that has been tested before.

A few things to check on:
0) Do you have the latest BIOS for the motherboard?
1) Does a newer kernel help at all (such as 2.6.14.6)?

I verified (again) that the bios is Intel's most recent. I then dowloaded the latest kernel tarball (2.6.15.2) from a mirror, and configured/complied it (using the RedHat base configuration). Things are better now, with 3 of the 4 GPUs being properly initialized, allowing for 6 monitors (I think, I have only been switching cables around, I have not setup for 8 monitors yet. (attached is the nvidia-bug-report with 3 GPUs working)

Originally Posted by netllama

2) Does a 64bit distribution help at all?

Do you think that would help on this 32bit system?

I am presently recompiling with SMP support, as I staed above it seemed to help in another case. I'll keep you posted.

Cheers
Chris

[netllama]

The new bug report makes the current problem much more clear:
############
NVRM: This PCI I/O region assigned to your NVIDIA device is invalid:
NVRM: BAR0 is 0M @ 0x00000000 (PCI:0003:04.0)
NVRM: The system BIOS may have misconfigured your graphics card.
nvidia: probe of 0000:03:04.0 failed with error -1
NVRM: The NVIDIA probe routine failed for 1 device(s).

(EE) NVIDIA(6): Failed to determine FB and I/O addresses!
(EE) NVIDIA(6): *** Aborting ***
(EE) NVIDIA(7): Failed to determine FB and I/O addresses!
(EE) NVIDIA(7): *** Aborting ***
############

Either the kernel, or the BIOS (more likely) has a bug which is preventing the last GPU from getting the resources it needs. This doesn't look like an nvidia driver bug, I'm afraid.

-Lonni

[chrispayne]

Well, I manged to get both NVS-400 cards to work, but in a different machine with a newer Intel motherboard. The working setup is:

Kernel: 2.6.9-22.0.2.EL (RedHat/SL errata kernel)
Motherboard: Intel 945GTP (latest bios, 3309 I think)
NVidia Driver: 1.0-8178

I attached the nvidia-bug-report.log for anyone who is interested in all the details. I suspect netlama was correct in suggesting this is a bios issue with the 865 based motherboard, but I have not pursued this with Intel at this time.

I have only had this setup working for a few hours, but I did notice that some applications are very slow to redraw, others seem fine. Firefox, for example, can take several seconds to redraw after a window is dragged over it, while some of our custom applications redraw almost instantly (as expected). During this the X cpu usage goes to almost 100%. Is there some config option I have set incorrectly?

Also, KDE is the window manager if that is relevant.

Cheers
Chris

[chrispayne]

Originally Posted by chrispayne

I have only had this setup working for a few hours, but I did notice that some applications are very slow to redraw, others seem fine. Firefox, for example, can take several seconds to redraw after a window is dragged over it, while some of our custom applications redraw almost instantly (as expected). During this the X cpu usage goes to almost 100%. Is there some config option I have set incorrectly?

I played around with some config options and found that using twinview mode has increased performance significantly. nvidia-bug-report.log attached for anyone interested.

BTW, an 8 monitor Matrix screen saver on a couple machines in a control room makes you feel like you actually are the "Operator" in "The Matrix".

Cheers
Chris