Go Back   nV News Forums > Linux Support Forums > NVIDIA Linux

Newegg Daily Deals

Reply
 
Thread Tools
Old 07-05-07, 03:28 PM   #1
jbarsellotti
Registered User
 
Join Date: Jul 2007
Posts: 5
Default Only 2 out of 5 GPU's starting

We are trying to drive 9 monitors (a 3x3 array) using two Quadro (NVS 440) and one GeForce 7300GS.
Motherboard - Asus Striker Extreme
CPU - Intel 6700 quad core
OS - Fedora 7 (and briefly Ubuntu 7 - same result)

The gpu's for the first Quadro are PCI:3:0:0 and PCI:4:0:0
For the second Quadro PCI:10:0:0 and PCI:11:0:0
And for the GeForce PCI:6:0:0

It appears that only the first two gpu's found in the "ServerLayout" section are started. We have changed the layout and a different two are started.

We also tried each of pci=noacpi, pci=biosirq, noapic and acpi=off - none seem to have any effect.

For gpu's that fail to start Xorg.0.log simply states -

"Failed to initialize the NVIDIA graphics device PCI:4:0:0"
"Failed to initialize the NVIDIA graphics device PCI:10:0:0"
"Failed to initialize the NVIDIA graphics device PCI:11:0:0"

In this case the gpu's at PCI:3:0:0 and PCI:6:0:0 are starting

Please see the attachments -

xorg.conf
Xorg.0.log
cards (the result of "cat /proc/driver/nvidia/cards/*")
Attached Files
File Type: log Xorg.0.log (60.1 KB, 103 views)
File Type: txt cards.txt (817 Bytes, 86 views)
File Type: txt xorg.conf.txt (7.2 KB, 111 views)
File Type: txt Xorg.0.log.txt (60.1 KB, 80 views)
jbarsellotti is offline   Reply With Quote
Old 07-05-07, 03:31 PM   #2
netllama
NVIDIA Corporation
 
Join Date: Dec 2004
Posts: 8,763
Default Re: Only 2 out of 5 GPU's starting

I'd guess that you're hitting the kernel's vmalloc limitation, however without a bug report, I have insufficient information to determine what is wrong. I'd suggest reading the driver README as well as the forum sticky posts.
netllama is offline   Reply With Quote
Old 07-05-07, 03:31 PM   #3
zander
NVIDIA Corporation
 
zander's Avatar
 
Join Date: Aug 2002
Posts: 3,740
Default Re: Only 2 out of 5 GPU's starting

Please generate and attach a complete nvidia-bug-report.log file after the problem has reproduced (see http://www.nvnews.net/vbulletin/showthread.php?t=46678); please also see the section on kernel virtual memory exhaustion in the README (http://us.download.nvidia.com/XFree8...hapter-09.html).
zander is offline   Reply With Quote
Old 07-06-07, 08:40 AM   #4
jbarsellotti
Registered User
 
Join Date: Jul 2007
Posts: 5
Default Re: Only 2 out of 5 GPU's starting

4 out of 5 gpu's working !!!

I first tried vmalloc=256MB alone which caused the nvidia kernel module not to load. Adding the uppermem line fixed it and 4 gpu's started. I added another 128MB (which should have been ample) but still only 4. I know gpu 5 is good. It seems there must be some other problem.

grub now looks like this

title Fedora (2.6.21-1.3228.fc7)
uppermem 524288
root (hd0,4)
kernel /boot/vmlinuz-2.6.21-1.3228.fc7 ro root=/dev/md5 video=vesaff vga=normal vmalloc=384MB
initrd /boot/initrd-2.6.21-1.3228.fc7.img

I have posted the bug report

john
Attached Files
File Type: gz nvidia-bug-report.log.gz (109.1 KB, 103 views)
jbarsellotti is offline   Reply With Quote
Old 07-06-07, 09:33 AM   #5
netllama
NVIDIA Corporation
 
Join Date: Dec 2004
Posts: 8,763
Default Re: Only 2 out of 5 GPU's starting

The only thing that I see in the bug report is:
(EE) NVIDIA(8): Failed to initialize the NVIDIA graphics device PCI:11:0:0.
(EE) NVIDIA(8): Please see the COMMON PROBLEMS section in the README for
(EE) NVIDIA(8): additional information.
(EE) NVIDIA(8): Failed to initialize the NVIDIA graphics device!

Was this bug report generated while the problem was present?
netllama is offline   Reply With Quote
Old 07-06-07, 10:49 AM   #6
jbarsellotti
Registered User
 
Join Date: Jul 2007
Posts: 5
Default Re: Only 2 out of 5 GPU's starting

The problem has changed from the original. Since adding "vmalloc=256MB" 4 out of 5 gpu's are starting. Increasing to 384MB did not help.

When I did "startx -- -logverbose 6" 4 out of 5 gpu's started. I then created the log.

john
jbarsellotti is offline   Reply With Quote
Old 07-06-07, 10:52 AM   #7
netllama
NVIDIA Corporation
 
Join Date: Dec 2004
Posts: 8,763
Default Re: Only 2 out of 5 GPU's starting

This doesn't look like a vmalloc exhaustion issue any longer (as you noted). The only error in your last bug report was what I noted above.

Its possible that your motherboard isn't capable of initializing 5 GPUs. Have you verified that you're using the latest BIOS?
netllama is offline   Reply With Quote
Old 07-06-07, 01:59 PM   #8
jbarsellotti
Registered User
 
Join Date: Jul 2007
Posts: 5
Default Re: Only 2 out of 5 GPU's starting

The bios is the latest (as is everything else). One thing I didn't mention before because I thought it would be easy to straighten once all gpu's are working - but now I'm not so sure.

If we label the intended layout as (a 3x3 array)

1 2 3
4 5 6
7 8 9

With 4 out of 5 gpu's we get

1 2 7
6 6 7
x x 3

It seems the "ServerLayout" section is being ignored. I changed from relative to absolute layout as follows (and changed the default layout to layout1)

Section "ServerLayout"
Identifier "Layout1"
Screen 0 "Screen0" 0 0
Screen 1 "Screen1" 1280 0
Screen 0 "Screen2" 2560 0
Screen 1 "Screen3" 0 1024
Screen 0 "Screen4" 1280 1024
Screen 1 "Screen5" 2560 1024
Screen 0 "Screen6" 0 2048
Screen 1 "Screen7" 1280 2048
Screen 0 "Screen8" 2560 2048
InputDevice "Keyboard0" "CoreKeyboard"
InputDevice "Mouse0" "CorePointer"
EndSection

and got

1 2 5
6 4 7
x x 3

Rather odd - but at least it can be fixed by moving cables


jb
jbarsellotti is offline   Reply With Quote

Old 07-07-07, 11:46 AM   #9
jbarsellotti
Registered User
 
Join Date: Jul 2007
Posts: 5
Default Re: Only 2 out of 5 GPU's starting

I've decided to abandon this - at least for now.

This morning I installed a 64 bit OS and all gpu's are starting.

In the past, every attempt to run 64 bit failed because there were just too many things I couldn't get to work properly. One of them was xinerama! It was just too easy to switch back to 32 bit where everything worked perfectly (that's when I was using only four monitors). Xinerama is now working and I'll be more persistent with the rest.

Thanks for the help

jb
jbarsellotti is offline   Reply With Quote
Reply


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


Similar Threads
Thread Thread Starter Forum Replies Last Post
Maintain Your Privacy by Manually Accepting and Rejecting "Cookies" (nV News) MikeC Open Forum 2 02-02-13 07:15 PM
Fujitsu Marries GPUs and Xeon in High-Density HPC Server News Archived News Items 0 06-13-12 06:30 PM
Life on Cloud Kepler: GPUs and Heterogenic Computing News Archived News Items 0 05-23-12 07:40 PM
Speeding EDA with GPUs News Archived News Items 0 05-06-12 06:50 PM
Academic Conference on GPUs Debuts Just Ahead of GTC News Archived News Items 0 05-04-12 08:20 PM

All times are GMT -5. The time now is 12:28 PM.


Powered by vBulletin® Version 3.7.1
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Copyright 1998 - 2014, nV News.