nV News Forums


nV News Forums (http://www.nvnews.net/vbulletin/index.php)
-   NVIDIA Linux (http://www.nvnews.net/vbulletin/forumdisplay.php?f=14)
-   -   Ubuntu 11.04 HPC with Z68/i7-2600k Mobo and Multiple Nvidia GTX cards (http://www.nvnews.net/vbulletin/showthread.php?t=164599)

jjamulla 07-21-11 05:22 AM

Ubuntu 11.04 HPC with Z68/i7-2600k Mobo and Multiple Nvidia GTX cards
Hi - I'm new here so I apologize if I'm in the wrong area.

I want to be able to install as many as 3 NVIDIA GTX (say 2 580s and a 460) cards in my Asus P8Z68-V Pro Mobo running Ubuntu 11.04, 64-bit with a i7-2600k CPU, for the purposes of running Einstein@Home.

I want to be able to use the On-board Intel Graphics for display, and the 3 cards for computation.

I think I've proved to myself that the actual physical installation is possible (size, heat, power wise now, and I can actually use the 3rd PCIe X16 slot (running at X4 though).
I bought a 1200W PS, and a special case).

I have the machine up and running currently with Ubuntu 11.04 and with BOINC running just from the CPUs, so all is well there. So far BOINC doesn't see any usable GPUs, even when I have the GTX 460 running my display. So this is a problem and I don't know why.

I finally figured out how to get a GTX 460 I already had, "working" in the 3rd X16 slot (for display only and recognized by the driver available through Ubuntu) by messing with the /etc/X11/xorg.conf file, and having Ubuntu install the driver, and then making sure the kernel module was being pulled from the right place.

First problem - The driver wouldn't initialize (nvidia from ubuntu) until I actually connected a monitor cable to the card?? Is there any way around that?
I don't want to do that, I want X/monitor output from the On-Chip Intel GPU.
And I don't know what will happen if I install the other graphics cards, will they not been seen until each of them has a monitor connected??

2nd problem - I have all the CUDA dev stuff loaded, and I can compile the GPU Compute SDK samples, but when I run them, (bandwidth test) doesn't find a CUDA enabled card (error 3 I think, didn't look it up yet). I hope this is becuase of the gcc 4.5?? (I'll try to go load gcc 4.3 or 4.4)

So - The big question is, does anyone have direct experience on how to get this sort of thing going, hopefully step by step, successfully?

I'd be very happy if someone told me what tricks to use in order to get driver up and going and recognize my GPUs without connecting monitors to them.

Is there anywhere that it's documented what the actual CAPABILITIES of the linux NVIDIA drivers are? Like what you can and can't do with them?

Any help would be greatly appreciated. Next step currently is to try to get gcc 4.3 or 4.4 loaded and see what happens then.

Thanks in Advance!

cheechr1 07-21-11 06:51 PM

Re: Ubuntu 11.04 HPC with Z68/i7-2600k Mobo and Multiple Nvidia GTX cards
1st issue you have:

I had this issue as well when I installed three GTX480's, also pretty much for GPU computing.

What worked for me was adding vmalloc=256M to my grub line in my current kernel. Example:

In your grub.cfg in the /boot/grub folder you will see a line similar to this:

menuentry 'Ubuntu, with Linux 2.6.38-10-generic-pae' --class ubuntu --class gnu-linux --class gnu --class os {
set gfxpayload=$linux_gfx_mode
insmod part_msdos
insmod ext2
set root='(/dev/sda,msdos1)'
search --no-floppy --fs-uuid --set=root 04f18a36-ca87-46e9-9705-1c7335a2c74a
linux /boot/vmlinuz-2.6.38-10-generic-pae root=UUID=04f18a36-ca87-46e9-9705-1c7335a2c74a ro quiet splash vt.handoff=7 vmalloc=256M
initrd /boot/initrd.img-2.6.38-10-generic-pae

Thats what allowed the driver to work for my cards. Notice where the vmalloc line is? The bunk thing about using this line in this way is that you have to check after everytime you think something may have messed with grub.cfg that your latest default kernel has this line or your cards wont work.

You should get pyrit from the repos to check if your cards are identified, it will probably be the easiest.

That should also solve your #2 problem, making the cards known to the system. Peace

P.S. heres a link to fix the GCC 4.5 and Cuda compilation junk cause I had that issue, right here: http://forums.nvidia.com/index.php?showtopic=196892

All times are GMT -5. The time now is 10:32 AM.

Powered by vBulletin® Version 3.7.1
Copyright ©2000 - 2015, Jelsoft Enterprises Ltd.
Copyright 1998 - 2014, nV News.