View Single Post
Old 05-20-11, 02:51 PM   #114
moenchmeyer
Registered User
 
Join Date: Oct 2007
Posts: 30
Default Re: Nvidia support for Xen

In my last two contributions to this thread I described problems with the nvidia driver and Xen in an Opensuse 11.4 environment. To get the nvidia driver (version 270.41.06) running with kernel 2.6.37.6-0.5-xen (x86_64) was no problem. However, the terminals tty1 to tty6 were no longer usable after the start of the X server as all the ttys besides tty7 appeared completely black.

The good news is, that, eventually, I got Opensuse 11.4 and it's XEN kernel running together with the latest nvidia driver in Dom0. I now have a graphical KDE desktop with 3D acceleration on tty7 - without running into the previous trouble with getting blank, black ttys (tty1 to tty6, tty10) after the start of the X server. A switch between the graphical desktop on tty7 and the other ttys by using CTRL ALT Fn or "chvt n", is possible, now, without any problems - and I can configure and use any physically available uvesafb mode on the ttys.

The hint of Armedblade to use the "uvesafb" module before starting X was quite right and the key to the solution. Many thanks to Armedblade !

But one has to get the "v86d" helper daemon running, too. And for me, who had never used "uvesafb" before, it was a bit of experimenting. As a complete overview over the steps necessary may be helpful for others, I'll describe briefly what steps are to be performed to set up an Opensuse 11.4 XEN system with the nvidia driver in the Dom0:

Step 1 - Installation of Opensuse with the Xen packages:
Install Opensuse 11.4 regularly, but check the option "hostserver for Xen Virtual Machine" when configuring the SW packages just before starting the installation. (Otherwise you must install the virtualization packages and the Xen kernel packages afterwards.) In addition install the kernel sources and compilers to be able to compile kernel modules later on.

Step 2 - after a boot of the default kernel (not the Xen kernel):
Get the latest nvidia driver (in my case 270.41.06). Try to install it in runlevel 3 by using the installation routine (sh NVIDIA-Linux-x86_64-270.41.06.run). The installation probably complains about the already running Nouveau driver. Answer positively to the suggestion of the nvidia setup program to install a file for the deactivation of the Nouveau driver.

Step 3 - after a boot of the default kernel:
Deactivate KMS, e.g. by using Yast and the editor for /etc/sysconfig: Set the option NO_KMS_INITRD under "system > kernel" to "yes". (Alternatively, you can edit the file "/etc/sysconfig/kernel", accordingly.)
Reboot into level 3 - but use the kernel option "nomodeset" in the grub dialog before booting - just to be on the safe side regarding KMS deactivation. Install the nvidia driver again and check that it is working correctly for the normal Opensuse 11.4 kernel. You can deinstall the Nouveau driver packages, now, if you want to.

Step 4 - boot the XEN kernel and install the nvidia driver in the XEN Dom0:
Reboot the system. In the Grub menu, switch to the option for starting XEN. Then boot the XEN kernel into runlevel 3. Login and install (and thereby recompile) the Nvidia driver module for the XEN kernel by performing

cd Your_Directory_With_THE_Nvidia_Driver
export IGNORE_XEN_PRESENCE=1 SYSSRC=/usr/src/linux-2.6.37.6-0.5 SYSOUT=/usr/src/linux-2.6.37.6-0.5-obj/x86_64/xen

sh ./NVIDIA-Linux-x86_64-270.41.06.run

See previous contributions in this thread for the meaning of the environment variables IGNORE_XEN_PRESENCE, SYSSRC and SYSOUT. The kernel versions in the above statements of course depend on what you actually have installed on your system. (The above is valid after a regular Opensuse 11.4 installation without any kernel updates.)

The compilation and installation of the nvidia driver module should work without any problems.

Step 5 - in the XEN environment (Dom0): Start the X server
Now, change to runlevel 5 (init 5). The X server should start successfully on tty7. Unfortunately, a CTRL ALT Fx or a "chvt x" afterwards leads to a black terminal ttyx. I.e., the tty is invisible - although, actually, it still is active and processes commands from the keyboard. Therefore, you can get back to your graphical desktop environment at any time by pressing CTRL ALT F7.

Step 6 - in the XEN Dom0: Verify that you have an uvesafb.ko module
Back again in in your graphical desktop on tty7, verify that the uvesafb.ko module is present in
/lib/modules/2.6.37.6-0.5-xen/kernel/drivers/video.

Opensuse's XEN kernel was compiled and installed together with this module. So, you should have "uvesafb" present on your Opensuse 11.4 system. If not, you must recompile the Xen kernel with this module and install it.

Step 7 - in the XEN Dom0: Install v86d
Now, get the sources for "v86d" (v86d0.1.10.tar.bz2) from here:
http://dev.gentoo.org/~spock/projects/uvesafb/

Unfortunately, there is no RPM for the helper daemon "v86d", which you could use for Opensuse 11.4 - at least I did not find any. However, you need a working "v86d", otherwise "uvesafb" will not work !

Therefore, extract the downloaded source archive as root and go through the standard steps "./configure" and "make". With "./configure" use the option "--with-x86emu". The compilation and linking should work without problems. Copy the resulting executable file "v86d" into the directory "/sbin".

Step 8 - in the XEN Dom0: uvesafb modes
Find out about the codes for available uvesafb (framebuffer) modes by entering the command:

cat /sys/bus/platform/drivers/uvesafb/uvesafb.0/vbe_modes

Step 9 - in the XEN Dom0: Create an uvesafb.conf file
As root create a configuration file "uvesafb.conf" in the directory "/etc/modprobe.d/". Set your preferred uvesafb mode (e.g. 1280x800-32), which you later on want to use for the ttys, by inserting the following line into the file :

options uvesafb mode_option=1280x800-32 scroll=ywrap


Step 10 - in the XEN Dom0: Modify your grub configuration and eliminate vesa vga parameters
Now, change your Grub boot loader configuration: Remove any "vga" parameters as
"vga=mode-0xnnn" or "vga=0xnnn" for the boot entries of the Xen kernel. "nnn" here represents your chosen standard vesa mode for your ttys. (You can use Yast for modifying the boot loader configuration or edit grub's boot loader files directly).

This step is an important and necessary step ! The standard vesa modes would later interfere with the "uvesafb" framebuffer modes. The standard vesa setup is not compatible with "uvesafb"!

Step 11 - Reboot Xen and load the uvesafb module:
Reboot the Xen kernel into runlevel 3. Of course, now, your boot messages on tty1 appear in a coarse 80x25 resolution as you have removed the vesa vga option in your boot configuration. Do not worry about that. You will use your uvesafb mode instead in a minute! At runlevel 3 login as root.

Now, try "modprobe uvesafb" and watch the changes. Check the last lines in "/var/log/messages" to verify that the uvesafb module was loaded without any errors. "uvesafb" should have used "v86d", recognized the screen and set your preferred terminal mode from the "uvesafb.conf" file. I.e., the terminal tty1 and all other ttys (2-6 and 10) should now appear in your chosen uvesafb mode - e.g. the mode 1280x800-32, if you chose that one.

Step 12 - enjoy your graphical desktop with 3D acceleration and switch to visible ttys (tty1 to tty6, tty10) whenever you like :
Use startx or go to runlevel 5 to start your X server. Verify, that a CTRL ALT Fx now leads to a visible tty in your chosen uvesafb mode. And be happy with the 3D acceleration of your nvidia card in your graphical desktop environment - whatever you need it for in a XEN Dom0!

Remaining steps - automize your boot up procedure
Modify your boot up scripts such that "uvesafb" is loaded before the X server is started.


Useful links:
https://wiki.archlinux.org/index.php/Uvesafb
http://dev.gentoo.org/~spock/projects/uvesafb/

Last edited by moenchmeyer; 05-20-11 at 04:40 PM. Reason: Correction of step 4
moenchmeyer is offline   Reply With Quote