install nvidia driver under xen successfully

[honny]

I use xen-3.0-testing(downloaded with hg). My nvidia card is Quadro FX 3400. OS is
Rocks 4.0(i386). The version of Nvidia driver is 1.0.8178, patched with 01-22-2006 patch.
I can play 3D games now. Thank Jacob for help.

Yong

diff -Nur NVIDIA-Linux-x86-1.0-8178-pkg1/usr/src/nv/Makefile.kbuild nv/Makefile.kbuild
--- NVIDIA-Linux-x86-1.0-8178-pkg1/usr/src/nv/Makefile.kbuild 2006-03-15 10:46:52.000000000 +0100
+++ nv/Makefile.kbuild 2006-03-15 12:18:16.000000000 +0100
@@ -262,7 +262,7 @@
# KBUILD build parameters.
#

-KBUILD_PARAMS += KBUILD_VERBOSE=1 -C $(KERNEL_SOURCES) SUBDIRS=$(PWD)
+KBUILD_PARAMS += KBUILD_VERBOSE=1 -C $(KERNEL_SOURCES) SUBDIRS=$(PWD) ARCH=xen

#
# NVIDIA sanity checks.
diff -Nur NVIDIA-Linux-x86-1.0-8178-pkg1/usr/src/nv/nv.c nv/nv.c
--- NVIDIA-Linux-x86-1.0-8178-pkg1/usr/src/nv/nv.c 2006-03-15 10:46:52.000000000 +0100
+++ nv/nv.c 2006-03-15 11:06:11.000000000 +0100
@@ -8,7 +8,6 @@
* _NVRM_COPYRIGHT_END_
*/

-
#include "nv-misc.h"
#include "os-interface.h"
#include "nv-linux.h"
@@ -924,6 +923,10 @@
{
unsigned long pat1, pat2;

+#ifdef CONFIG_XEN
+ nv_pat_enabled=1;
+#endif
+
if (nv_pat_enabled)
return 1;

@@ -952,6 +955,10 @@
{
unsigned long pat1, pat2;

+#ifdef CONFIG_XEN
+ nv_pat_enabled=0;
+#endif
+
if (!nv_pat_enabled)
return;

@@ -2015,7 +2022,11 @@
return -ENXIO;
}

+#ifdef CONFIG_XEN
+ if (io_remap_page_range(vma, vma->vm_start,
+#else
if (NV_REMAP_PAGE_RANGE(vma->vm_start,
+#endif
NV_VMA_OFFSET(vma),
NV_VMA_SIZE(vma),
vma->vm_page_prot))
@@ -2039,8 +2050,11 @@
return -ENXIO;
}
}
-
+#ifdef CONFIG_XEN
+ if (io_remap_page_range(vma, vma->vm_start,
+#else
if (NV_REMAP_PAGE_RANGE(vma->vm_start,
+#endif
NV_VMA_OFFSET(vma),
NV_VMA_SIZE(vma),
vma->vm_page_prot))
@@ -2078,7 +2092,11 @@
NV_ATOMIC_INC(at->usage_count);
nv_up(nvl->at_lock);

+#ifdef CONFIG_XEN
+ if (io_remap_page_range(vma, vma->vm_start,
+#else
if (NV_REMAP_PAGE_RANGE(vma->vm_start,
+#endif
NV_VMA_OFFSET(vma),
NV_VMA_SIZE(vma),
vma->vm_page_prot))
diff -Nur NVIDIA-Linux-x86-1.0-8178-pkg1/usr/src/nv/nv-linux.h nv/nv-linux.h
--- NVIDIA-Linux-x86-1.0-8178-pkg1/usr/src/nv/nv-linux.h 2006-03-15 10:46:52.000000000 +0100
+++ nv/nv-linux.h 2006-03-15 10:58:08.000000000 +0100
@@ -202,9 +202,13 @@
* tiny, and the kernel panics when it is exhausted. try to warn the user that
* they need to boost the size of their pool.
*/
+#ifdef CONFIG_XEN
+#undef CONFIG_SWIOTLB
+#else
#if defined(CONFIG_SWIOTLB)
#define NV_SWIOTLB 1
#endif
+#endif

/*
* early 2.6 kernels changed their swiotlb codepath, running into a
diff -Nur NVIDIA-Linux-x86-1.0-8178-pkg1/usr/src/nv/nv-vm.c nv/nv-vm.c
--- NVIDIA-Linux-x86-1.0-8178-pkg1/usr/src/nv/nv-vm.c 2006-03-15 10:46:52.000000000 +0100
+++ nv/nv-vm.c 2006-03-15 10:59:04.000000000 +0100
@@ -544,7 +544,11 @@
page_ptr = at->page_table[i];
page_ptr->phys_addr = phys_addr;
page_ptr->virt_addr = virt_addr;
+#ifdef CONFIG_XEN
+ page_ptr->dma_addr = phys_to_machine(page_ptr->phys_addr);
+#else
page_ptr->dma_addr = page_ptr->phys_addr;
+#endif

/* lock the page for dma purposes */
nv_lock_page(page_ptr);
diff -Nur NVIDIA-Linux-x86-1.0-8178-pkg1/usr/src/nv/os-agp.c nv/os-agp.c
--- NVIDIA-Linux-x86-1.0-8178-pkg1/usr/src/nv/os-agp.c 2005-12-15 01:57:35.000000000 +0100
+++ nv/os-agp.c 2006-03-15 10:59:15.000000000 +0100
@@ -303,7 +303,11 @@

page_ptr->phys_addr = (ptr->memory[i] & PAGE_MASK);
page_ptr->virt_addr = (unsigned long) __va(page_ptr->phys_addr);
+#ifdef CONFIG_XEN
+ page_ptr->dma_addr = phys_to_machine(page_ptr->phys_addr);
+#else
page_ptr->dma_addr = page_ptr->phys_addr;
+#endif
}

return RM_OK;

[chunkey]

THX!!!

but, one little wish... can you _attach_ the patch instead of posting it?
(sometimes these forums destroies/modifies important parts, and it's easier
for textbrowsers (like lynx) to get the source).

[honny]

here is the patch.

[gilboa]

Does it work?
Last patch I saw did get the driver to compile but when the driver was actually used with xen enabled kernel, all hell broke lose.

Cheers,

[chunkey]

Originally Posted by honny

I use xen-3.0-testing(downloaded with hg). My nvidia card is Quadro FX 3400. OS is
Rocks 4.0(i386). The version of Nvidia driver is 1.0.8178, patched with 01-22-2006 patch. I can play 3D games now. Thank Jacob for help.

Yong

But, I'm testing it.

[honny]

attention: Dom0 kernel must export symbol xen_tlb_flush, Be sure that ARCH=xen can work.

Now I have run some 3D programs.

[gilboa]

Baah! I should really work on my reading skills.

FC5 is just round the corner (Monday?) with Xen/x86_64 support.
I'll give it a shot when it's out.

Thanks.

[pe1chl]

I am trying to get a mental picture of what you are trying to do...
Is the nvidia driver running as an isolated device driver in XEN, or are you running Linux in a XEN virtual machine and have the nvidia driver loaded in that vm?
I am not (yet) a XEN user, I use VMware. I know how it is done there. When running another OS under VMware, the videocard is a "VMware emulated videocard", not the real nvidia, so I don't load the driver for it. The driver is in the host OS, which is Linux in my case. But there it is just used as part of the X server which displays the VMware window.
But (I think) there is no 3D acceleration present in the VMware virtual machine, which would be tunneled through to the nvidia driver in the Linux host OS.
Assuming you are using an isolated device driver, how are the VMs going to communicate with the driver? Is there a unified driver interface for accelerated 3D drivers?
Or are you (for now) only using the 3D from the host OS?

[honny]

Xen is paravirtualization. So it needs to install nvidia driver into dom0 at least.
You can see Xen documents to get more infomation.

Originally Posted by pe1chl

Is the nvidia driver running as an isolated device driver in XEN, or are you running Linux in a XEN virtual machine and have the nvidia driver loaded in that vm?
Assuming you are using an isolated device driver, how are the VMs going to communicate with the driver? Is there a unified driver interface for accelerated 3D drivers?
Or are you (for now) only using the 3D from the host OS?

[deBeuk]

Originally Posted by gilboa

Baah! I should really work on my reading skills.

FC5 is just round the corner (Monday?) with Xen/x86_64 support.
I'll give it a shot when it's out.

Thanks.

I've installed FC5-test3, but it seems amd64-agp is unsupported in xen, and it's probably not something that's going to get fixed before FC5 gets released.
A shame really, the nvidia driver currently doesn't support the onboard K8T800 Pro AGP bridge on my mobo.
Overall performance with a normal non-hypervisor kernel was a bit better than on FC4 x86_64 fwiw.

[gilboa]

... Don't need no AGP... me on PCI-E (Dev machine in sig)
The only problem I foresee, is vmware. I use it dialy.
AFAIK vmware doesn't play well with Xen...

Cheers,

[pe1chl]

Originally Posted by honny

You can see Xen documents to get more infomation.

I browsed some XEN documents before posting that reply, and read about isolated device drivers, but what I do not yet understand is how a virtual machine can talk to such a device driver without having some agent driver loaded that takes the place of the usual video driver in that machine.
This agent presumably is not the full nvidia driver but some thin layer that takes all the functioncalls a full driver can handle, and passes them trough XEN to the isolated driver.
I wondered if full 3D acceleration is available in such an environment in current versions of XEN.

But well, it will probably be some time before I can switch from VMware to XEN and probably I will learn enough in the meantime to answer this question myself.