nV News Forums


nV News Forums (http://www.nvnews.net/vbulletin/index.php)
-   NVIDIA FreeBSD (http://www.nvnews.net/vbulletin/forumdisplay.php?f=47)
-   -   zfs and nvidia on FreeBSD x86? (http://www.nvnews.net/vbulletin/showthread.php?t=119706)

spaghetti 09-20-08 02:52 AM

zfs and nvidia on FreeBSD x86?
I'm trying to get zfs (with kernel config as described in http://wiki.freebsd.org/ZFSTuningGuide) and the nvidia driver working. Anybody had any success with this?

My xorg is hanging at start with 100% cpu but I can ssh in and kill it. Everything works but is pathetically slow when using the nv driver.

I get the "NVRM Xid..." errors after trying to start xorg.

Freebsd 7.1 beta x86 with latest nvidia driver (also tried from March 08 and Dec 07)
options KVA_PAGES=512
system has 3GB ram


norbot 09-20-08 11:05 AM

Re: zfs and nvidia on FreeBSD x86?
Unfortunately all I can do is confirm your experience. I have the same issues after increasing KVA_PAGES. The best I can do is leave the kernel alone and max out kmem in loader.conf. I use


I get reasonable (not indefinite) uptimes with this but my workload is light.

norbot 09-20-08 11:09 AM

Re: zfs and nvidia on FreeBSD x86?
One more caution. Do not increase kmem past 512M with the default KVA_PAGES setting. If you do, your system will not boot. If you find yourself in this situation, you can recover by overriding these settings from the boot loaders command line.


spaghetti 09-20-08 02:25 PM

Re: zfs and nvidia on FreeBSD x86?
thanks norbot, I appreciate the confirmation.

when you say reasonable uptimes do you mean days, weeks or months...? When it finally does crash is there any risk of data corruption?


norbot 09-21-08 02:01 PM

Re: zfs and nvidia on FreeBSD x86?
A little background. The kmem setting controls how much memory is available for various kernel tasks. kmem is a shared memory pool. If one kernel task uses a lot, less is available to others. When the kernel runs of this type of memory, it panics and crashes.

Kernel tasks typically use different amounts of kmem at different times. They will allocate more when they need it and release it when they are done. ZFS is very hungry for this type of memory. The "arc" settings try to put a limit on how much ZFS will try to claim. Unfortunately, these limits are not absolute, they are targets. ZFS will sometimes exceed these limits for short periods of time.

In my workload (general desktop), another large consumer of kmem is the nvidia driver. The driver will allocate kmem to support graphic applications. When I experience panics, it is typically because both ZFS and nvidia are exerting memory pressure at the same time.


Originally Posted by spaghetti (Post 1782895)
when you say reasonable uptimes do you mean days, weeks or months...?

Many days if I am careful about what I am doing.


Originally Posted by spaghetti (Post 1782895)
When it finally does crash is there any risk of data corruption?

I use ZFS for (almost) everything. ZFS is very robust when it comes to crashes. I have not yet experienced any data loss or corruption.

ZFS is great. This the reason why I subject myself to this instability. It is very unfortunate that ZFS and nvidia do not play nicely together. If only the nvidia driver supported larger KVA_PAGES settings ...

zBeeble42 09-25-08 01:59 PM

Re: zfs and nvidia on FreeBSD x86?
I can weigh in here. Based on using the 7.1-BETA and on using the following settings gleaned from the FreeBSD ZFS tuning wiki page regarding 32bit systems with memory pressure...


... and I find everything somewhat stable.

Now... I occaisionally use wine to play eve-online while in FreeBSD. I find the experience somewhat variable. Beyond the fact that the nvidia driver has some serious drawbacks supporting my hardware (8700M-SLI), I find that X resource usage will sometime prohibit allocation of shaders (or I think that's what fails).

I find that sometimes I have to exit some hogs first. I can almost never run eve if xchat is running. If I've been too busy with firefox, closing it sometimes helps... although unlike xchat, it can run as long as I don't have too many tabs open :).

On my system, I have a UFS root and ZFS for everything else.

All times are GMT -5. The time now is 07:53 PM.

Powered by vBulletin® Version 3.7.1
Copyright ©2000 - 2015, Jelsoft Enterprises Ltd.
Copyright 1998 - 2014, nV News.