Go Back   nV News Forums > Linux Support Forums > NVIDIA FreeBSD

Newegg Daily Deals

Reply
 
Thread Tools
Old 09-20-08, 02:52 AM   #1
spaghetti
Registered User
 
Join Date: Sep 2008
Posts: 4
Default zfs and nvidia on FreeBSD x86?

I'm trying to get zfs (with kernel config as described in http://wiki.freebsd.org/ZFSTuningGuide) and the nvidia driver working. Anybody had any success with this?

My xorg is hanging at start with 100% cpu but I can ssh in and kill it. Everything works but is pathetically slow when using the nv driver.

I get the "NVRM Xid..." errors after trying to start xorg.

Freebsd 7.1 beta x86 with latest nvidia driver (also tried from March 08 and Dec 07)
vm.kmem_size="1536M"
vm.kmem_size_max="1536M"
options KVA_PAGES=512
system has 3GB ram

thanks
spaghetti is offline   Reply With Quote
Old 09-20-08, 11:05 AM   #2
norbot
Registered User
 
Join Date: Feb 2007
Posts: 9
Default Re: zfs and nvidia on FreeBSD x86?

Unfortunately all I can do is confirm your experience. I have the same issues after increasing KVA_PAGES. The best I can do is leave the kernel alone and max out kmem in loader.conf. I use

vm.kmem_size="512M"
vm.kmem_size_max="512M"
vfs.zfs.arc_max="160M"
vfs.zfs.arc_min="100M"
vfs.zfs.vdev.cache.size="5M"
vfs.zfs.prefetch_disable="1"

I get reasonable (not indefinite) uptimes with this but my workload is light.
norbot is offline   Reply With Quote
Old 09-20-08, 11:09 AM   #3
norbot
Registered User
 
Join Date: Feb 2007
Posts: 9
Default Re: zfs and nvidia on FreeBSD x86?

One more caution. Do not increase kmem past 512M with the default KVA_PAGES setting. If you do, your system will not boot. If you find yourself in this situation, you can recover by overriding these settings from the boot loaders command line.

Cheers.
norbot is offline   Reply With Quote
Old 09-20-08, 02:25 PM   #4
spaghetti
Registered User
 
Join Date: Sep 2008
Posts: 4
Default Re: zfs and nvidia on FreeBSD x86?

thanks norbot, I appreciate the confirmation.

when you say reasonable uptimes do you mean days, weeks or months...? When it finally does crash is there any risk of data corruption?

ian
spaghetti is offline   Reply With Quote
Old 09-21-08, 02:01 PM   #5
norbot
Registered User
 
Join Date: Feb 2007
Posts: 9
Default Re: zfs and nvidia on FreeBSD x86?

A little background. The kmem setting controls how much memory is available for various kernel tasks. kmem is a shared memory pool. If one kernel task uses a lot, less is available to others. When the kernel runs of this type of memory, it panics and crashes.

Kernel tasks typically use different amounts of kmem at different times. They will allocate more when they need it and release it when they are done. ZFS is very hungry for this type of memory. The "arc" settings try to put a limit on how much ZFS will try to claim. Unfortunately, these limits are not absolute, they are targets. ZFS will sometimes exceed these limits for short periods of time.

In my workload (general desktop), another large consumer of kmem is the nvidia driver. The driver will allocate kmem to support graphic applications. When I experience panics, it is typically because both ZFS and nvidia are exerting memory pressure at the same time.


Quote:
Originally Posted by spaghetti View Post
when you say reasonable uptimes do you mean days, weeks or months...?
Many days if I am careful about what I am doing.


Quote:
Originally Posted by spaghetti View Post
When it finally does crash is there any risk of data corruption?
I use ZFS for (almost) everything. ZFS is very robust when it comes to crashes. I have not yet experienced any data loss or corruption.

ZFS is great. This the reason why I subject myself to this instability. It is very unfortunate that ZFS and nvidia do not play nicely together. If only the nvidia driver supported larger KVA_PAGES settings ...
norbot is offline   Reply With Quote
Old 09-25-08, 01:59 PM   #6
zBeeble42
Registered User
 
Join Date: Feb 2008
Posts: 40
Default Re: zfs and nvidia on FreeBSD x86?

I can weigh in here. Based on using the 7.1-BETA and on using the following settings gleaned from the FreeBSD ZFS tuning wiki page regarding 32bit systems with memory pressure...

vfs.zfs.debug=1
vm.kmem_size="330M"
vm.kmem_size_max="330M"
vfs.zfs.arc_max="40M"
vfs.zfs.vdev.cache.size="5M"

... and I find everything somewhat stable.

Now... I occaisionally use wine to play eve-online while in FreeBSD. I find the experience somewhat variable. Beyond the fact that the nvidia driver has some serious drawbacks supporting my hardware (8700M-SLI), I find that X resource usage will sometime prohibit allocation of shaders (or I think that's what fails).

I find that sometimes I have to exit some hogs first. I can almost never run eve if xchat is running. If I've been too busy with firefox, closing it sometimes helps... although unlike xchat, it can run as long as I don't have too many tabs open .

On my system, I have a UFS root and ZFS for everything else.
zBeeble42 is offline   Reply With Quote
Reply


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 05:41 AM.


Powered by vBulletin® Version 3.7.1
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Copyright 1998 - 2014, nV News.