Go Back   nV News Forums > Linux Support Forums > NVIDIA Linux

Newegg Daily Deals

Thread Tools
Old 10-13-08, 02:56 PM   #1
Join Date: Aug 2004
Posts: 11
Default ipi_handler usage in 177.80 driver

I've been doing some testing with the 177.80 driver on both
i386 and x86_64 systems, and I noticed a difference in behavior
between the 177.13 and 177.80 drivers that I'd like to inquire about:

- In the 177.13 driver, on just the first start of X after bootup,
there would be one 10 millisecond (approximately) interruption on all
cpus due to the os_raise_smp_barrier() call that comes out of the open
rm_init_adapter() call. After that point, I wouldn't notice anymore
10 millisecond os_raise_smp_barrier()/ipi_handler() occurrences.

- But in the 177.80 driver, I see a 10 millisecond ipi_handler() cross
processor interrupt on all CPUs for every start of X and also
for every stop of X.

Since this type of system interference makes it difficult to execute
any time critical applications on any of the cpus while a stop or start
X is occurring, I was Just wondering if this could somehow be worked
around, or if this change in the 177.80 driver fixed some missing
initialization/cleanup sequence that must be done in this manner?

Thank you for any information.
johnb is offline   Reply With Quote
Old 10-13-08, 03:04 PM   #2
NVIDIA Corporation
zander's Avatar
Join Date: Aug 2002
Posts: 3,740
Default Re: ipi_handler usage in 177.80 driver

I can only speculate without more information (i.e. stack traces from nv_execute_on_all_cpus()), but my best guess is that what you're seeing are the driver's attempts to flush the caches of all CPUs in the system. With the exception of some kernel versions, the driver traditionally gave the kernel the benefit of the doubt and relied on it to perform this operation, but since this has been shown to be broken (in different and sometimes subtle ways) on a wide range of kernels and configurations, we were recently forced to enable the explicit flush logic at all times. If you know the kernel you're using flushes CPU caches correctly (either in response to global_flush_tlb() or implicitly, depending on the kernel's version), you can modify your driver installation to avoid the manual heavy-weight flushes.
zander is offline   Reply With Quote
Old 10-14-08, 07:42 AM   #3
Join Date: Aug 2004
Posts: 11
Default Re: ipi_handler usage in 177.80 driver

Hi zander,

Thanks for the reply.
Here's some more info and stackwalk backs of what I'm seeing,
in case you have the time to take a look.

I should mention that the os_raise_smp_barrier() calls
below don't occur on all systems that I've tested. Only
some of the systems exhibit this behavior.

Arch : x86-64
BIOS : 2004Q3
Brand : Dual Core AMD Opteron(tm) Processor 275
CPUid : AMD Dual Core Opteron (Italy JH-E6), 940-pin, 90nm Processor 275
CPUs : 2 dual-core (4 virtual) at 2.21GHz
Make : TYAN Computer Corp
Memory : 1.9 GB
Model : S2895
SCSI : LSI/Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320
Swap : 5.0 GB
VGA : nVidia Corporation NV40 [GeForce 6800 Ultra/GeForce 6800 GT]
SMP PREEMPT Mon Oct 13 23:53:28 EDT 2008 x86_64

NVRM version: NVIDIA UNIX x86_64 Kernel Module 177.80
Wed Oct 1 14:43:46 PDT 2008
GCC version: gcc version 4.1.2 20070626 (Red Hat 4.1.2-14)

nVidia Corporation NV45 [GeForce 6800 GTO] (rev a2)

Model: Unknown
IRQ: 19
Video BIOS: ??.??.??.??.??
Card Type: PCI-E
DMA Size: 32 bits
DMA Mask: 0xffffffff
Bus Location: 02.00.0

When X is started, the open of /dev/nvidia0 causes the
following os_raise_smp_barrier(), which does:

NV_SMP_CALL_FUNCTION(ipi_handler, NULL, 0)

Call Trace:
[<ffffffffa0005a95>] :nvidias_raise_smp_barrier+0x95/0xa8
[<ffffffffa0355ab3>] :nvidia:_nv004089rm+0x1b/0x35
[<ffffffffa007a83b>] ? :nvidia:_nv014786rm+0xc4/0x354
[<ffffffffa007bc39>] ? :nvidia:_nv014804rm+0x363/0x93c
[<ffffffffa007c372>] ? :nvidia:_nv014805rm+0x160/0x175
[<ffffffffa005c95b>] ? :nvidia:_nv014822rm+0x3c5/0x412
[<ffffffffa0320e49>] ? :nvidia:_nv003879rm+0x831/0xb5e
[<ffffffffa0320469>] ? :nvidia:_nv003836rm+0x8e/0xb7
[<ffffffffa0324ba1>] ? :nvidia:_nv003868rm+0x1e/0x23
[<ffffffffa031faf1>] ? :nvidia:_nv003894rm+0x26c/0x27a
[<ffffffffa00c14f2>] ? :nvidia:_nv006672rm+0xef2/0xfd2
[<ffffffffa01057b5>] ? :nvidia:_nv002480rm+0x355/0x393
[<ffffffffa014bd27>] ? :nvidia:_nv011781rm+0x1b7/0x1c5
[<ffffffffa030bb5a>] ? :nvidia:_nv008872rm+0x261/0x2f4
[<ffffffffa030b497>] ? :nvidia:_nv008527rm+0x62/0xa3
[<ffffffffa020c3c2>] ? :nvidia:_nv008738rm+0xd/0x12
[<ffffffffa0361c39>] ? :nvidia:_nv003071rm+0x11e/0x1e9
[<ffffffffa03627df>] ? :nvidia:_nv003077rm+0x38a/0x527
[<ffffffffa035cd52>] ? :nvidia:rm_init_adapter+0x8f/0xe5
[<ffffffffa0003541>] ? :nvidia:nv_kern_open+0x420/0x58d
[<ffffffff8024eef7>] ? down+0x33/0x38
[<ffffffff8029c82b>] ? chrdev_open+0x183/0x1d6
[<ffffffff807b4d75>] ? _spin_unlock+0x30/0x4b
[<ffffffff8029c6a8>] ? chrdev_open+0x0/0x1d6
[<ffffffff8029801a>] ? __dentry_open+0x167/0x28a
[<ffffffff8029816b>] ? nameidata_to_filp+0x2e/0x3f
[<ffffffff802a4fc9>] ? do_filp_open+0x325/0x744
[<ffffffff802a2c73>] ? getname+0x1b/0x57
[<ffffffff804213aa>] ? __strncpy_from_user+0x27/0x51
[<ffffffff80297cea>] ? get_unused_fd_flags+0x115/0x126
[<ffffffff80297d50>] ? do_sys_open+0x55/0x16e
[<ffffffff80210851>] ? syscall64_enter_hook+0x56/0x58
[<ffffffff8041e5f6>] ? __up_read+0x1c/0x97
[<ffffffff8021423e>] ? syscall_trace_enter_wrapper+0x11/0x3f
[<ffffffff80297e92>] ? sys_open+0x1b/0x1d
[<ffffffff8020c2b4>] ? tracesys+0xd5/0x142

And a similar call occurs on the close of /dev/nvidia0
when X is stopped/killed:

Call Trace:
[<ffffffffa0005a95>] :nvidias_raise_smp_barrier+0x95/0xa8
[<ffffffffa0355ab3>] :nvidia:_nv004089rm+0x1b/0x35
[<ffffffffa007a83b>] ? :nvidia:_nv014786rm+0xc4/0x354
[<ffffffffa007bc39>] ? :nvidia:_nv014804rm+0x363/0x93c
[<ffffffffa007c372>] ? :nvidia:_nv014805rm+0x160/0x175
[<ffffffffa005c95b>] ? :nvidia:_nv014822rm+0x3c5/0x412
[<ffffffffa0320c1d>] ? :nvidia:_nv003879rm+0x605/0xb5e
[<ffffffffa0320469>] ? :nvidia:_nv003836rm+0x8e/0xb7
[<ffffffffa031f7c0>] ? :nvidia:_nv003871rm+0x8b/0x91
[<ffffffffa0362c73>] ? :nvidia:_nv003130rm+0xe8/0x13a
[<ffffffffa035ce37>] ? :nvidia:rm_disable_adapter+0x8f/0xe5
[<ffffffffa000386f>] ? :nvidia:nv_kern_close+0x1c1/0x34b
[<ffffffff807b4e5c>] ? _spin_unlock_irqrestore+0x34/0x50
[<ffffffff8029acca>] ? __fput+0xc5/0x181
[<ffffffff8029afef>] ? fput+0x14/0x16
[<ffffffff80297bca>] ? filp_close+0x66/0x71
[<ffffffff802991ad>] ? sys_close+0x13b/0x181
[<ffffffff8026bda7>] ? hracct_exit_syscall+0x1c/0x52
[<ffffffff8021423e>] ? syscall_trace_enter_wrapper+0x11/0x3f
[<ffffffff8020c2b4>] ? tracesys+0xd5/0x142

Thanks in advance for any additional comments.
johnb is offline   Reply With Quote
Old 10-14-08, 11:39 AM   #4
Registered User
Join Date: May 2006
Posts: 57
Default Re: ipi_handler usage in 177.80 driver

Originally Posted by zander View Post
... you can modify your driver installation to avoid the manual heavy-weight flushes.
hvengel is offline   Reply With Quote

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
295.40 Drivers and Xorg 1:7.6+12 HIGH cpu usage norrland NVIDIA Linux 2 05-13-12 09:11 AM
Redhat 8.0 NVIDIA works - INSTRUCTIONS STEEL1 NVIDIA Linux 267 04-15-03 06:48 PM
Getting the proprietary nvidia driver to run with Debian 3.0 r0 (woody) Katchina404 NVIDIA Linux 9 01-12-03 08:49 AM
failing to initialize NV driver PsyShook NVIDIA Linux 10 09-19-02 12:41 PM
Nvidia Driver causes crash on Standby/Suspend dasher NVIDIA Linux 5 09-16-02 05:57 PM

All times are GMT -5. The time now is 03:19 AM.

Powered by vBulletin® Version 3.7.1
Copyright ©2000 - 2015, Jelsoft Enterprises Ltd.
Copyright 1998 - 2014, nV News.