NVIDIA/Linux setup for VirtualGL on node with no monitor

[mwd]

I was wondering if anyone has a NVIDIA card setup on a Visualization node
that does not have a monitor.

We have a M2070Q on a Viz node for remote graphics via VirtualGL/TuboVNC
and this configuration works without a problem from one workstation to another
workstation.

The main differences are:
1. Use of the M2070Q on the Viz node
2. Monitor on the workstation and booting to X11

Both are running the same NVIDIA 290 driver (but i have also tried the 270).

I was not sure if I was missing something with the M2070Q configuration
or some special settings.

Here is what I see from vglrun with trace:

vglrun +tr -d :1 glxspheres64
[VGL] dlopen (filename=libc.so.6 flag=1 retval=0x7f51b71664c8)
[VGL] dlopen (filename=NULL flag=257 retval=0x3be9a21188)
[VGL] dlopen (filename=libselinux.so.1 flag=1 retval=0x019325c0)
[VGL] dlopen (filename=NULL flag=1 retval=0x3be9a21188)
[VGL] dlopen (filename=/lib64/libc.so.6 flag=1 retval=0x7f51b71664c8)
[VGL] dlopen (filename=/lib64/libdl.so.2 flag=1 retval=0x7f51b7662660)
[VGL] dlopen (filename=/lib64/libpthread.so.0 flag=1 retval=0x7f51b7165000)
[VGL] dlopen (filename=libnvidia-tls.so.290.10 flag=1 retval=0x7f51b7164000)
[VGL] dlopen (filename=NULL flag=1 retval=0x3be9a21188)
[VGL] dlopen (filename=libc.so.6 flag=1 retval=0x7f51b71664c8)
Polygons in scene: 62464
[VGL] XOpenDisplay (name=NULL
[VGL] XQueryExtension (dpy=0x01946080(:1.0) name=XKEYBOARD *major_opcode=0 *fi
rst_event=170 *first_error=1 ) 0.042915 ms
[VGL] dpy=0x01946080(:1.0) ) 1.899958 ms
[VGL] glXChooseVisual (dpy=0x01946080(:1.0) screen=0 attrib_list=[0x0004 0x0008=
0x0008 0x0009=0x0008 0x000a=0x0008 0x000c=0x0001 0x0005 ] Xlib: extension "GLX"
missing on display ":1.0".
ERROR (596): Could not obtain RGB visual with requested properties

vglrun glxinfo -v
Just returns the; lib: extension "GLX" missing on display ":1.0"

Any hints would be appreciated.

Mark

[snash]

Mark,

Have you configured the xorg.conf to run without a monitor? The easiest way to do that is to use:
nvidia-xconfig -a --virtual=1024x768 --use-display-device=none

[Plagman]

Please also post the Xorg.log file from the NVIDIA-driven X Server.

[mwd]

Thank you both. I tried the xconfig command. This is with the HP NVIDIA driver 270.
Also below that I tried with the 290 driver directly from NVIDIA.

If I try 'startx' I get:
(EE) No devices detected
Fatal server error: no screens found


nvidia-xconfig -a --virtual=1024x768 --use-display-device=none

The /etc/X11/xorg.conf is:
# nvidia-xconfig: X configuration file generated by nvidia-xconfig
# nvidia-xconfig: version 270.41.22 (buildmeister@swio-display-x86-rhel47-06.nvidia.com) Tue May 31 02:16:0
3 PDT 2011

Section "ServerLayout"
Identifier "Layout0"
Screen 0 "Screen0"
InputDevice "Keyboard0" "CoreKeyboard"
InputDevice "Mouse0" "CorePointer"
EndSection

Section "Files"
FontPath "/usr/share/fonts/default/Type1"
EndSection

Section "InputDevice"

# generated from default
Identifier "Mouse0"
Driver "mouse"
Option "Protocol" "auto"
Option "Device" "/dev/input/mice"
Option "Emulate3Buttons" "no"
Option "ZAxisMapping" "4 5"
EndSection

Section "InputDevice"

# generated from data in "/etc/sysconfig/keyboard"
Identifier "Keyboard0"
Driver "kbd"
Option "XkbLayout" "us"
Option "XkbModel" "pc105"
EndSection

Section "Monitor"
Identifier "Monitor0"
VendorName "Unknown"
ModelName "Unknown"
HorizSync 28.0 - 33.0
VertRefresh 43.0 - 72.0
Option "DPMS"
EndSection

Section "Device"
Identifier "Device0"
Driver "nvidia"
VendorName "NVIDIA Corporation"
BoardName "Tesla M2070-Q"
EndSection

Section "Screen"
Identifier "Screen0"
Device "Device0"
Monitor "Monitor0"
DefaultDepth 24
Option "UseDisplayDevice" "none"
SubSection "Display"
Virtual 1024 768
Depth 24
EndSubSection
EndSection


/usr/bin/nvidia-installer -i
Welcome to the NVIDIA Software Installer for Unix/Linux
Tagging shared libraries with chcon -t textrel_shlib_t.

The currently installed driver is: 'NVIDIA Accelerated Graphics Driver for Linux-x86_64' (version:
270.41.22).

Also I have selinux in permissive mode for now.


lsmod | grep nv
nvidia 10715578 0
i2c_core 31276 5 nvidia,radeon,drm_kms_helper,drm,i2c_algo_bit


[ 1341.188]
X.Org X Server 1.10.4
Release Date: 2011-08-19
[ 1341.194] X Protocol Version 11, Revision 0
[ 1341.196] Build Operating System: jetta 2.6.32-220.el6.x86_64
[ 1341.198] Current Operating System: Linux orb.princeton.edu 2.6.32-220.2.1.el6.x86_64 #1 SMP Fri Dec 23 23:
48:54 EST 2011 x86_64
[ 1341.201] Kernel command line: ro root=/dev/mapper/vg_orb-lv_root rd_NO_LUKS rd_LVM_LV=vg_orb/lv_root rd_NO
_MD quiet SYSFONT=latarcyrheb-sun16 rhgb crashkernel=135M@0M KEYBOARDTYPE=pc KEYTABLE=us rd_LVM_LV=vg_orb/lv_
swap rd_NO_DM LANG=en_US.UTF-8 rdblacklist=nouveau
[ 1341.206] Build Date: 14 December 2011 11:04:56PM
[ 1341.208] Build ID: xorg-x11-server 1.10.4-6.el6_2.1
[ 1341.211] Current version of pixman: 0.18.4
[ 1341.214] Before reporting problems, check https://www.redhat.com/apps/support/
to make sure that you have the latest version.
[ 1341.219] Markers: (--) probed, (**) from config file, (==) default setting,
(++) from command line, (!!) notice, (II) informational,
(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
[ 1341.228] (==) Log file: "/var/log/Xorg.0.log", Time: Wed Jan 18 15:20:12 2012
[ 1341.231] (==) Using config file: "/etc/X11/xorg.conf"
[ 1341.235] (==) ServerLayout "Layout0"
[ 1341.235] (**) |-->Screen "Screen0" (0)
[ 1341.235] (**) | |-->Monitor "Monitor0"
[ 1341.235] (**) | |-->Device "Device0"
[ 1341.235] (**) |-->Input Device "Keyboard0"
[ 1341.235] (**) |-->Input Device "Mouse0"
[ 1341.235] (==) Automatically adding devices
[ 1341.235] (==) Automatically enabling devices
[ 1341.235] (**) FontPath set to:
/usr/share/fonts/default/Type1,
catalogue:/etc/X11/fontpath.d,
built-ins
[ 1341.235] (==) ModulePath set to "/usr/lib64/xorg/modules"
[ 1341.235] (WW) Hotplugging is on, devices using drivers 'kbd', 'mouse' or 'vmmouse' will be disabled.
[ 1341.235] (WW) Disabling Keyboard0
[ 1341.235] (WW) Disabling Mouse0
[ 1341.235] (II) Loader magic: 0x7d6d00
[ 1341.235] (II) Module ABI versions:
[ 1341.235] X.Org ANSI C Emulation: 0.4
[ 1341.235] X.Org Video Driver: 10.0
[ 1341.235] X.Org XInput driver : 12.2
[ 1341.235] X.Org Server Extension : 5.0
[ 1341.238] (--) PCI:*(0:1:3:0) 1002:515e:103c:31fb rev 2, Mem @ 0xe0000000/134217728, 0xf99f0000/65536, I/O
@ 0x00002000/256, BIOS @ 0x????????/131072
[ 1341.238] (--) PCI: (0:6:0:0) 10de:06df:10de:087f rev 163, Mem @ 0xfa000000/33554432, 0xf0000000/134217728,
0xec000000/67108864, I/O @ 0x00005000/128, BIOS @ 0x????????/524288
[ 1341.238] (II) LoadModule: "extmod"
[ 1341.238] (II) Loading /usr/lib64/xorg/modules/extensions/libextmod.so
[ 1341.239] (II) Module extmod: vendor="X.Org Foundation"
[ 1341.239] compiled for 1.10.4, module version = 1.0.0
[ 1341.239] Module class: X.Org Server Extension
[ 1341.239] ABI class: X.Org Server Extension, version 5.0
[ 1341.239] (II) Loading extension SELinux
[ 1341.239] (II) Loading extension MIT-SCREEN-SAVER
[ 1341.239] (II) Loading extension XFree86-VidModeExtension
[ 1341.239] (II) Loading extension XFree86-DGA
[ 1341.239] (II) Loading extension DPMS
[ 1341.239] (II) Loading extension XVideo
[ 1341.239] (II) Loading extension XVideo-MotionCompensation
[ 1341.239] (II) Loading extension X-Resource
[ 1341.239] (II) LoadModule: "dbe"
[ 1341.239] (II) Loading /usr/lib64/xorg/modules/extensions/libdbe.so
[ 1341.239] (II) Module dbe: vendor="X.Org Foundation"
[ 1341.239] compiled for 1.10.4, module version = 1.0.0
[ 1341.239] Module class: X.Org Server Extension
[ 1341.239] ABI class: X.Org Server Extension, version 5.0
[ 1341.239] (II) Loading extension DOUBLE-BUFFER
[ 1341.239] (II) LoadModule: "glx"
[ 1341.239] (II) Loading /usr/lib64/xorg/modules/extensions/libglx.so
[ 1341.249] (II) Module glx: vendor="NVIDIA Corporation"
[ 1341.249] compiled for 4.0.2, module version = 1.0.0
[ 1341.249] Module class: X.Org Server Extension
[ 1341.249] (II) NVIDIA GLX Module 270.41.22 Tue May 31 02:12:18 PDT 2011
[ 1341.249] (II) Loading extension GLX
[ 1341.249] (II) LoadModule: "record"
[ 1341.249] (II) Loading /usr/lib64/xorg/modules/extensions/librecord.so
[ 1341.249] (II) Module record: vendor="X.Org Foundation"
[ 1341.249] compiled for 1.10.4, module version = 1.13.0
[ 1341.249] Module class: X.Org Server Extension
[ 1341.249] ABI class: X.Org Server Extension, version 5.0
[ 1341.249] (II) Loading extension RECORD
[ 1341.249] (II) LoadModule: "dri"
[ 1341.250] (II) Loading /usr/lib64/xorg/modules/extensions/libdri.so
[ 1341.250] (II) Module dri: vendor="X.Org Foundation"
[ 1341.250] compiled for 1.10.4, module version = 1.0.0
[ 1341.250] ABI class: X.Org Server Extension, version 5.0
[ 1341.250] (II) Loading extension XFree86-DRI
[ 1341.250] (II) LoadModule: "dri2"
[ 1341.250] (II) Loading /usr/lib64/xorg/modules/extensions/libdri2.so
[ 1341.250] (II) Module dri2: vendor="X.Org Foundation"
[ 1341.250] compiled for 1.10.4, module version = 1.2.0
[ 1341.250] ABI class: X.Org Server Extension, version 5.0
[ 1341.250] (II) Loading extension DRI2
[ 1341.250] (II) LoadModule: "nvidia"
[ 1341.250] (II) Loading /usr/lib64/xorg/modules/drivers/nvidia_drv.so
[ 1341.251] (II) Module nvidia: vendor="NVIDIA Corporation"
[ 1341.251] compiled for 4.0.2, module version = 1.0.0
[ 1341.251] Module class: X.Org Video Driver
[ 1341.251] (II) NVIDIA dlloader X Driver 270.41.22 Tue May 31 01:57:27 PDT 2011
[ 1341.251] (II) NVIDIA Unified Driver for all Supported NVIDIA GPUs
[ 1341.251] (--) using VT number 7

[ 1341.265] (EE) No devices detected.
[ 1341.266]
Fatal server error:
[ 1341.266] no screens found
[ 1341.266]
Please consult the Red Hat, Inc. support
at https://www.redhat.com/apps/support/
for help.
[ 1341.266] Please also check the log file at "/var/log/Xorg.0.log" for additional information.
[ 1341.266]
[root@orb log]# lspci -vvv | grep -i M2070
06:00.0 3D controller: nVidia Corporation GF100 [Tesla M2070Q] (rev a3)


================================================== ============
NVIDIA 290 Driver:

/usr/bin/nvidia-installer -i
Welcome to the NVIDIA Software Installer for Unix/Linux
Tagging shared libraries with chcon -t textrel_shlib_t.
The currently installed driver is: 'NVIDIA Accelerated Graphics Driver for Linux-x86_64' (version: 290.10).

[root@orb ~]# lsmod | grep nv
nvidia 12123185 0
i2c_core 31276 5 nvidia,radeon,drm_kms_helper,drm,i2c_algo_bit
[root@orb ~]# ls -ld /dev/nvi*
crw-rw-rw-. 1 root root 195, 0 Jan 18 15:07 /dev/nvidia0
crw-rw-rw-. 1 root root 195, 255 Jan 18 15:07 /dev/nvidiactl


The Log is basically the same:
cat /var/log/Xorg.0.log
[ 241.561]
X.Org X Server 1.10.4
Release Date: 2011-08-19
[ 241.562] X Protocol Version 11, Revision 0
[ 241.562] Build Operating System: jetta 2.6.32-220.el6.x86_64
[ 241.562] Current Operating System: Linux orb.princeton.edu 2.6.32-220.2.1.el6.x86_64 #1 SMP Fri Dec 23 23:48:54 EST 2011 x86_64
[ 241.562] Kernel command line: ro root=/dev/mapper/vg_orb-lv_root rd_NO_LUKS rd_LVM_LV=vg_orb/lv_root rd_NO_MD quiet SYSFONT=latarcyrheb-sun16 rhgb crashkernel=135M@0M KEYBOARDTYPE=pc KEYTABLE=us rd_LVM_LV=vg_orb/lv_swap rd_NO_DM LANG=en_US.UTF-8 rdblacklist=nouveau consoleblank=0 console=tty0 console=ttyS1,19200n8 nouveau.modeset=0
[ 241.562] Build Date: 14 December 2011 11:04:56PM
[ 241.562] Build ID: xorg-x11-server 1.10.4-6.el6_2.1
[ 241.562] Current version of pixman: 0.18.4
[ 241.562] Before reporting problems, check https://www.redhat.com/apps/support/
to make sure that you have the latest version.
[ 241.562] Markers: (--) probed, (**) from config file, (==) default setting,
(++) from command line, (!!) notice, (II) informational,
(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
[ 241.563] (==) Log file: "/var/log/Xorg.0.log", Time: Wed Jan 18 15:45:14 2012
[ 241.563] (==) Using config file: "/etc/X11/xorg.conf"
[ 241.563] (==) ServerLayout "Layout0"
[ 241.563] (**) |-->Screen "Screen0" (0)
[ 241.563] (**) | |-->Monitor "Monitor0"
[ 241.563] (**) | |-->Device "Device0"
[ 241.563] (**) |-->Input Device "Keyboard0"
[ 241.563] (**) |-->Input Device "Mouse0"
[ 241.563] (==) Automatically adding devices
[ 241.563] (==) Automatically enabling devices
[ 241.564] (**) FontPath set to:
/usr/share/fonts/default/Type1,
catalogue:/etc/X11/fontpath.d,
built-ins
[ 241.564] (==) ModulePath set to "/usr/lib64/xorg/modules"
[ 241.564] (WW) Hotplugging is on, devices using drivers 'kbd', 'mouse' or 'vmmouse' will be disabled.
[ 241.564] (WW) Disabling Keyboard0
[ 241.564] (WW) Disabling Mouse0
[ 241.564] (II) Loader magic: 0x7d6d00
[ 241.564] (II) Module ABI versions:
[ 241.564] X.Org ANSI C Emulation: 0.4
[ 241.564] X.Org Video Driver: 10.0
[ 241.564] X.Org XInput driver : 12.2
[ 241.564] X.Org Server Extension : 5.0
[ 241.566] (--) PCI:*(0:1:3:0) 1002:515e:103c:31fb rev 2, Mem @ 0xe0000000/134217728, 0xf99f0000/65536, I/O @ 0x00002000/256, BIOS @ 0x????????/131072
[ 241.566] (--) PCI: (0:6:0:0) 10de:06df:10de:087f rev 163, Mem @ 0xfa000000/33554432, 0xf0000000/134217728, 0xec000000/67108864, I/O @ 0x00005000/128, BIOS @ 0x????????/524288
[ 241.567] (II) LoadModule: "extmod"
[ 241.567] (II) Loading /usr/lib64/xorg/modules/extensions/libextmod.so
[ 241.567] (II) Module extmod: vendor="X.Org Foundation"
[ 241.567] compiled for 1.10.4, module version = 1.0.0
[ 241.567] Module class: X.Org Server Extension
[ 241.567] ABI class: X.Org Server Extension, version 5.0
[ 241.567] (II) Loading extension SELinux
[ 241.567] (II) Loading extension MIT-SCREEN-SAVER
[ 241.567] (II) Loading extension XFree86-VidModeExtension
[ 241.567] (II) Loading extension XFree86-DGA
[ 241.567] (II) Loading extension DPMS
[ 241.567] (II) Loading extension XVideo
[ 241.567] (II) Loading extension XVideo-MotionCompensation
[ 241.567] (II) Loading extension X-Resource
[ 241.567] (II) LoadModule: "dbe"
[ 241.568] (II) Loading /usr/lib64/xorg/modules/extensions/libdbe.so
[ 241.568] (II) Module dbe: vendor="X.Org Foundation"
[ 241.568] compiled for 1.10.4, module version = 1.0.0
[ 241.568] Module class: X.Org Server Extension
[ 241.568] ABI class: X.Org Server Extension, version 5.0
[ 241.568] (II) Loading extension DOUBLE-BUFFER
[ 241.568] (II) LoadModule: "glx"
[ 241.568] (II) Loading /usr/lib64/xorg/modules/extensions/libglx.so
[ 241.580] (II) Module glx: vendor="NVIDIA Corporation"
[ 241.580] compiled for 4.0.2, module version = 1.0.0
[ 241.580] Module class: X.Org Server Extension
[ 241.580] (II) NVIDIA GLX Module 290.10 Wed Nov 16 18:01:24 PST 2011
[ 241.580] (II) Loading extension GLX
[ 241.580] (II) LoadModule: "record"
[ 241.581] (II) Loading /usr/lib64/xorg/modules/extensions/librecord.so
[ 241.581] (II) Module record: vendor="X.Org Foundation"
[ 241.581] compiled for 1.10.4, module version = 1.13.0
[ 241.581] Module class: X.Org Server Extension
[ 241.581] ABI class: X.Org Server Extension, version 5.0
[ 241.581] (II) Loading extension RECORD
[ 241.581] (II) LoadModule: "dri"
[ 241.581] (II) Loading /usr/lib64/xorg/modules/extensions/libdri.so
[ 241.581] (II) Module dri: vendor="X.Org Foundation"
[ 241.581] compiled for 1.10.4, module version = 1.0.0
[ 241.581] ABI class: X.Org Server Extension, version 5.0
[ 241.581] (II) Loading extension XFree86-DRI
[ 241.581] (II) LoadModule: "dri2"
[ 241.581] (II) Loading /usr/lib64/xorg/modules/extensions/libdri2.so
[ 241.581] (II) Module dri2: vendor="X.Org Foundation"
[ 241.581] compiled for 1.10.4, module version = 1.2.0
[ 241.581] ABI class: X.Org Server Extension, version 5.0
[ 241.581] (II) Loading extension DRI2
[ 241.581] (II) LoadModule: "nvidia"
[ 241.582] (II) Loading /usr/lib64/xorg/modules/drivers/nvidia_drv.so
[ 241.582] (II) Module nvidia: vendor="NVIDIA Corporation"
[ 241.582] compiled for 4.0.2, module version = 1.0.0
[ 241.582] Module class: X.Org Video Driver
[ 241.582] (II) NVIDIA dlloader X Driver 290.10 Wed Nov 16 17:41:10 PST 2011
[ 241.582] (II) NVIDIA Unified Driver for all Supported NVIDIA GPUs
[ 241.583] (--) using VT number 7

[ 241.592] (EE) No devices detected.
[ 241.592]
Fatal server error:
[ 241.592] no screens found
[ 241.592]
Please consult the Red Hat, Inc. support
at https://www.redhat.com/apps/support/
for help.
[ 241.592] Please also check the log file at "/var/log/Xorg.0.log" for additional information.
[ 241.592]

[mwd]

Also I noticed that the NVIDIA is not listed as the 'VGA compatible' card..
But as the 3D controller.

lspci -vvv | egrep -i "3D controller|nvidia|VGA comp"

01:03.0 VGA compatible controller: ATI Technologies Inc ES1000 (rev 02) (prog-if 00 [VGA controller])
06:00.0 3D controller: nVidia Corporation GF100 [Tesla M2070Q] (rev a3)
Subsystem: nVidia Corporation Device 087f
Kernel driver in use: nvidia
Kernel modules: nvidia, nouveau

Mark

[snash]

That may be the clue. The M2070Q can be either a 3D Controller, or a VGA device, depending on what VBIOS is flashed on the card.

Since your xorg.conf does not specify the BusID of the device, I think X is getting confused and picking the onboard ATI device instead.

Can you add:


BusID "PCI:06:0:0"

To your Device section in xorg.conf and see if that helps?

[mwd]

Thank you! I did try this earlier. But then the console gets the message:
vgaarb: this pci device is not a vga device

I think your comment on the VBIOS flashed is probably what I am looking for.
I will contact HP and NVIDIA to see what they recommend since this is a new
machine.

Apparently they do not distribute the VBIOS:
http://nvidia.custhelp.com/app/answe.../flash%20vbios

Thank you!

Mark