Excessive RSS for Xorg
I am trying to debug a problem in which the RSS of the Xorg process will hold constant for a long period of time, then it will start growing until we are forced to restart the X server (the machine has 2GB and the XServer will consume nearly all of it). We have not been able to reproduce the problem in our development or test systems, but have seen it in production (where the users are getting annoyed!).
When this last happened we ran xrestop and it did not report any significant memory usage (All was < 50MB). We tried closing all of the running applications and the RSS for X was still 750MB!
The machine configurations are RHEL4U4 with the Nvidia driver 1.0-9746, and each has 6 heads supported on an NVS440 and NVS285.
Does anyone have any ideas for how I can debug this problem? I tried a stress test by starting "x11perf -all" on each of the six heads, and although there is some other kind of bug that causes the Xorg to eventually become unresponsive (problem in X or in the Nvidia drivers??), I cannot reproduce the memory utilization problem. I looked at the output of Xorg.log (debuglevel 6) and the only notable entry was "not allocating video overlay" for the second, fourth, and sixth video devices. What does this mean?
Thanks in advance for any advice!