Go Back   nV News Forums > Linux Support Forums > NVIDIA Linux

Newegg Daily Deals

Reply
 
Thread Tools
Old 04-13-11, 06:21 PM   #1
kometata
Registered User
 
Join Date: Apr 2011
Posts: 12
Default Pair GTX590 are limited to 553 MHZ under Linux

Hi all,
I used 270.40 several days and now 270.41 driver, but I was shocked today because I saw that GTX590 riches only 553 MHZ with these drivers, i.e not more than performance level 2 !?? I performed very hard calculations, but not able to reach level 3. I use Asus GTX590 (612Mhz) and Suse 11.3. Unfortunately, I have already posted benchmark results for MD programs such as for example Amber, NAMD and so on to the corresponding mail lists, but it looks that these were not real numbers. Importantly this affects also our job too!

Any ideas? Could someone help to solve this issue?
Thank you in advance!



P.S. I saw something similar here:
http://www.nvnews.net/vbulletin/show...=159169&page=2
but the card is different..
Attached Files
File Type: gz nvidia-bug-report.log.gz (77.2 KB, 87 views)

Last edited by kometata; 09-11-11 at 10:49 PM.
kometata is offline   Reply With Quote
Old 04-15-11, 10:21 PM   #2
kometata
Registered User
 
Join Date: Apr 2011
Posts: 12
Default Re: GTX590 riches only 553 MHZ with 270.40 and 270.41 drivers

Additional note!

One card works fine! The issue arises when you attach a second GTX590. Then the cards work only up to the level 2 (553Mhz). I don't need of SLI and use the cards for CUDA applications. The power supply should be ok (recommended by Nvidia - AX1200). I had similar problems with beta 270 drivers under Windows, but solved by older driver, thus I think that this is really driver issue.

Someone with similar problems? Any ideas?
kometata is offline   Reply With Quote
Old 04-18-11, 05:32 PM   #3
danix
NVIDIA Corporation
 
danix's Avatar
 
Join Date: Feb 2010
Location: Santa Clara, CA
Posts: 237
Default Re: GTX590 riches only 553 MHZ with 270.40 and 270.41 drivers

Do you mean to say that each card works fine in isolation, but when two cards are installed, they both fail to clock up to the highest level?
danix is offline   Reply With Quote
Old 04-18-11, 05:50 PM   #4
danix
NVIDIA Corporation
 
danix's Avatar
 
Join Date: Feb 2010
Location: Santa Clara, CA
Posts: 237
Default Re: GTX590 riches only 553 MHZ with 270.40 and 270.41 drivers

Actually, now that I think about it, 1200W does seem a bit on the low side for two GTX 590s under load. The GPUs are probably staying clocked down because there's insufficient power. Not sure why this would work for you on Windows - do you still see the higher clocks even when under a heavy GPU load?
danix is offline   Reply With Quote
Old 04-18-11, 11:05 PM   #5
kometata
Registered User
 
Join Date: Apr 2011
Posts: 12
Default Re: GTX590 riches only 553 MHZ with 270.40 and 270.41 drivers

Hi danix,
Thank you very much for your answer!

Quote:
Originally Posted by danix View Post
Do you mean to say that each card works fine in isolation, but when two cards are installed, they both fail to clock up to the highest level?
Yes, exactly.

Quote:
Originally Posted by danix View Post
Actually, now that I think about it, 1200W does seem a bit on the low side for two GTX 590s under load. The GPUs are probably staying clocked down because there's insufficient power. Not sure why this would work for you on Windows - do you still see the higher clocks even when under a heavy GPU load?
Yes, potentially the PSU could be one of the reasons and it sounds logical. The PSU was initially supposed to be the problem by Nvidia Linux team too.. However, I donít thinks that this is PSU problem because:

1) As you can see from the picture below, yes, under Windows there are no problems. I ran Heaven2.1 benchmark and simultaneously CUDA calculations on 4 GPUs and the clocks are fine. Our Cuda applications scale no so good as Heaven bench and SLI, thus about the GPUs load there is no logic (or I am missing something?) to be worse under Linux..

2) The cards work well in QUAD-SLI under Windows and there were numerous of the reviews and customers withhold any problems with +1100W PSUís and even less. So, why the problem is only under Linux?

3) Recently it was reported that the power consumption of GTX590 for a typical CUDA application is about only 260W.

4) I use certificated by Nvidia Quad-SLI PSU- Corsair AX1200.

Is it possible to be some driver limitation due to some power protection? Moreover, the cards not reach level 3 even without any load!? Or just bug in current Linux drivers?
I donít know, but really need of help and will be very thankful if you and someone other can help/fix this issue!! I am sure that this will be important for many others who plan to use these cards for CUDA.

I also kindly ask someone who has 2 GTX590 cards (or can obtain information from somewhere), just to see and report here if they both work at reference speed under Linux (with PSU 1200W or below)? Maybe Nvidia could perform such simple experiment just to be clear?



In the above example, the program Hex never scale more than 30-40% on 4 GPUs, i.e. this is normal.
kometata is offline   Reply With Quote
Old 04-21-11, 03:03 PM   #6
kometata
Registered User
 
Join Date: Apr 2011
Posts: 12
Default Re: GTX590 riches only 553 MHZ with 270.40 and 270.41 drivers

Nvidia team tested 2 GTX590 with 260.19.21 drivers and they worked ok. The suggestion was again the PSU..and even they wrote me that will suggest and probably drop AX1200 from the recommenced PSU list! However, I reproduced their results with 260.19.44 drivers! The two cards work withholds any problems (at full speed) with the older generation drivers!
Thus, the problem is not the PSU, but 270.xx drivers including the official release!!!!!

My problem now is that I am not able to use my CUDA programs with these drivers...and looking for some temporary solution.
So, what kind of solution can be found? How I can try to "unlock" 270.xx drivers. Could some bugfix to be released? My kernel is 2.6.34-12.3, should I try to update the kernel and actually in what direction to think and what tricks can be applied?
Can we expect Nvidia to fix this, problem??

The conclusion is that currently you can use only single GTX590 under Linux for serious calculations, i.e. GTX590 will be not considered for science until/if this problem be fixed.

kometata is offline   Reply With Quote
Old 07-05-11, 01:10 PM   #7
kometata
Registered User
 
Join Date: Apr 2011
Posts: 12
Default Re: Pair GTX590 are limited to 553 MHZ under Linux

Hello,
More than two months ago we had a discussion here about my problem to use two GTX590 cards under Suse 11.3 Linux at full speed of 612Mhz, but the problem is still not resolved from Nvidia. They can reach only to performance level 2 (553Mhz core).
When you plan to fix this terrible problem for your high-end gamer cards and your costumers??????

The last response from Nvidia team was:

" Hi,
I tested again on the same system but this time using 1200W power supply and I could not reproduce the problem. Also I used a different OS, OpenSuse 11.4 x64.
So it strongly appears that there is an issue specific to your system and not a driver bug.
Let me know if I can help further.
Best regards,
...
"
However, I started to doubt about the quality of Nvidia last tests, because I received a message from colleague of mine who has the same problem as my. I use AMD CPU and motherboard, but he tested with several Intel systems. Moreover, he work on Fedora and tested on several versions. For me it is unbelievable too that there is a so big difference between OpenSuse 11.3 (my OS) and 11.4 (Nvidia test). I also tested with several combination of Nvidia cards, and no problem. For instance, I work now with GTX590 in a combination with GTX580. On my second machine I use GTX590 and GTX470 and again no problems. Another and important factor is that another my colleague wish to purchase 10 GTX590 cards for 5 PC's and asked me for advice.. What advice I can give in that situation..?

All of us use GTX 590 for scientific CUDA applications mostly Molecular Dynamics.

Thus, could Nvidia finally realize that THIS IS A DRIVER problem and could they help us to use Nvidia high-end gamer cards for our applications??? How long we should wait for that??

Regards

Last edited by kometata; 09-11-11 at 10:53 PM.
kometata is offline   Reply With Quote
Old 09-11-11, 10:31 PM   #8
kometata
Registered User
 
Join Date: Apr 2011
Posts: 12
Default Re: Pair GTX590 are limited to 553 MHZ under Linux

Quote:
Originally Posted by kometata View Post
The last response from Nvidia team was:

" Hi,
I tested again on the same system but this time using 1200W power supply and I could not reproduce the problem. Also I used a different OS, OpenSuse 11.4 x64.
So it strongly appears that there is an issue specific to your system and not a driver bug.
Let me know if I can help further.
Best regards,
...
"
The same person from Nvidia after FEW months FINALLY found out the problem:

Quote:
Originally Posted by Nvidia
Hi...,
This problem is definitely a bug which I have reproduced and reported to Engineering to investigate and fix.

Since Engineering has a very large workload, it will likely take some time before they can work on this.

If it is acceptable, I will close this case and will inform you when a fix is available to test.

Please let me know.

Best regards,
....
In other words DONíT USE two GTX590 cards under Linux!!

I share this ONLY because there is no official information in the net about this problem.
kometata is offline   Reply With Quote

Old 10-12-11, 10:23 PM   #9
kometata
Registered User
 
Join Date: Apr 2011
Posts: 12
Default Re: GTX590 riches only 553 MHZ with 270.40 and 270.41 drivers

Can you fix this bug? 6 months is reasonable time to wait ?

Do you have any plans for that?
kometata is offline   Reply With Quote
Old 11-02-11, 06:51 AM   #10
stadel
Registered User
 
Join Date: Nov 2011
Posts: 1
Default Re: GTX590 riches only 553 MHZ with 270.40 and 270.41 drivers

We have the exact same problem and we have 20 gtx590 cards in 5 nodes.
Also, we tried unsuccessfully to up the fan speeds on the cards from 50%, all we got was a seg fault. Further, setting Option "Coolbits" "1" did not give us the extra menu tab in nvidia-settings. Any help on these issues would be greatly appreciated! We can probably live with 553 MHz, but it is irritating.

We are running Scientific Linux (carbon):

Linux version 2.6.32-131.12.1.el6.x86_64 (mockbuild@spacewalk.fnal.gov) (gcc version 4.4.5 20110214 (Red Hat 4.4.5-6) (GCC) ) #1 SMP Tue Aug 23 11:13:45 CDT 2011

Here the output regarding nvidia from dmesg.

nvidia: module license 'NVIDIA' taints kernel.
Disabling lock debugging due to kernel taint
ib_qib 0000:0a:00.0: IB0:1 got a lid: 0x4
nvidia 0000:04:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
nvidia 0000:04:00.0: setting latency timer to 64
nvidia 0000:05:00.0: enabling device (0006 -> 0007)
nvidia 0000:05:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
nvidia 0000:05:00.0: setting latency timer to 64
vgaarb: device changed decodes: PCI:0000:05:00.0,olddecodes=io+mem,decodes=nonew ns=none
nvidia 0000:08:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
nvidia 0000:08:00.0: setting latency timer to 64
nvidia 0000:09:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
nvidia 0000:09:00.0: setting latency timer to 64
vgaarb: device changed decodes: PCI:0000:09:00.0,olddecodes=io+mem,decodes=nonew ns=io+mem
vgaarb: transferring owner from PCI:0000:09:00.0 to PCI:0000:8a:00.0
nvidia 0000:85:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
nvidia 0000:85:00.0: setting latency timer to 64
nvidia 0000:86:00.0: enabling device (0006 -> 0007)
nvidia 0000:86:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
nvidia 0000:86:00.0: setting latency timer to 64
vgaarb: device changed decodes: PCI:0000:86:00.0,olddecodes=io+mem,decodes=nonew ns=none
nvidia 0000:89:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
nvidia 0000:89:00.0: setting latency timer to 64
nvidia 0000:8a:00.0: enabling device (0006 -> 0007)
nvidia 0000:8a:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
nvidia 0000:8a:00.0: setting latency timer to 64
vgaarb: device changed decodes: PCI:0000:8a:00.0,olddecodes=io+mem,decodes=nonew ns=none
NVRM: loading NVIDIA UNIX x86_64 Kernel Module 270.41.19 Mon May 16 23:32:08 PDT 2011
stadel is offline   Reply With Quote
Old 11-02-11, 08:55 AM   #11
artem
Registered User
 
Join Date: Jun 2006
Posts: 713
Default Re: GTX590 riches only 553 MHZ with 270.40 and 270.41 drivers

Quote:
Originally Posted by kometata View Post
Someone with similar problems? Any ideas?
I doubt there are more than a few people in the whole world running Linux and GTX590 in SLI.
artem is offline   Reply With Quote
Old 11-03-11, 07:32 PM   #12
kometata
Registered User
 
Join Date: Apr 2011
Posts: 12
Default Re: GTX590's only 553 MHZ under Linux

Quote:
Originally Posted by artem View Post
I doubt there are more than a few people in the whole world running Linux and GTX590 in SLI.
I am not running GTX590 in SLI, just using two cards for CUDA applications. The cards not working under Linux!!! and also there are many other people with the same problem, mainly serious institutions. How you will feel if you have 20 cards and they don't work properly ..

Quote:
Originally Posted by stadel View Post
We have the exact same problem and we have 20 gtx590 cards in 5 nodes.
Also, we tried unsuccessfully to up the fan speeds on the cards from 50%, all we got was a seg fault. Further, setting Option "Coolbits" "1" did not give us the extra menu tab in nvidia-settings. Any help on these issues would be greatly appreciated!
stadel Instead of "Coolbits" "1" use "Coolbits" "5". It will work fine for the fan control. I tested that on many systems.
kometata is offline   Reply With Quote
Reply


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 05:04 AM.


Powered by vBulletin® Version 3.7.1
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Copyright ©1998 - 2014, nV News.