PDA

View Full Version : Nvidia cheats on 3DMark with 177.39 drivers !!!


Pages : [1] 2

Vasot02
06-24-08, 05:33 AM
NVIDIA IS VOILATING Futuremark rules with the latest PhysX drivers, and doing it in the usual sleazy way. The rules are simple, violating them isn't a trick, and doing so in order to pump up your numbers is the height of unethical behavior.

Well no, it isn't the height, this is, but calling Nvidia on unethical behavior is what you might deem a target-rich environment. That said, the company's behavior this time could be done in an ethical way, but it chose not to. The explanation needs a little background though, so bear with us.

3DMark Vantage has four major components, two CPU and two GPU. One of the CPU subtests is a physics-based test. The physics test is based on the Ageia PhysX API, a fairly widespread API in use by a large number of games. Between the time that 3DMark Vantage development was started and the time it was released, Nvidia bought Ageia.

The problem is that the PhysX DLLs, and for that matter, the whole API is now owned by Nvidia. In and of itself, this is not a problem, especially if the company involved had a history of honesty, integrity, and fair play. Nvidia has none of these attributes, and has a proven history of cheating on 3DMark.

To be fair, ATI has been caught at the same thing as well, but nothing lately, and Intel compilers come with curious optimisation defaults as well. No one is clean, but only Nvidia seems to take dishonesty as a corporate mandate.

So, with the latest driver, Forceware 177.39 drivers, Nvidia put its now in-house PhysX APIs into the drivers. Instead of it running on the CPU or on the PhysX chip, it is running it on the GPU. It owns the GPU and it's drivers along with the physics API and all those drivers. This is a dangerous situation.

There are two problems with Nvidia doing this, it isn't a legal driver for 3DMark, and it isn't even running the same program as others who run 3DMark. Either one is enough to preclude people from using those drivers and calling the results 3DMark scores.

If you look at the 3DMark Vantage Driver Approval Policy, section 3.5 clearly states, "Based on the specification and design of the CPU tests, GPU make, type or driver version may not have a significant effect on the results of either of the CPU tests as indicated in Section 7.3 of the 3DMark Vantage specification and whitepaper." When you run a CPU test on a GPU, it clearly violates the rules.

The other problem is that when you install the drivers, they replace the PhysX DLLs with a completely different set of DLLs. If you look at the Nvidia PhysX reviewers guide, a PDF that NV hands out to help people write up their newest toys, they say the following for UT3 installation.

http://images.vnu.net/gb/inquirer/news/2008/06/23/nvidia-cheats-3dmark-177/nvidia_ppu_ut3.jpg

That looks completely above board!

Note steps 4 and 5 that say "Uninstall the existing AGEIA PhysX v7.11.13 driver (installs with UT3 installation)." and "Install the new PhysX 8.06.12 driver." Same with 3DMark Vantage, and they offer the helpful hint of "GeForce PhysX is enabled in CPU Test 2. We recommend testing in Performance Preset for the best final score with GeForce PhysX. In Extreme Preset the score is mainly determined by the GPU score. A faster CPU Test 2 result will not make much difference."

This means two things, when you are running 3DMark Vantage with the 177.39 drivers, you are not doing the same work as every other driver running 3DMark Vantage. You are doing a completely different workload on 25 per cent of the tests. To rub salt into the wound, Nvidia then tells you that the Extreme preset, the one meant for high-end GPUs, doesn't show off the cheat sufficiently, so use one that weights it more heavily. What gall.

The end result is about a 10n per cent increase in scores, and a claimed 7.5x advantage on the physics subtest. You can see how they word it for yourself.

http://images.vnu.net/gb/inquirer/news/2008/06/23/nvidia-cheats-3dmark-177/nvidia_ppu_vantage.jpg

If the glove doesn't fit, you must acquit.

If you look at it as a whole, Nvidia is doing two things, first of all doing a different workload than the rest of the world and claiming it is the same, and second not following the Futuremark rules. This is nothing less than blatant dishonesty.

We asked Futuremark for a comment, and they referred us to the above Driver Approval Policy, and then to the approved driver page here.

Nvidia has not submitted 177.39 for approval, and likely will never do so because the chance of it being approved are something between zero and having to buy Futuremark. Until they submit a bad driver, no harm, no foul. The Futuremark policy is that if it isn't up on ORB, it isn't a 3DMark score, and that is quite sensible.

Nvidia engineers know that there is no way they can get this driver approved, so they don't try. They know they are not running 3DMark, but they don't even try to hide it. They are however disingenuously doing a different workload and trying to cynically pass it off as the same old workload. There is a word for this behavior, cheating. µ

Note: At time of this writing, the ATI Catalyst 8.6 drivers are not approved yet, the latest valid set is 8.5. Nvidia's latest approved is 175.16, and Intel isn't even submitting a set, it seems they don't want the world to know something. In any case, this does not preclude all three companies from quoting scores liberally with unapproved drivers.

Bad industry, no cookies for you.

Reviewers take note.

http://www.theinquirer.net/gb/inquirer/news/2008/06/23/nvidia-cheats-3dmark-177

Wolfhound
06-24-08, 05:47 AM
NVIDIA IS VOILATING Futuremark rules with the latest PhysX drivers, and doing it in the usual sleazy way. The rules are simple, violating them isn't a trick, and doing so in order to pump up your numbers is the height of unethical behavior.

Well no, it isn't the height, this is, but calling Nvidia on unethical behavior is what you might deem a target-rich environment. That said, the company's behavior this time could be done in an ethical way, but it chose not to. The explanation needs a little background though, so bear with us.

3DMark Vantage has four major components, two CPU and two GPU. One of the CPU subtests is a physics-based test. The physics test is based on the Ageia PhysX API, a fairly widespread API in use by a large number of games. Between the time that 3DMark Vantage development was started and the time it was released, Nvidia bought Ageia.

The problem is that the PhysX DLLs, and for that matter, the whole API is now owned by Nvidia. In and of itself, this is not a problem, especially if the company involved had a history of honesty, integrity, and fair play. Nvidia has none of these attributes, and has a proven history of cheating on 3DMark.

To be fair, ATI has been caught at the same thing as well, but nothing lately, and Intel compilers come with curious optimisation defaults as well. No one is clean, but only Nvidia seems to take dishonesty as a corporate mandate.

So, with the latest driver, Forceware 177.39 drivers, Nvidia put its now in-house PhysX APIs into the drivers. Instead of it running on the CPU or on the PhysX chip, it is running it on the GPU. It owns the GPU and it's drivers along with the physics API and all those drivers. This is a dangerous situation.

There are two problems with Nvidia doing this, it isn't a legal driver for 3DMark, and it isn't even running the same program as others who run 3DMark. Either one is enough to preclude people from using those drivers and calling the results 3DMark scores.

If you look at the 3DMark Vantage Driver Approval Policy, section 3.5 clearly states, "Based on the specification and design of the CPU tests, GPU make, type or driver version may not have a significant effect on the results of either of the CPU tests as indicated in Section 7.3 of the 3DMark Vantage specification and whitepaper." When you run a CPU test on a GPU, it clearly violates the rules.

The other problem is that when you install the drivers, they replace the PhysX DLLs with a completely different set of DLLs. If you look at the Nvidia PhysX reviewers guide, a PDF that NV hands out to help people write up their newest toys, they say the following for UT3 installation.

http://images.vnu.net/gb/inquirer/news/2008/06/23/nvidia-cheats-3dmark-177/nvidia_ppu_ut3.jpg

That looks completely above board!

Note steps 4 and 5 that say "Uninstall the existing AGEIA PhysX v7.11.13 driver (installs with UT3 installation)." and "Install the new PhysX 8.06.12 driver." Same with 3DMark Vantage, and they offer the helpful hint of "GeForce PhysX is enabled in CPU Test 2. We recommend testing in Performance Preset for the best final score with GeForce PhysX. In Extreme Preset the score is mainly determined by the GPU score. A faster CPU Test 2 result will not make much difference."

This means two things, when you are running 3DMark Vantage with the 177.39 drivers, you are not doing the same work as every other driver running 3DMark Vantage. You are doing a completely different workload on 25 per cent of the tests. To rub salt into the wound, Nvidia then tells you that the Extreme preset, the one meant for high-end GPUs, doesn't show off the cheat sufficiently, so use one that weights it more heavily. What gall.

The end result is about a 10n per cent increase in scores, and a claimed 7.5x advantage on the physics subtest. You can see how they word it for yourself.

http://images.vnu.net/gb/inquirer/news/2008/06/23/nvidia-cheats-3dmark-177/nvidia_ppu_vantage.jpg

If the glove doesn't fit, you must acquit.

If you look at it as a whole, Nvidia is doing two things, first of all doing a different workload than the rest of the world and claiming it is the same, and second not following the Futuremark rules. This is nothing less than blatant dishonesty.

We asked Futuremark for a comment, and they referred us to the above Driver Approval Policy, and then to the approved driver page here.

Nvidia has not submitted 177.39 for approval, and likely will never do so because the chance of it being approved are something between zero and having to buy Futuremark. Until they submit a bad driver, no harm, no foul. The Futuremark policy is that if it isn't up on ORB, it isn't a 3DMark score, and that is quite sensible.

Nvidia engineers know that there is no way they can get this driver approved, so they don't try. They know they are not running 3DMark, but they don't even try to hide it. They are however disingenuously doing a different workload and trying to cynically pass it off as the same old workload. There is a word for this behavior, cheating. µ

Note: At time of this writing, the ATI Catalyst 8.6 drivers are not approved yet, the latest valid set is 8.5. Nvidia's latest approved is 175.16, and Intel isn't even submitting a set, it seems they don't want the world to know something. In any case, this does not preclude all three companies from quoting scores liberally with unapproved drivers.

Bad industry, no cookies for you.

Reviewers take note.

http://www.theinquirer.net/gb/inquirer/news/2008/06/23/nvidia-cheats-3dmark-177

Surely you can do the same with a PhysX PCI card, so I don´t see nvidia´s cheating any where, and more probably be that the driver will not be send to futuremark for approval due to the restrictions that futuremark has, so is probably that they have two drivers one with PhysX dlls and other without for approval, I think you have a mean way of thinking towards Nvidia. Futuremark 3D Mark 06 or Vantage is a sh*t, so why is so serious if (insert brand here) cheats or you think they cheat?, I prefer playing games not having the greatest of all numbers on some benchmarks.

Q
06-24-08, 06:34 AM
I've been to one World Fair, a picnic and a rodeo and that's the stupidest thing I've ever heard!

nekrosoft13
06-24-08, 07:03 AM
inq is so ****ing pathetic

Revs
06-24-08, 07:09 AM
Yeah The Inq. is ever so slightly biased :p

SH64
06-24-08, 07:13 AM
Who gives a **** ? let me know when they cheat in Crysis.

wwec
06-24-08, 07:18 AM
It's Vantage's code that's tripped up if it uses the GPU - NVidia has simply enabled hardware physx in the drivers for general use...

DMA
06-24-08, 07:54 AM
Charlie needs a good beating. What's his problem anyway? :box:

Runningman
06-24-08, 07:58 AM
oh, who cares...its not like we play with 3dmark other then to show how big our peepee's are...

Sowk
06-24-08, 09:06 AM
If nvidia can Cheat at Crysis and make it run at 90+ FPS @1920X1200 with us not being able to see any difference... BRING IT ON. If thats the case, cheat on everything as long as there is no downside to the end-user.

Tygerwoody
06-24-08, 09:16 AM
Who cares? Really? Its not like Nvidia has something to prove. They are the best video card company in the world. As long as the end user isn't damaged in any way(which they aren't), let them do what they want.

LovingSticky
06-24-08, 09:25 AM
INQ is so stupid it *is* even funny :-)

NVIDIA added PhysX support via CUDA to their GPU, which is a great accomplishment! I love having my "PhysX for free" with drivers download.

"When you run a CPU test on a GPU, it clearly violates the rules." -- Why not, if you can?! Blame obsolete ATI drivers for not being able to accelerate PhysX via GPU.

Crow_Nest
06-24-08, 10:01 AM
Who cares just as long as the cheat allows me better framerate in my games.

sniggle
06-24-08, 10:02 AM
Based on the specification and design of the CPU tests, GPU make, type or driver version may not have a significant effect on the results of either of the CPU tests as indicated in Section 7.3 of the 3DMark Vantage specification and whitepaper.

I think this particular situation is a bit unique in the context of this requirement. The past 3dmark "cheats" have been related to optimizations and cheats in the way things get rendered graphically. Magically releasing a driver that makes a 3d score go up by 25% without any detailed whitepaper detailing the specific legal enhancements done to the core driver logic would mean the driver is likely cheating.

In THIS case, it's more of an advancement in how the hardware is being used. I doubt that Futuremark took into account how the drivers would change to allow such an efficient improvement in how hardware gets utilized in a case like this, so they put such a clause in their agreement. However this type of improvement I think is 100% legal because it's tapping into power that was already there with the advent of stream processing, but there was no software yet. Now that we have the software, we can all benefit "legally".

Futuremark simply needs to make a benchmark that is both taxing on 3D and physics/AI at the same time. nVidia can't magically pull extra processing power out of all the stream processors; they can only balance how they get used between handling 3D tasks and physics tasks. A benchmark that stresses both would just be a natural progression of how the hardware and software involved is maturing.

Gimlis
06-24-08, 10:18 AM
Futuremark simply needs to make a benchmark that is both taxing on 3D and physics/AI at the same time. nVidia can't simply magically pull extra processing power out of all the stream processors; they can only balance how they get used between handling 3D tasks and physics tasks. A benchmark that stresses both would just be a natural progression of how the hardware and software involved is maturing.

good post . I've wondered why they always tested them "CPU GPU" seperatly & didn't have a test designed to stress both at once in past releases. and with GPU based physics they may add such a combined test

weeds
06-24-08, 03:50 PM
Charlie needs a good beating. What's his problem anyway? :box:

HA, HA.:D
http://jonpeddie.com/jpr/blogs/attempted_rapprochement/

SLippe
06-24-08, 03:58 PM
So, are the 177.39 drivers the only set that takes advantage or enables the use of PhysX? I tried them (see the other thread), but I'm now using the 175.80 set with same PhysX driver and I can't tell any difference. Is there a demo or easy way to tell if it's working? As for the original post in here, who cares!

Edit: I just uninstalled the PhysX 8.06 drivers and it made no difference in my Rainbow Six Vegas play.

Edit: Nevermind! UT3 and Vantage are the only progs that supports the GeForce PhysX, so far. Duh! No need for me to install it, again, then.

DSC
06-25-08, 09:21 PM
http://www.hothardware.com/News/NVIDIA_Responds_To_GPU_PhysX_Cheating_Allegation/

A word from Futuremark: And what does Futuremark have to say about this? Oliver Baltuch of Futuremark had this to offer with respect to NVIDIA's new technology, 3DMark Vantage and the industry as a whole:
"The driver in question has not been submitted for authorization and is for demo purposes only. NVIDIA has followed the correct rules for driver authorization and the BDP by sending us the 177.35 driver (the same as AMD has sent us the 8.6 driver), both of which are currently undergoing the authorization process in our Quality Assurance area at this moment. Only drivers that have passed WHQL and our driver authorization process have comparable results that will be allowed for use in our ORB database and hall of fame.

Other drivers which have not been submitted will not be commented on. Otherwise, we would have to inspect every Beta and press driver that is released. Outside of this matter, we have been introduced to this technology from NVIDIA and it is truly innovative for future games and game development. As you know, we have an actual game coming as well and it could also make use of PhysX on the GPU.”

Charlie is an asshole idiot with zero understanding of anything. See, even Futuremark is considering using GPU PhysX with their new game in development.

sammy sung
06-25-08, 09:46 PM
Charlie needs a good beating. What's his problem anyway? :box:


Had a small mail exchange with charlie,my last response remain unanswered:



I object to you fanboy,some objectiveness sometimes wouldnt hurt .

----- Original Message -----
From: "Charlie Demerjian" <charlie@stonearch.net>
To: <@telia.com>
Sent: den 12 June 2008 14:59
Subject: Re: Moron much ?


> Wow, pithy reply devoid of any factual statements or things objected to.
> Does mom let you out of the basement much?
>
> @telia.com wrote:
>> Emailed from The Inquirer...
>> http://www.theinquirer.net/gb/inquirer/news/2008/06/12/nvidia-changes-gt200-dates
>>
>> Charlie Demerjian you sir are a moron of gargantuan proportions,hats off
>> to you because when i thought you couldnt possibly make a bigger ass of
>> yourself here you are proving me wrong...thumbs up
>>
>
> __________ NOD32 3180 (20080612) Information __________
>
> This message was checked by NOD32 antivirus system.
> http://www.eset.com

Amuro
06-25-08, 09:53 PM
Can't wait for the game from FM.

RejZoR
06-26-08, 03:00 AM
So how is that cheating? If you'd have actual crappy PhysX card installed it would be ok, but if graphic card stretches the muscles instead for the very same purpose, thats wrong? It's not like the PhysX feature is there just to cheat the score. It can be used in actual games. And since Ageia is no more i don't see a reason why all the "NVIDIA is cheating" trashing again. Thats not cheating, thats extra value to all the >G80 users.
But i guess they just hate extra features without paying extra for them.
I mean they provided PhysX functionality for masses without any need to buy crappy dedicated PhysX card.
It just proves how physics can be nicely (and faster) calculated on GPU.
If Futuremark is unable to properly impliment scoring system, thats not NVIDIA's fault...

BronzeGod
06-26-08, 02:09 PM
Hey guys, cheating is giving the customer more FPS, something ATi hasn't done in a while LOL!

DeusGear
06-26-08, 02:53 PM
This is not cheating. The card just gained a great ability that improves physics and doesn't sacrifice any image quality to get it's no different then using a regular physics card.

chaosmaker
06-27-08, 04:58 AM
http://www.pcper.com/article.php?aid=584

probably the best in-depth discussion between the players you will find

I can't believe people were stupid enough to think nvidia would pay for phys-x and not use it, in some capacity. just because others can't keep up doesn't mean the "leader" is cheating

Sgt_Pitt
07-07-08, 12:25 PM
Who is 3d mark anyway ??

They arent the governing body of 3d benchmarking or anything, they are just another company trying to make a buck with a service.

Nvidia dont have to answer to anyone, they dont have to comply in any way shape or form to any of 3d marks guidelines or tests