NVIDIA's offical response to Futuremark

[StealthHawk]

Originally posted by eagle17
I still say it was a cheat and I don't think anyone is fooled buy this... this is just as bad as the quake/quack problem ati had.

It's worse by far. Quack only existed(with degraded IQ) in one driver set. How many driver sets did nvidia have which either "cheated" or had some bugs lowering IQ in them? At least 5.

[jAkUp]

i dont think there is really any good way to test the different fps in games anymore. 3dmark01 used to be good... (besides the fact it was WAY WAY to oversensitive to fsb overclocks... but at default fsb its pretty good.) 3dmark03 is trash imo... its nice to tweak your score and see how your system compares to someone else's with the same hardware... it looks decent... but i feel its poorly optimized, and not really a good idea of where games are headed. (the ragdoll physics test, and the sound test, i felt was a needed feature in benchies, which is one of the strongpoints of 3dmark03, again imho)

remember when quake2 had that ingame benchmark? i think it was ~timerefresh... or something like that... where the camera would spin around? theres no way to fix that benchie... because you could always do it in a different spot... is that accurate? why dont the newer games have something like this?

[ragejg]

Originally posted by solofly
This is almost better than a movie...

troo-dat...

rest o yr post made sense too...

it's a sick sick world, huh?

this thread's gettin a little bloated...

[GlowStick]

Originally posted by jAkUp
i dont think there is really any good way to test the different fps in games anymore. 3dmark01 used to be good... 3dmark03 is trash imo...

remember when quake2 had that ingame benchmark? i think it was ~timerefresh... or something like that... where the camera would spin around? theres no way to fix that benchie... because you could always do it in a different spot... is that accurate? why dont the newer games have something like this?

OH yeah i rember that, it was a pritty cool feature.

Didnt someone make a map thats a octogon with a bunch of polys and scripted it so a bunch of guys just run in circles

[Jovi]

Originally posted by StealthHawk
Let's leave what happens on Rage3d over at Rage3d, shall we?
And what purpose would that serve if it is relevent to the conversation at hand? I have read at least a dozen sites listed here and neary a mention to them. :confused:

[Red Dog]

Futuremark = SELL OUT!

Nvidia needs a complete management shake up.

Talk about a quagmire of questionable business ethics.

needs to be some serious bitch slapping. Talk about ruining a corporations reputation, the last 6 months have been infuriating for an Nvidia fan.

[StealthHawk]

Originally posted by bkswaney
OK.... So NV's card cannot do this... hummm maybe they should. ;)

Well, at least FM fixed it. So now the 9800 and 5900 r very close in 3DM03. :)

PowerVR uses a fundamentally different architecture than any other 3d card IHV. PowerVR uses tile based rendering, which makes it easier to do hidden surface removal and render only what is seen in any given scene.

nvidia, ATI, et al have already invested time and money in making immediate mode renderers more efficient, I'd be surprised if they switched to TBRs.

[muzz]

I hope they at least got a reach around........ and were able to apply a thick wad of KY B4 hand.

[StealthHawk]

Originally posted by Neural
Ive just been reading over everyones thoughts, so just thought i would mention a few of my own. I know I'm new here so flame away if you like.

First, part of me thinks detecting a benchmark and changing its default paths is "wrong", whether "wrong" means "cheat" or "optimization". I would sincerely like to know if someone here could could explain how nVidia could hide the clip panes in their drivers?

*BUT* there may be another way of looking at this.

I think that whether we like it or not, all the big players in software and hardware think it is acceptable to optimize software code so that certain hardware runs better/faster. Intel has been doing the same thing (working to optimize code for their own products) for years as nVidia is trying to do, and for the same reasons as nVidia. Intel even has specific names for their optimizations, i.e. SSE, SSE2, 3.... and intel would be blown out in most applications by AMD if it werent for these optimizations even in BENCHMARKS where you can bet Intel is making sure their optimized instructions aer implemented. While this does effectively assure that AMD will never control the market, I still find it hard to fault intel for making its products work better as long as windows still looks like windows, and UT2K3 still plays like UT2K3. I notice alot of people using P4's in here so it stands to reason that most feel that this is acceptable.

So would it be the same if nVidia optimized software for their architecture, IF everything still looked and played the same? This all assuming that clip panes are of course not used as that would of course be wrong.

On a personal note, I like my 9500np128 mostly for its better image quality, but 8 pipes at 375/642 also runs games pretty good too. ;) Hooray for competition!

I agree with your post. But you said it yourself, it would only be the same if nvidia's "optimizations" didn't change the way things looked, which they do.

ATI's optimizations didn't change the IQ ;)

[StealthHawk]

Originally posted by jAkUp
remember when quake2 had that ingame benchmark? i think it was ~timerefresh... or something like that... where the camera would spin around? theres no way to fix that benchie... because you could always do it in a different spot... is that accurate? why dont the newer games have something like this?

Because it's useless as an indicator of real world gaming performance. Unless all you do is spin around in one spot while playing a game.

[digitalwanderer]

Originally posted by ChrisRay
*forest gump voice*

What do I think of this responce...? It surprises me..


so far thats all I have to say about that...
Thanks Chris, that's the closest I've came to a smile over this all night. :mad:

Originally posted by bkswaney
No problem. :)
I just like getting all the facts straight and not hear say. :)
So they did cheat. Even tho FM now say they did not.
Boy what a mess. :D
Y'know I never thought I'd hear you say that, perhaps this makes it worth it after all. :wry grin peeking thru the clenched and grinding teeth:

Originally posted by ChrisRay
There is something ironic about this to me. a discussion me and digital had on another thread....
....Maybe I was right?
There goes that tiny bit of a smile trying to peek thru. :rolleyes:

Originally posted by Jovi
This you over on rage? Cracked me up. I think both sides of the issue need to lay low and kill the smack talk.
I've never really been the "lay low" kind of guy, much more the "stomp about and shout" kind. :mad:

Originally posted by Jovi
And what purpose would that serve if it is relevent to the conversation at hand? I have read at least a dozen sites listed here and neary a mention to them. :confused:
Yeah, but you're posting ME from over there. If I wanted to say that here I'd say that here, but I know it's inapropriate for this forum and the mods would get miffed at me since I should know better so I post it over there where such behavoir is much more expected/allowed from me. Every site has got it's own set of norms/different flavor and what I post over there ain't always appropriate over here.

If you really need to quote my inflamatory stuff from over there you could probably get away with linking it, but I'd appreciate it if you wouldn't directly quote my rants from over there as I am trying REALLY HARD not to rant here about this. :nono:

[Lfctony]

Didn't cheat!:rofl Well in my book they did!

[StealthHawk]

Originally posted by Jovi
And what purpose would that serve if it is relevent to the conversation at hand? I have read at least a dozen sites listed here and neary a mention to them. :confused:

It is not relevant what someone at Rage3d says at all, if they do not say the same thing or say it in the same way here at nV News.

What are you trying to do, character assassinate DigitalWanderer? I don't care, nor should anyone else, how he handles himself on other sites, as long as he keeps himself under control here.

[ChrisRay]

There goes that tiny bit of a smile trying to peek thru.

I was obviously being sarcastic. It's just an ironic sarcastic joke about our previous conversation :(

[digitalwanderer]

Originally posted by StealthHawk
It is not relevant what someone at Rage3d says at all, if they do not say the same thing or say it in the same way here at nV News.

What are you trying to do, character assassinate DigitalWanderer? I don't care, nor should anyone else, how he handles himself on other sites, as long as he keeps himself under control here.
JINX!!!!! :p

(Ok, for some reason THAT got a grin out of me! :) )

[digitalwanderer]

Originally posted by ChrisRay
I was obviously being sarcastic
So was I Chris, sorry if I implied otherwise. :)

[CapsLock]

Muzz- I'm hoping the new kt600 chipset pans out, its in production now, preliminary tests show it to be faster than nf2, lets hope they have the compatibility/stability thing down and then:

"POOF- all is now AOK" (I love that, talk about irredeemable)

Actually as I seriously disprove of Nvidia's tactic lately, this is good. I thought they were going to get away with playing it quiet. Now, THIS is going to BLOW UP REAL GOOD.


TTTOOOOOTTTTTAAAALLLLLLLYYYYYYY UNBELIEVABLE.

Wonder how the rest of the computer industry will react to this?
This effects ALL benchmarking on ALL Hardware!

Incredible.

CapsLock

[Ady]

Originally posted by jAkUp
i dont think there is really any good way to test the different fps in games anymore. 3dmark01 used to be good... (besides the fact it was WAY WAY to oversensitive to fsb overclocks... but at default fsb its pretty good.) 3dmark03 is trash imo...

I don't entirely agree. Do we really know if everyone was cheating in 3dmark01 or not? Sure they didn't get caught, but that doesn't mean anything. For memory there was a driver release or two that gave some huge performance increases in that benchmark. We don't really know they weren't cheats do we?

Obviously as 3dmark01 got older it turned into more of a system benchmark rather than a video card bechmark which is kind of a handy thing to have I guess. Just not any good at all for comparing video cards.

3dmark03 wasn't that bad, I think it seemed a bit like 01 when that first came out. Pretty intensive for a video card. There was only really one card at the time that could run the whole thing. The only reasons that I could see why so many people were dissapointed in 3dmark03 was because it chugged and they got really low scores when they were used to getting high scores, and of course the public trashing of the benchmark by nvidia didn't help it. Due to this recent news, I think it's totally worthless. I also think all benchmarks will be. The popular ones more likely.

Originally posted by jAkUp
remember when quake2 had that ingame benchmark? i think it was ~timerefresh... or something like that... where the camera would spin around? theres no way to fix that benchie... because you could always do it in a different spot... is that accurate? why dont the newer games have something like this?

yes, I remember that. It's in half-life also.(same engine i think) It's far too inaccurate though. It depends on where you stand and what you are looking at.

[ChrisRay]

Originally posted by digitalwanderer
So was I Chris, sorry if I implied otherwise. :)


gotcha. Heh It's really hard for me to form an opinion on this issue. I'm kinda torn between my hatred for 3dmark and the kinda utter ludicrousy of the whole situation :p

[ChrisW]

Well, now ATI with their 1.9% increase will officially be labeled 'cheaters' while nVidia with their 24% increase and special clipping planes and scene detection that would not clear the frame buffer and changing pixel shader 2.0 into some wierd combination of FP16 and int12 is officially 'optimizing'. Seems ATI gets screwed no matter what they do. The worst part of this is almost nobody outside of these fansites will ever know any of this happened. I guess all the review sites will now use 3DMark03 in their reviews now that nVidia is happy. You guys can start praising 3DMark03 as a benchmark now since all you do it repeat whatever stance nVidia has at the moment.

[Neural]

Originally posted by reever2
But in the realm of processors we KNOW what is being optimized and what is not, we also know that both the processors support and can be optimized for SSE, in benchmarks if one board can be optimized one way, and the other one cant its not a fair benchmark, in games its a different story

So if nVidia published their optimizations that would clear alot of this up wouldnt it, hmmm i hadnt thought of that, I think its something they should think about.

Originally posted by Ady
Neural: Did you really mean to say "how nVidia could hide the clip panes in their drivers?" or how did they put them there? Obviously hiding anything in drivers isn't really an issue. it's not like everyone searches through every file to discover such things. You wouldn't be able to find anything unless you had the source code and you seriously knew what you were looking for anyways.

I don't really know anything of Intel optimizing or whatever. I think it would be obvious though if intel decided to optimize just for a particular synthetic benchmark then that would be completely wrong. Especially if it gave consumers a wrong idea of how a particular CPU performed. Obviously optimizing for an actual program is great. If it runs it runs and if it runs even faster then all the better.

It's different when you are talking graphics though. It's the same if you're talking about a synthetic benchmark, it shouldn't be specifically optimized for to improve it's performance at all. When it comes to games then us consumers deserve to see the game exact as it's developer intended it to be shown. It's not right to be cheated of that right. Cheating in a game benchmark is the worst of them all. Giving us consumers the wrong impression of how a video card will perform in that actual game. I know I sure don't want to be fooled.

It actually swings both ways, Intel makes sure most of the benchmarks are optimized, not because they care what we the consumer do with the benchmarks, but because many if not all of the manufacturers/distributors use these to determine what they will buy/sell. Only SSE intsructions are only implemented in software we use sometimes months even years after they are used in benchmarks, because mainstream software is written for now so that it becomes obsolete sooner, whereas benchmarks are more forward looking. Part of me would like to see nVidia take the Intel approach if their achitecture is so much different that it needs special instructions, be up front, and allow other companys to liscense it. The other part of me is afraid of this because of how Intel uses their instructions, and the other companys inability to keep up with the ever changing SSE support, to bully the market. If ATi ends up in AMD's position , we could all be in trouble.

[StealthHawk]

Originally posted by ChrisRay
gotcha. Heh It's really hard for me to form an opinion on this issue. I'm kinda torn between my hatred for 3dmark and the kinda utter ludicrousy of the whole situation :p

Futuremark caving in for nvidia should only increase your hate for them.

I know that I've lost a lot of respect for Futuremark, and if nvidia is able to use their cheating drivers to get higher scores then 3dmark03 is now officially 100% useless(if it wasn't already) beyond the individual feature tests like fillrate, etc. Futuremark was forced to shoot themselves in the foot, sad really.

[Zenikase]

Being a longtime nVidia fanboy (and I probably continue to be so for a long time), this is pretty crappy.

I don't really care much for the politics of the whole situation. For me, what matters in the end is getting the best performance possible out of my product without making compromises in image quality and without altering the original intended result. But things like useless benchmark cheats (that's what they are, apparently) only serve to add driver code bloat and the end user loses, since you'll only get that type of speedy performance on that specific program. Personally, I think the rule of thumb for video card companies should be to create drivers that are optimized for ALL GENERAL 3D APPLICATIONS. In my opinion, this is the only way one can produce comparable results for different products, based on hardware and driver efficiency.

To touch on another subject, all this talk about NV3x's poor fragment shader performance is really unjustified. A quick glance at Carmack's last .plan update shows a comparison of ATi and nVidia's top dogs:

The R200 path has a slight speed advantage over the ARB2 [DX9-optimized] path on the R300, but only by a small margin, so it defaults to using the ARB2 path for the quality improvements. The NV30 runs the ARB2 path MUCH slower than the NV30 [faster than NV20, but not DX9-optimized] path. Half the speed at the moment. This is unfortunate, because when you do an exact, apples-to-apples comparison using exactly the same API, the R300 looks twice as fast, but when you use the vendor-specific paths, the NV30 wins.

The reason for this is that ATI does everything at high precision all the time, while Nvidia internally supports three different precisions with different performances. To make it even more complicated, the exact precision that ATI uses is in between the floating point precisions offered by Nvidia, so when Nvidia runs fragment programs, they are at a higher precision than ATI's, which is some justification for the slower speed. Nvidia assures me that there is a lot of room for improving the fragment program performance with improved driver compiler technology.


This is the kind of stuff that comes with new technology, and you can't really compare the two side-by-side until some type of standard is set (which probably won't come for a while).

[muzz]

Yep.

Only problem is the "STANDARD" won't be good enough or something else foolish like that..... and all benchies will be won by the almighty $

[jbirney]

This is a really sad day.

It goes to show you that you can lower iq to cheat in benchmarks (notice they did not say anything about clip planes or zbuffer errors) and get away for it so long as your the biggest company. what I find even more sad is that a future version of 3dmarks may allow for 12/16 bit precision when the DX9 spec is 24 bit. It went from bad to worse now to just down right pittfiull.