Go Back   nV News Forums > Hardware Forums > Benchmarking And Overclocking

Newegg Daily Deals

Closed Thread
 
Thread Tools
Old 05-24-03, 06:41 AM   #397
StealthHawk
Guest
 
Posts: n/a
Default

Quote:
Originally posted by Adagio
Nvidia yesterday responded to claims of cheating in 3dmark2003 saying this:

A representative at Nvidia questioned the validity of Futuremark's conclusions. "Since Nvidia is not part of the Futuremark beta program (a program which costs of hundreds of thousands of dollars to participate in), we do not get a chance to work with Futuremark on writing the shaders like we would with a real applications developer," the representative said. "We don't know what they did, but it looks like they have intentionally tried to create a scenario that makes our products look bad."

But how much does it cost to enter the Beta plan, I mean what are the actual minimum costs involved per year? is it $300,000 a year? $200,000 a year? maybe even $100,000 a year?

I'll tell you - it's $5000.

http://discuss.futuremark.com/forum/...=1#Post2288998

"hundreds of thousands of dollars" -> $5000? nVidia's staff isn't even able to count

What's $5000 to a company like nVidia? Nothing ... all they just say now is pure nonesense and blabla ...
The minimum is $5000, that's not the maximum. nvidia was a strategic beta member. That was the highest possible member.
 
Old 05-24-03, 06:50 AM   #398
Behemoth
radeon 9800 pro
 
Behemoth's Avatar
 
Join Date: Apr 2003
Location: Darkness Falls
Posts: 841
Default

Quote:
Originally posted by StealthHawk
You're making an assumption as well as accusing ATI of lying. Notice that nvidia never officially said they weren't cheating. They just went off on a tirade about how Futuremark is evil and is trying to make them look bad

In a game, what ATI did was clearly optimizing. However, in a benchmark, some people consider it cheating if the driver touches anything.
Discussion
Discussion
There are other examples where people consider reordering the instructions cheating, look in other threads.

ATI has come clean with what's going on, we have to assume they are telling the truth. As you can see, people consider this cheating, so to appease them, they are removing the optimizations.



I can't tell if you're being sarcastic here

I'll assume you aren't.

a) There are no clipping planes in ATI drivers. This has been verified by both B3D and Futuremark.
b) There is no drop in IQ on ATI cards. What would they even fallback to? They support one FP precisions, unlike nvidia.
Visual Poof
i didnt say ati was lying, and yes ati didnt say they were cheating.
but if those purposely altered sky and water shaders are valid PS2 shaders i dont see why they remove them in next driver, the only reason why they remove them in next driver i can think of is they are not valid PS2 shaders.
Behemoth is offline  
Old 05-24-03, 06:55 AM   #399
Behemoth
radeon 9800 pro
 
Behemoth's Avatar
 
Join Date: Apr 2003
Location: Darkness Falls
Posts: 841
Default

Quote:
I can't tell if you're being sarcastic here

I'll assume you aren't.

a) There are no clipping planes in ATI drivers. This has been verified by both B3D and Futuremark.
b) There is no drop in IQ on ATI cards. What would they even fallback to? They support one FP precisions, unlike nvidia.
Visual Poof
sorry, i was being sarcastic, by "on the rail", i meant the sky and water shaders only work in GT4 situration, they dont work as fast in normal unaltered fashion as shown in 3DMark GT4.
Behemoth is offline  
Old 05-24-03, 07:05 AM   #400
Onde Pik
Thrakhath nar Kiranka
 
Onde Pik's Avatar
 
Join Date: Aug 2002
Location: Kilrah
Posts: 92
Default

Quote:
Originally posted by Behemoth
ati altered how sky and water shaders worked in GT4, if it worked in all siturations, why dont ati implement this technique into next driver instead of getting rid of it? the only reason i can think of is that, its siturational, it also only works "on the rail"
They changed the excution order of instructions, as I pointed out with an example this can be done on ANY architechture to improve performance without changing the end result 1 iota. And what are you talking about with the next driver??? How do you know they wont? Got a link to verify this?

If you dont grasp what is going on, then plz stop replying.
__________________
Ek’rah skabak erg Thrak’Kilrah maks Ragnith
Onde Pik is offline  
Old 05-24-03, 07:06 AM   #401
MrWho
Registered User
 
MrWho's Avatar
 
Join Date: Jan 2003
Location: Coimbra, Portugal
Posts: 39
Send a message via Yahoo to MrWho
Default

It's almost that time that comes every year when I must start thinking on upgrading my graphics card. I've been an nVidia customer for some time (TNT2, Geforce, and now my GF3 Ti500), but now the choice's much harder.

For some time now (and not because of this, though it helped) I've been considering changing over to ATI. I'm still afraid of their drivers (Although I've read they got much better than what they used to be), but the FX5800 doesn't inspire much confidence to me either (the noise, the screensaver bug, the benchmarks when compared to ATI's cards, and now this).

okay, you guys tell me ATI cheated too. What's that supposed to mean? Should I think "well, if ATI cheated too, then it's okay!"? The fact that ATI "only" cheated for little more than 1% makes me think if it wasn't just a fluke, a piece of bad programming in their drivers. Being true, that's bad because it shows that they're not capable yet of doing top-notch drivers, but it's good because it may prove that they didn't intend to do it. Now I know that nVidia has some good programmers writting their drivers (nVidia is to me the best driver supplier in the whole hardware department, not just gfx cards), which implies that whatever "cheats" are found inside their drivers (and, even worse, 29% cheats) can't be there by accident.

I change my mind easily when I feel I'm being played for a sucker. I was a 3Dfx fan until the advent of the Voodoo3 series - at that time I changed over to the TnT2. Now maybe that time is comming again, I'm still not sure. But one thing I'm sure: I'm an ex-nVidia fannatic - now I just plain like them, but I won't jump in their defence so easily.
__________________
MrWho of Schizofrenia
MrWho is offline  
Old 05-24-03, 07:21 AM   #402
Behemoth
radeon 9800 pro
 
Behemoth's Avatar
 
Join Date: Apr 2003
Location: Darkness Falls
Posts: 841
Default

Quote:
Originally posted by Onde Pik
They changed the excution order of instructions, as I pointed out with an example this can be done on ANY architechture to improve performance without changing the end result 1 iota. And what are you talking about with the next driver??? How do you know they wont? Got a link to verify this?

If you dont grasp what is going on, then plz stop replying.
instead of letting the PS2 shader itself doing the flow of control, ati manually tells GT4 sky and water shaders how to shuffle instructions, hows that different to inserting clip plane to tell the video card ignore some of the workload, in this case ati is telling the video card to ignore some of the decision makings to get higher score, because the shaders were not doing full PS2 work.
the difference is nvidia fails to maintain visual output while ati succeeds to maintain it. but futuremark was not trying to measure how good PS2 could be in that particular case, the siturational modification was not what futuremark was trying to measure. hence ati is gonna to remove it in next driver, ati *helps* the score by hinting the shaders how to shuffle instructions.

Quote:
"The 1.9% performance gain comes from optimization of the two DX9 shaders (water and sky) in Game Test 4 . We render the scene exactly as intended by Futuremark, in full-precision floating point. Our shaders are mathematically and functionally identical to Futuremark's and there are no visual artifacts; we simply shuffle instructions to take advantage of our architecture. These are exactly the sort of optimizations that work in games to improve frame rates without reducing image quality and as such, are a realistic approach to a benchmark intended to measure in-game performance. However, we recognize that these can be used by some people to call into question the legitimacy of benchmark results, and so we are removing them from our driver as soon as is physically possible. We expect them to be gone by the next release of CATALYST."

Last edited by Behemoth; 05-24-03 at 07:26 AM.
Behemoth is offline  
Old 05-24-03, 07:50 AM   #403
Hanners
Elite Bastard
 
Hanners's Avatar
 
Join Date: Jan 2003
Posts: 984
Default

Quote:
Originally posted by Behemoth
instead of letting the PS2 shader itself doing the flow of control, ati manually tells GT4 sky and water shaders how to shuffle instructions, hows that different to inserting clip plane to tell the video card ignore some of the workload, in this case ati is telling the video card to ignore some of the decision makings to get higher score, because the shaders were not doing full PS2 work.
the difference is nvidia fails to maintain visual output while ati succeeds to maintain it. but futuremark was not trying to measure how good PS2 could be in that particular case, the siturational modification was not what futuremark was trying to measure. hence ati is gonna to remove it in next driver, ati *helps* the score by hinting the shaders how to shuffle instructions.
I would say Pixel Shader 2.0 performance makes up a large proportion of how GT 4 performs.

Going by what Tim Sweeney of Epic has to say about things, what ATi did was a valid optimisation, but nontheless seeing as it is only implemented in the specific case of 3DMark 2003 GT4 it should be removed.

The whole point of a benchmark is that it runs 100% how the developer intends it, so even shuffling instructions is a no-go in my book.
__________________
Owner / Editor-in-Chief - Elite Bastards
Hanners is offline  
Old 05-24-03, 07:55 AM   #404
Behemoth
radeon 9800 pro
 
Behemoth's Avatar
 
Join Date: Apr 2003
Location: Darkness Falls
Posts: 841
Default

Quote:
Originally posted by Hanners
I would say Pixel Shader 2.0 performance makes up a large proportion of how GT 4 performs.

Going by what Tim Sweeney of Epic has to say about things, what ATi did was a valid optimisation, but nontheless seeing as it is only implemented in the specific case of 3DMark 2003 GT4 it should be removed.

The whole point of a benchmark is that it runs 100% how the developer intends it, so even shuffling instructions is a no-go in my book.
wow cant believe you agree with me on this one
Behemoth is offline  

Old 05-24-03, 08:00 AM   #405
Morrow
Atari STE 4-bit color
 
Morrow's Avatar
 
Join Date: May 2003
Posts: 798
Default

I'm wondering if FutureMark would not have written something about the 8% drop in performance in GT4 using ATI cards, had ATI ever officially admitted that they also replaced shaders programs in 3dmark03? Probably not.

Shows how trustworthy any company nowadays is. Why did ATI give their official statement on 3dmark03 just one day after nvidia has been proved cheating, why not weeks before?

Why are nvidia's shader optimizations called cheats and ATI shader optimzations legitimate, when FutureMark wants to test standard performance with standard DX calls and not specifically optimized routines for any hardware vendor?

Why is ATI removing those optimizations in the next Catalyst release even if they think that those optimizations are legitimate and do not question the fairness of this "synthetic benchmarking"?

This all shows again how obsolete benchmarks à la 3dmark03 have become even if FutureMark is investigating strange driver behaviours and making changes to the 3dmark03 code to prevent cheating. Doesn't this show how easily any company can optimize their drivers to fake their results in any way they want.

Going back to use my videocard for what I have bought it in the first place.... playing games
Morrow is offline  
Old 05-24-03, 08:06 AM   #406
Hanners
Elite Bastard
 
Hanners's Avatar
 
Join Date: Jan 2003
Posts: 984
Default

Quote:
Originally posted by Morrow
I'm wondering if FutureMark would not have written something about the 8% drop in performance in GT4 using ATI cards, had ATI ever officially admitted that they also replaced shaders programs in 3dmark03? Probably not.
I believe they would, yes.

Quote:
Originally posted by Morrow
Shows how trustworthy any company nowadays is. Why did ATI give their official statement on 3dmark03 just one day after nvidia has been proved cheating, why not weeks before?
I get the impression ATi hadn't even considered switching the order of a few shader instructrions around as a cheat until this whole situation came to light.

Quote:
Originally posted by Morrow
Why are nvidia's shader optimizations called cheats and ATI shader optimzations legitimate, when FutureMark wants to test standard performance with standard DX calls and not specifically optimized routines for any hardware vendor?
Simple - nVidia are replacing FutureMark's shaders wholesale with their own, and suffering reduced image quality as a result. ATi are using FutureMark's shaders, but switching the order of some of the instructions so it runs faster on their architecture.

Quote:
Originally posted by Morrow
Why is ATI removing those optimizations in the next Catalyst release even if they think that those optimizations are legitimate and do not question the fairness of this "synthetic benchmarking"?
Because at the end of the day it isn't about what ATi think, it's about what FutureMark and the consumers think. In general people feel that it is a cheat, therefore ATi will remove them from their drivers.
__________________
Owner / Editor-in-Chief - Elite Bastards
Hanners is offline  
Old 05-24-03, 08:27 AM   #407
Madstrom
Registered User
 
Madstrom's Avatar
 
Join Date: Jan 2003
Location: Athens, GA
Posts: 8
Send a message via ICQ to Madstrom Send a message via AIM to Madstrom Send a message via Yahoo to Madstrom
Default

What I find disturbing is the difference in responses from both companies. ATI answers about their "optimizations" and plans to remove them, nVidia responds "WAAAAAAAAA!WAAAAAAAAA! They want to make us look bad.WAAAAAAAAA!"

This makes me wonder about the other benchmarks. We know that they can be "optimized." Just like the ATI Quake III cheat!

That cheat kept me away from ATI for a long while. Heck I didn't even try ATI until I found an 8500 for $106.00.

Now I feel the same about nVidia

Luckily all my previous nVidia cards were the good, solid ones (TNT, TNT2, GeForce, GeForce 2 GTS, GeForce 3 Ti 200, GeForce 4 Ti 4400) that kicked ass without cheats.
__________________
BPG_folding (4 of 4)
XP 2400+/ XP 2100+
1GB 3200/ 256MB 2100
A7N8X/ K7S5A
80GB Maxtor, 2x120MB WD/ 60GB WD
ATI 9700Pro 128MB/ ATI 8500 128MB
nVidia Soundstorm/ SIS Onboard
Antec 430 True Power/ Antec 350 Smart Power

Last edited by Madstrom; 05-24-03 at 08:31 AM.
Madstrom is offline  
Old 05-24-03, 08:55 AM   #408
chrismm28
Registered User
 
Join Date: May 2003
Posts: 53
Default

Quote:
Originally posted by Behemoth
instead of letting the PS2 shader itself doing the flow of control, ati manually tells GT4 sky and water shaders how to shuffle instructions, hows that different to inserting clip plane to tell the video card ignore some of the workload, in this case ati is telling the video card to ignore some of the decision makings to get higher score, because the shaders were not doing full PS2 work.
the difference is nvidia fails to maintain visual output while ati succeeds to maintain it. but futuremark was not trying to measure how good PS2 could be in that particular case, the siturational modification was not what futuremark was trying to measure. hence ati is gonna to remove it in next driver, ati *helps* the score by hinting the shaders how to shuffle instructions.
No the ATI drivers were rendering the scene exactly as the 3dmark code wanted, just in different order. This can be done in any situation, including real games. Its what optimisations are, from any company. Nvidias drivers actually replaced the 3dmark code completely to create a similar looking scene that was easier for the nv3X to render, but it was not identical.
If you had an exam to do you might not do all the questions in order, but come back to some later.. this is ATI.
If you had an exam but found the questions to hard, you would change them so that the answers were similar, but the question itself was easier... this is nvidia.
chrismm28 is offline  
Closed Thread


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


Similar Threads
Thread Thread Starter Forum Replies Last Post
Need Help Installing NVIDIA Tesla M2070Q in Linux RHEL5 Ferianto85 NVIDIA Linux 0 05-18-12 09:35 PM
Current NVIDIA FreeBSD graphics driver releases zander NVIDIA FreeBSD 0 01-27-09 06:22 PM
Current NVIDIA Linux graphics driver releases AaronP NVIDIA Linux 0 11-06-08 05:39 PM

All times are GMT -5. The time now is 07:02 AM.


Powered by vBulletin® Version 3.7.1
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Copyright ©1998 - 2014, nV News.