Go Back   nV News Forums > Hardware Forums > Benchmarking And Overclocking

Newegg Daily Deals

Reply
 
Thread Tools
Old 11-13-03, 10:37 AM   #1
The Baron
Guest
 
Posts: n/a
Default New [H] Editorial on Benchmarking

Yes, here we go again, ladies.

Reading it now. Should be fun.
  Reply With Quote
Old 11-13-03, 10:40 AM   #2
John Reynolds
Registered User
 
Join Date: Aug 2002
Posts: 365
Default

Heh, just e-mailed Kyle after reading his editorial. I didn't flame him, but I did point out the amount of PR crap that was being blown up his backside.
John Reynolds is offline   Reply With Quote
Old 11-13-03, 10:46 AM   #3
The Baron
Guest
 
Posts: n/a
Default

okay. Kyle is insane. I'll leave it at that. he's just nuts.
  Reply With Quote
Old 11-13-03, 11:08 AM   #4
TheTaz
Registered User
 
Join Date: Jul 2002
Posts: 621
Default

Actually, I kind of agree with him.

Not all the crap he spewed about nVidia, but some of what he spewed about futuremark.

While I applaud and defend Futuremark for enforcing their rules, and also while I've pointed out that 3DMark is not intended to reflect "real game performance"...

I still think it's a BAD tool.

ANY company that is going to be "responsible" for essentially "policing an industry, via measuring performance", and then takes money from the IHV's and OEM's... well... to me that's a conflict of interest. It open's a whole "can of worms". Who's to say one side isn't "paying more money" than the other side, which is swaying the results? Heck, there's already someone on B3D forums telling Dave that "Dell asked for this patch because they're selling ATi cards, now."

While Futuremark takes money from IHV's and OEM's... I can't really see myself puting any real faith in their product, in general.

Taz
TheTaz is offline   Reply With Quote
Old 11-13-03, 11:10 AM   #5
The Baron
Guest
 
Posts: n/a
Default

3DMark03's relevance is dead now. It was not dead when it came out. We have DX9 games now. We didn't have DX9 games then. So... I dunno what I'm trying to say. But it's probably something.
  Reply With Quote
Old 11-13-03, 11:24 AM   #6
Sazar
Sayonara !!!
 
Sazar's Avatar
 
Join Date: Jan 2003
Location: Austin, Texas
Posts: 9,297
Default

Quote:
Originally posted by John Reynolds
Heh, just e-mailed Kyle after reading his editorial. I didn't flame him, but I did point out the amount of PR crap that was being blown up his backside.


that reminded me of this one guy who went to a gas station and took an air pump and stuck it up his backside.. and exploded lol...

Sazar is offline   Reply With Quote
Old 11-13-03, 11:33 AM   #7
John Reynolds
Registered User
 
Join Date: Aug 2002
Posts: 365
Default

Quote:
Originally posted by The Baron
3DMark03's relevance is dead now. It was not dead when it came out. We have DX9 games now. We didn't have DX9 games then. So... I dunno what I'm trying to say. But it's probably something.
Are you sure about that, because how many game developers continue to police their code in ongoing efforts to circumvent cheating after their game has been published?
John Reynolds is offline   Reply With Quote
Old 11-13-03, 11:43 AM   #8
ChrisW
"I was wrong", said Chris
 
Join Date: Jul 2002
Location: standing in the corner!
Posts: 620
Default

I think this quote should explain everything:
Quote:
This latest patch from Futuremark is yet another revision of 3DMark03 specifically designed to defeat our Unified Compilier Technology, which evaluates shaders and in some cases substitutes hand tuned shaders, but increasingly simply applies the run-time compiler to generate optimal code. With the 52.16 drivers and the new patch, our perf drops 15%.

Clearly our compiler has gotten much better, as image quality remains exactly the same, the only thing that happens is a 10-15% drop in performance.
Notice they used "yet another revision...designed to defeat our Unified Compiler Technology"! The only way for them to be doing this again is for nVidia to just rename all their previous "optimizations" (cheats) to a "Unified Compiler". I think this is official confirmation that is what they are doing.

"Substitute hand tuned shaders"? That is exactly what they were doing in the past. People were lead to believe this new "compiler" just re-odered shader instructions to make them much more efficient, not simply detecting shaders and replacing them with new shaders designed to simply achieve a higher score. NVidia should be ashamed of themselves for purposely misleading the public (again). They are smart enough to know that "Unified Compiler Technology" sounds much better than "cheater".
__________________
AIW 9700 Pro | 1.3GHz CeleronT | 512MB PC133 SDRAM | ECS PCIPAT Mobo (Intel 815EP)
RadLinker/RadClocker

Last edited by ChrisW; 11-13-03 at 11:51 AM.
ChrisW is offline   Reply With Quote

Old 11-13-03, 11:49 AM   #9
Uttar
Registered User
 
Uttar's Avatar
 
Join Date: Aug 2002
Posts: 1,354
Send a message via AIM to Uttar Send a message via Yahoo to Uttar
Default

Quote:
Originally posted by ChrisW
I think this quote should explain everything:

Notice they used "yet another revision...designed to defeat our Unified Compiler Technology"! The only way for them to be doing this again is for nVidia to just rename all their previous "optimizations" (cheats) to a "Unified Compiler". I think this is official confirmation that is what they are doing.


AAARGH!
The compiler is for real guys. Stop inveting BS - everyone is, and it's just annoying me.

The effects of the compiler cannot be as good as hand-tuned code most of the time, but it's pretty darn good. And developers will have making better than what the compiler does.

Disabling the compiler seems like BS though. It's still on, or their scores would be even significantly lower.


Uttar
Uttar is offline   Reply With Quote
Old 11-13-03, 11:53 AM   #10
ChrisW
"I was wrong", said Chris
 
Join Date: Jul 2002
Location: standing in the corner!
Posts: 620
Default

Quote:
Originally posted by Uttar
AAARGH!
The compiler is for real guys. Stop inveting BS - everyone is, and it's just annoying me.
What are you talking about? That quote came directly from the link. If anyone is making stuff up, it is either nVidia or [H]. It says in the actual quote that they are detecting shaders and replacing them with hand written shaders. I'm not making up anything. That is not my definition of a compiler.
__________________
AIW 9700 Pro | 1.3GHz CeleronT | 512MB PC133 SDRAM | ECS PCIPAT Mobo (Intel 815EP)
RadLinker/RadClocker
ChrisW is offline   Reply With Quote
Old 11-13-03, 12:03 PM   #11
ChrisW
"I was wrong", said Chris
 
Join Date: Jul 2002
Location: standing in the corner!
Posts: 620
Default

Quote:
What we expect will happen is that we'll be forced to expend more engineering effort to update our compiler's fingerprinter to be more intelligent, specifically to make it intelligent in its ability to optimize code even when application developers are trying to specifically defeat compilation and optimal code generation.
I don't know about you, but telling us they are going to make it harder for developers to detect when nVidia's drivers are cheating adds a bad taste to my mouth. I don't see how anyone can defend these tactics. And "fingerprinter"...if that is not designed to specifically detect things in the benchmark I don't know what it is. Basically, they are saying they should have made it harder for them to find. I wonder how much other stuff got through they could not find?
__________________
AIW 9700 Pro | 1.3GHz CeleronT | 512MB PC133 SDRAM | ECS PCIPAT Mobo (Intel 815EP)
RadLinker/RadClocker

Last edited by ChrisW; 11-13-03 at 12:11 PM.
ChrisW is offline   Reply With Quote
Old 11-13-03, 12:09 PM   #12
DMA
Zzzleepy
 
DMA's Avatar
 
Join Date: Apr 2003
Location: In bed
Posts: 997
Default

Quote:
Originally posted by ChrisW
I don't know about you, but telling us they are going to make it harder for developers to detect when nVidia's drivers are cheating adds a bad taste to my mouth. I don't see how anyone can defend these tactics. And "fingerprinter"...if that is not designed to specifically detect things in the benchmark I don't know what it is.
Well, their compiler is still running probably. I doubt that FM can disable that. So don't take NV's PR machine to seriously.

ATI's compiler stuff added in one of their driver sets (3.5/3.6?) is probably still alive and kicking too
(Dunno if that's the same thing so don't kill me please )
__________________
Gigabyte EP35-DS4 | E6850@3.8GHz |
8800GT@800/2000MHz | 4 GB Corsair 6400|
DMA is offline   Reply With Quote
Reply


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


Similar Threads
Thread Thread Starter Forum Replies Last Post
Benchmarking Analytical Queries on a GPU News Archived News Items 0 05-20-12 08:00 AM
NVIDIA GeForce GTX 670 Video Card Review @ [H] News GeForce GTX 670 Reviews 0 05-10-12 11:11 AM
unigine Benchmarking with GTX285 and 302.07 on KDE4. This is normal? sl1pkn07 NVIDIA Linux 3 05-10-12 07:11 AM
Benchmarking AMD's 768-Shader Pitcairn: Not For Public Consumption News Archived News Items 0 05-08-12 02:30 AM
Hardball Editorial legion88 Feedback Forum 1 09-02-02 06:45 PM

All times are GMT -5. The time now is 09:53 PM.


Powered by vBulletin® Version 3.7.1
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Copyright 1998 - 2014, nV News.