nV News Forums

 
 

nV News Forums (http://www.nvnews.net/vbulletin/index.php)
-   Benchmarking And Overclocking (http://www.nvnews.net/vbulletin/forumdisplay.php?f=24)
-   -   New [H] Editorial on Benchmarking (http://www.nvnews.net/vbulletin/showthread.php?t=20935)

The Baron 11-13-03 10:37 AM

New [H] Editorial on Benchmarking
 
Yes, here we go again, ladies.

Reading it now. Should be fun.

John Reynolds 11-13-03 10:40 AM

Heh, just e-mailed Kyle after reading his editorial. I didn't flame him, but I did point out the amount of PR crap that was being blown up his backside.

The Baron 11-13-03 10:46 AM

okay. Kyle is insane. I'll leave it at that. he's just nuts.

TheTaz 11-13-03 11:08 AM

Actually, I kind of agree with him.

Not all the crap he spewed about nVidia, but some of what he spewed about futuremark.

While I applaud and defend Futuremark for enforcing their rules, and also while I've pointed out that 3DMark is not intended to reflect "real game performance"...

I still think it's a BAD tool.

ANY company that is going to be "responsible" for essentially "policing an industry, via measuring performance", and then takes money from the IHV's and OEM's... well... to me that's a conflict of interest. It open's a whole "can of worms". Who's to say one side isn't "paying more money" than the other side, which is swaying the results? Heck, there's already someone on B3D forums telling Dave that "Dell asked for this patch because they're selling ATi cards, now."

While Futuremark takes money from IHV's and OEM's... I can't really see myself puting any real faith in their product, in general.

Taz

The Baron 11-13-03 11:10 AM

3DMark03's relevance is dead now. It was not dead when it came out. We have DX9 games now. We didn't have DX9 games then. So... I dunno what I'm trying to say. But it's probably something.

Sazar 11-13-03 11:24 AM

Quote:

Originally posted by John Reynolds
Heh, just e-mailed Kyle after reading his editorial. I didn't flame him, but I did point out the amount of PR crap that was being blown up his backside.
:rofl

that reminded me of this one guy who went to a gas station and took an air pump and stuck it up his backside.. and exploded lol...

:rofl

John Reynolds 11-13-03 11:33 AM

Quote:

Originally posted by The Baron
3DMark03's relevance is dead now. It was not dead when it came out. We have DX9 games now. We didn't have DX9 games then. So... I dunno what I'm trying to say. But it's probably something.
Are you sure about that, because how many game developers continue to police their code in ongoing efforts to circumvent cheating after their game has been published?

ChrisW 11-13-03 11:43 AM

I think this quote should explain everything:
Quote:

This latest patch from Futuremark is yet another revision of 3DMark03 specifically designed to defeat our Unified Compilier Technology, which evaluates shaders and in some cases substitutes hand tuned shaders, but increasingly simply applies the run-time compiler to generate optimal code. With the 52.16 drivers and the new patch, our perf drops 15%.

Clearly our compiler has gotten much better, as image quality remains exactly the same, the only thing that happens is a 10-15% drop in performance.
Notice they used "yet another revision...designed to defeat our Unified Compiler Technology"! The only way for them to be doing this again is for nVidia to just rename all their previous "optimizations" (cheats) to a "Unified Compiler". I think this is official confirmation that is what they are doing. :(

"Substitute hand tuned shaders"? That is exactly what they were doing in the past. People were lead to believe this new "compiler" just re-odered shader instructions to make them much more efficient, not simply detecting shaders and replacing them with new shaders designed to simply achieve a higher score. NVidia should be ashamed of themselves for purposely misleading the public (again). They are smart enough to know that "Unified Compiler Technology" sounds much better than "cheater".

Uttar 11-13-03 11:49 AM

Quote:

Originally posted by ChrisW
I think this quote should explain everything:

Notice they used "yet another revision...designed to defeat our Unified Compiler Technology"! The only way for them to be doing this again is for nVidia to just rename all their previous "optimizations" (cheats) to a "Unified Compiler". I think this is official confirmation that is what they are doing. :(



AAARGH!:rolleyes:
The compiler is for real guys. Stop inveting BS - everyone is, and it's just annoying me.

The effects of the compiler cannot be as good as hand-tuned code most of the time, but it's pretty darn good. And developers will have making better than what the compiler does.

Disabling the compiler seems like BS though. It's still on, or their scores would be even significantly lower.


Uttar

ChrisW 11-13-03 11:53 AM

Quote:

Originally posted by Uttar
AAARGH!:rolleyes:
The compiler is for real guys. Stop inveting BS - everyone is, and it's just annoying me.

What are you talking about? :confused: That quote came directly from the link. If anyone is making stuff up, it is either nVidia or [H]. It says in the actual quote that they are detecting shaders and replacing them with hand written shaders. I'm not making up anything. That is not my definition of a compiler.

ChrisW 11-13-03 12:03 PM

Quote:

What we expect will happen is that we'll be forced to expend more engineering effort to update our compiler's fingerprinter to be more intelligent, specifically to make it intelligent in its ability to optimize code even when application developers are trying to specifically defeat compilation and optimal code generation.
I don't know about you, but telling us they are going to make it harder for developers to detect when nVidia's drivers are cheating adds a bad taste to my mouth. I don't see how anyone can defend these tactics. And "fingerprinter"...if that is not designed to specifically detect things in the benchmark I don't know what it is. Basically, they are saying they should have made it harder for them to find. I wonder how much other stuff got through they could not find?

DMA 11-13-03 12:09 PM

Quote:

Originally posted by ChrisW
I don't know about you, but telling us they are going to make it harder for developers to detect when nVidia's drivers are cheating adds a bad taste to my mouth. I don't see how anyone can defend these tactics. And "fingerprinter"...if that is not designed to specifically detect things in the benchmark I don't know what it is.
Well, their compiler is still running probably. I doubt that FM can disable that. So don't take NV's PR machine to seriously.

ATI's compiler stuff added in one of their driver sets (3.5/3.6?) is probably still alive and kicking too :)
(Dunno if that's the same thing so don't kill me please :D )


All times are GMT -5. The time now is 05:51 AM.

Powered by vBulletin® Version 3.7.1
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Copyright 1998 - 2014, nV News.