My thoughts on "Optimization"

[digitalwanderer]

Originally posted by Nv40
I would like that [NVNEWS] post (jimmyjames123) summarry of the current situation..in the front page of this site ,it is very important that every gamer outhere knows about those facts ,and let them judge if what Nvidia have done is ok for them or not. because i see frequently post here and there of people that doesnt have a clue of how this "cheat" or "optimization" is any diferent to what ATI and NVIDIA does in games to get best performance wihout the decrease in IQ.

In a game it's ok, in a benchmark it's not. I'm pretty sure most people know that one...unless they're fanboys.

[Sazar]

Originally posted by Nv40
exactly.. well said.

I would like that [NVNEWS] post (jimmyjames123) summarry of the current situtation..in the front page of this site ,it is very important that every gamer outhere knows about those facts ,and let them judge if what Nvidia have done is ok for them or not. because i see frequently post here and there off people that doesnt have a clue of how this "cheat" or "optimization" is any diferent to what ATI and NVIDIA does in games to get best performance wihout the decrease in IQ.

why am I not suprised that you agree with these 'facts'



one point... to contradict a FACT...

the doom3 benchmark scenes were changed by ID... the benchmark itself was not changed... ie the engine and everything else remained the same...

ID's reasons for this were that they felt the scenes nvidia had recorded showed their product in a better light than what was expected... again... ID did not change the BENCHMARK... they changed the scenes used... AFAIK...

here btw is futuremark's response to nvidia's accusations...

I think perhaps both you and jimmy should have a read as to the reasonings given...

http://www.futuremark.com/companyinf...discussion.pdf

note : if it were NOT for 3dmark and tests like it... we would not have an idea of HOW teh gf FX nv30's architecture was organised... we would all still be running round saying its an 8x1 setup... instead of what it really is..

[jimmyjames123]

Cutting out the majority of the data to be passed to the card in effect makes the obtained value irrelevant since it merely reflects how well the card can render a portion of the data rather than the entire scene.

There is no indication that the majority of the data is cut out. There is simply no way to say how much of a boost is given.

Apparently you don't understand that the intent of the benchmark is to plug through all of the data and let the card's routines sort out how to handle it and then render it.

Since when is there a handbook on the intent of a benchmark? We already know that both NVIDIA and ATI both optimize for this benchmark, and we know that ATI has access to the developer's version of this benchmark. And obviously Futuremark's intent is not the same as NVIDIA or ATI. NVIDIA (and ATI) wants the benchmark to be as fast and as smooth as possible on their cards without corrupting image quality that we can see.

Thus the scores one receives in a benchmark with the FX5900 is incongruous to the actual performance one will achieve in gameplay.

The scores one receives in a benchmark are never perfectly congruous to actual gaming performance. That was the major beef with 3dmark03 in the first place. The GeForce FX 5900 Ultra has had it's performance well documented by several reviewers, and for the most part extremely well in virtually all benchmarks and games even compared to everything else that is out there.

[ChrisRay]

I must wonder, Has anyone actually looked at what these drivers "have" done. Other than 3dmark? These boards improved AF quality across the board.

Without peformance loss. Was anyone else surprised by this? I didn't think it was possible. Yet obviously it was.

[Clockwork]

Originally posted by Sazar

the doom3 benchmark scenes were changed by ID... the benchmark itself was not changed... ie the engine and everything else remained the same...

ID's reasons for this were that they felt the scenes nvidia had recorded showed their product in a better light than what was expected... again... ID did not change the BENCHMARK... they changed the scenes used... AFAIK...

Indeed. Some people choose to forget that nVidia already had their own demo prepared before id told them they they couldn't use it. One can assume most of the optimizations made for that demo were still completely applicable to the demo id ended up providing....

Why can't people understand that?


Alas, that is not what this thread is for. This is about 3dmark.

[ChrisRay]

Originally posted by Clockwork
Indeed. Some people choose to forget that nVidia already had their own demo prepared before id told them they they couldn't use it. One can assume most of the optimizations made for that demo were still completely applicable to the demo id ended up providing....

Why can't people understand that?


Alas, that is not what this thread is for. This is about 3dmark.

How many threads do we need here for just 3dmark? I think It's ok to break topic here in the interest of the bigger topic. If you wanna talk about 3dmark I can point you to about 3 to 4 other threads on the front page here.

[Clockwork]

Hey man, don't blame me. I didn't start the thread.

[ChrisRay]

Originally posted by Clockwork
Hey man, don't blame me. I didn't start the thread.

Ya I know, But the thread has slowly generated into talking about drivers anyway, Ethical ness of Nvidia, ect I think It's safe to use other topics in the general interest of this one

[jimmyjames123]

note : if it were NOT for 3dmark and tests like it... we would not have an idea of HOW teh gf FX nv30's architecture was organised... we would all still be running round saying its an 8x1 setup... instead of what it really is..

That's just a hardware spec, and most of us already know that actual gaming benchmark's will tell the story behind performance. Also, with the new Detonator FX drivers, the NV30 performs very respectably--much better than we all thought a month or two ago.

[Sazar]

Originally posted by jimmyjames123
There is no indication that the majority of the data is cut out. There is simply no way to say how much of a boost is given.

Since when is there a handbook on the intent of a benchmark? We already know that both NVIDIA and ATI both optimize for this benchmark, and we know that ATI has access to the developer's version of this benchmark. And obviously Futuremark's intent is not the same as NVIDIA or ATI. NVIDIA (and ATI) wants the benchmark to be as fast and as smooth as possible on their cards without corrupting image quality that we can see.

apparently it is not so obvious... else there would be far fewer people in these forums questioning the content of your posts

the point is yes... make the benchmarks as FAST as possible... but if you alter the way the benchmark itself is working... that is no longer 'optimizing'

that is what everyone has been trying to tell you... and to an extent... nv40... but that basic point is not being picked up

this PRODUCT (3dmark03) works in a particular manner... IF the way it works is tampered with.. it is NOT 'optimizing'

on the other hand if nvidia or ati or anyone else find a way to improve performance through more efficient driver optimizations WITHOUT altering teh way 3dmark03 works... then great.. thats legit...

futuremark designed the benchmark in a specific manner... READ the whitepaper and see what they stated.. and read the link I provided above for a response to what nvidia alleged...

the amount the cheats are boosting scores may not be known but the fact that there is a boost is not disputed I take it

[Nv40]

Originally posted by Sazar
one point... to contradict a FACT...

the doom3 benchmark scenes were changed by ID... the benchmark itself was not changed... ie the engine and everything else remained the same...

ID's reasons for this were that they felt the scenes nvidia had recorded showed their product in a better light than what was expected... again... ID did not change the BENCHMARK... they changed the scenes used... AFAIK...
[/b]

its not needed here the semantics ,the point is Doom3 benchamrks
were correct....we have J.C word that they represent an accurate
benchmark of the FInal game. however is it true that the game is far
from be FInal . THere are more levels and single player missions to be done, but the engine is finished.

Means that ATI cards have an oportunity to score much better in the final game ,but not because the game is not final ,but because ATI like any other IHV can improve their drivers.

[gordon151]

Originally posted by jimmyjames123
The idea is that there are no concrete "rules as to how the benchmark is run". There is no concrete definition of what makes up a driver "optimization". NVIDIA does not even have authorized access to the 3dmark03 developer tools that some other websites are using. Futuremark themselves is very inconsistent. They don't allow non-WHQL drivers, but they allow overclocked graphics cards and cpu's. Don't you think that "skews" results in a way that driver "optimizations" wouldn't? That introduces yet another variable, and the graphics cards (and cpu's) that overclock better will have a natural advantage.

The idea is concrete which is something that you're simply not absorbing. It's not how you render all the information it is that you do render all the information and that you do it in a consistent matter which isn't specific to that benchmark. This is what I would assume Futuremark had hoped would be the situation with 3DMark03 and is the very basis of maintaining a very even field in terms of benchmarking.

It's also important to note this represents no effort on the part of Nvidia to increase driver performance. Performance is gained by reducing the amount of information rendered, which is extremely pointless since that makes comparisons between competing platforms impossible. That is why people refer to this as "cheating" and not "optimizing" as it changes the benchmark not the drivers.

To suggest that such a move is ok is utterly ridiculous and I'm not sure why you keep entertaining that thought.