My thoughts on "Optimization"

[jimmyjames123]

I have been reading about this 3dmark03 "Rendering Issue".

For one, I think the journalism at Extremetech and Beyond3d was a bit odd. Never do they mention the following relevant points:

1) Image quality was NOT compromised with the new FX 44.03 driver. In fact, most people have noted improvements in Image Quality, even in 3dmark03.

2) All images in 3dmark03 that we actually SEE are rendered correctly.

These points might sound obvious to some, but to someone casually reading or to someone who isn't very familiar with graphics card terminology, these points wouldn't be so obvious.

Now, ask yourself this question:

If we can get smoother and faster performance, without any loss of image quality, isn't that a good thing?

If I had an FX NVIDIA card, I would be happy that NVIDIA could "optimize" for 3dmark03 without any loss of image quality and without affecting any images that we actually see on screen

It is no secret that both NVIDIA and ATI "optimize" their drivers for a benchmark like 3dmark03. If they can do it without compromising image quality, all the better (IMO).

Some other points that are worth noting:

1) NVIDIA does not have authorized access to the developer's build of 3dmark, while ATI (and Extremetech, Beyond3d, etc) does, where they can roam around anywhere even off-center of the actual image displayed.

2) Futuremark strangely only allows WHQL certified drivers for published online results for their 3dmark program, but they allow overclocked graphics cards and cpu's (and note how Futuremark buries the Detonator FX driver underneath the others on their homepage annoucements, not even mentioning about how it is WHQL certified for GeForce FX cards).

3) [H]OCP talked with ATI privately about the quake/quack driver cheat issue for more than a month before writing their article. ATI repeatedly denied cheating, and [H]OCP released their article after they proved them wrong (showing that image quality was compromised on Quake 3 while enhancing performance).

4) The NVIDIA FX Detonator driver increased both performance and IQ noticeably for a variety of benchmarks and games on the FX graphics cards, not simply 3dmark03.

5) The Doom3 benchmark's used in the latest tests were chosen by ID and were not seen by NVIDIA in advance.

All in all, it boils down to what one considers an "optimization" vs. a "cheat". If a graphics card company can make performance smoother and faster without compromising image quality and without affecting what we actually see, then I would say this is a good thing (especially considering that NVIDIA and ATI both optimize for benchmarks in the first place).

[Dazz]

LMAO who the hells gonna admit they are cheating LMFAO

[digitalwanderer]

Originally posted by jimmyjames123
1) Image quality was NOT compromised with the new FX 44.03 driver. In fact, most people have noted improvements in Image Quality, even in 3dmark03.

2) All images in 3dmark03 that we actually SEE are rendered correctly.

Well, that is definately a nice summation of one popular point of view around here. There seems to be just about as many people though who'd jump up and say something like, "But it is a BENCHMARK, and it is meant to be run in a certain way so as to give an accurate comparison of performance. nVidia's driver cheats compromise that by bypassing the benchmark's parameters, which is NOT optimizing but is fraudulently inflating their scores to sell more cards."

I'm one of them people.

Now if this were a GAME we were talking about I would agree with you, as optimizations that don't affect image quality/what you see ARE pure optimizations and nothing but a benefit to the consumer....but this ain't a game and the cheating is cheeseball.

[reever2]

Why dont people understand that this optimization can only be done in a benchmark? Static clip planes can not be put into games, if at any time the camera in 3dmark turned around you would see the clipping, i would say not being able to see 3/4 of the environment a huge detrimental effect to IQ

[jimmyjames123]

"But it is a BENCHMARK, and it is meant to be run in a certain way so as to give an accurate comparison of performance. nVidia's driver cheats compromise that by bypassing the benchmark's parameters, which is NOT optimizing but is fraudulently inflating their scores to sell more cards."

Trust me, I fully understand this point of view. However, we all know that NVIDIA and ATI actively "optimize" their drivers for enhanced performance in 3dmark programs. This alone undercuts the argument about an "accurate comparision of performance", because it has been repeatedly shown that driver "optimizations" alone can (sometimes significantly) improve performance.

There is also the issue about what is an "optimization" and what is a "cheat". There is no agreement about what this distinction is. If NVIDIA (or ATI for that matter) can improve performance without compromising image quality, I'd like to think of that as an "optimization".

If anything, maybe this will help Futuremark to create a benchmark that is less susceptible to "optimizations", assuming that they truly want to create an "impartial" benchmarking program.

[jimmyjames123]

if at any time the camera in 3dmark turned around you would see the clipping, i would say not being able to see 3/4 of the environment a huge detrimental effect to IQ

The fact of the matter is that the cameras in 3dmark don't get turned around. So NVIDIA is "optimizing" for this particular benchmark. It is certainly no secret that both NVIDIA and ATI optimize for this benchmark. Users of FX cards now get smoother and faster performance and better image quality in 3dmark03 in what we can see. This whole issue is a matter of perspective.

[dpollard55]

Originally posted by jimmyjames123
The fact of the matter is that the cameras in 3dmark don't get turned around. So NVIDIA is "optimizing" for this particular benchmark. It is certainly no secret that both NVIDIA and ATI optimize for this benchmark. Users of FX cards now get smoother and faster performance and better image quality in 3dmark03 in what we can see. This whole issue is a matter of perspective.

Well said, and my thoughts exactly.

[digitalwanderer]

Originally posted by jimmyjames123
Trust me, I fully understand this point of view. However, we all know that NVIDIA and ATI actively "optimize" their drivers for enhanced performance in 3dmark programs. This alone undercuts the argument about an "accurate comparision of performance", because it has been repeatedly shown that driver "optimizations" alone can (sometimes significantly) improve performance.

There is also the issue about what is an "optimization" and what is a "cheat". There is no agreement about what this distinction is. If NVIDIA (or ATI for that matter) can improve performance without compromising image quality, I'd like to think of that as an "optimization".

If anything, maybe this will help Futuremark to create a benchmark that is less susceptible to "optimizations", assuming that they truly want to create an "impartial" benchmarking program.

Just because nVidia got busted with a flat-out cheat it does NOT invalidate 3dm2k3 as a benchmark, it invalidates nVidia as a trusted manufacturer.

[jimmyjames123]

Just because nVidia got busted with a flat-out cheat it does NOT invalidate 3dm2k3 as a benchmark, it invalidates nVidia as a trusted manufacturer.

According to that logic, ATI should not have been a trusted manufacturer ever since the quake/quack fiasco. As for 3dmark03 as a benchmark, I think there has already been enough talk about the shortcomings of this program as an "accurate" real-world benchmark tool. Also, we are back at the issue of "optimization" vs. "cheat".

I don't know how many times this has to be repeated, but here we go again: It is certainly no secret that both NVIDIA and ATI optimize for this benchmark. Users of FX cards now get smoother and faster performance and better image quality in 3dmark03 in what we can see.

[John Reynolds]

Originally posted by jimmyjames123
I don't know how many times this has to be repeated, but here we go again: It is certainly no secret that both NVIDIA and ATI optimize for this benchmark. Users of FX cards now get smoother and faster performance and better image quality in 3dmark03 in what we can see.

Someone started a thread on this forum before the ET article was published stating that they were seeing pixel smearing in 3DMark.

Plain and simple, whether you want to call it a cheat or an optimization, Nvidia boards are not rendering this synthetic correctly. The human eye might have a hard time detecting this, but it's there nonetheless.

[jimmyjames123]

Someone started a thread on this forum before the ET article was published stating that they were seeing pixel smearing in 3DMark. Plain and simple, whether you want to call it a cheat or an optimization, Nvidia boards are not rendering this synthetic correctly. The human eye might have a hard time detecting this, but it's there nonetheless.

I've read several reviews on the FX 5900 Ultra, and no reviewer has mentioned any corruption of image quality with 3dmark03 using the new Detonator FX 44.03 drivers with what we can actually see. In fact, it seems that both Image quality and performance are improved, sometimes significantly. Also, I read somewhere about how there was a pixel smear due to a video capture error, not an smearing of the image in the actual demo.

[Dazz]

Originally posted by jimmyjames123
I've read several reviews on the FX 5900 Ultra, and no reviewer has mentioned any corruption of image quality with 3dmark03 using the new Detonator FX 44.03 drivers. In fact, it seems that both Image quality and performance are improved, sometimes significantly.

Thats due to people leaving the bench running while they are away from the computer.