View Single Post
Old 05-25-03, 12:58 PM   #22
Atari STE 4-bit color
Morrow's Avatar
Join Date: May 2003
Posts: 798

You don't get it! optimization... reordering shader programs.... that's all beautiful and true but isn't allowed by FutureMark!

let's see, official FutureMark statement made one week after the release of 3dmark03 concerning driver optimizations:

* A concern has been expressed, that synthetic benchmarks force hardware manufactures to optimize drivers for that specific benchmark (3dmark03).

* This is because ajy specific driver tuning might produce results that are not genuinely comparable.

* Furthermore, 3DMark03 includes advanced analytical tools to enable independent observers to catch any potential questionable driver optimizations. By taking a tough stance against any kind of driver optimization, the media can discourage this practice.

You see, it's not about making optimization which are normally done in games it's about changing the 3dmark code in a way that your hardware can produce higher scores! To summarize the FM's statement: ANY optimization or change in a driver to put your hardware in a better light is not legitimate and by that a cheat.

Of course nvidia cheated a lot more and probably also even a lot "better", ATI is by no means harmless and not unguilty considering this whole 3dmark03 driver optimization/cheat debacle.

Optimizations are legitimate in games but when the purpose of a benchmark is to demonstrate raw performance and not optimized performance of graphiccards, both companies are now guilty of not following the rules. The fact that ATI is still a FM beta member and paying FM and nevertheless cheating on the FM rules, makes the situation certainly not less severe.

Don't come now with, but Tim Sweeney said... What Tim said is true for games but not for 3dmark03! Big difference. Rules are rules
I got this post from another news website and found it very interesting. The poster seems to have some valid points, even though that puts nvidia not in a better light. It's not about that.

He just seems trying to open the eyes of many people which have now been fooled by ATI PR who are trying to keep themselves out of the driver optimizations discussions by saying that they can do driver optimizations and not cheats although FM rules explicitely forbid to use any optimizations for their benches...

In fact, this whole 3dmark03 driver foundings are nothing new and nothing to be surprised of. Cheat/optimizations have been practised by every company since day one.

There is just one thing which confuses me: FM said they want to benchmark standard 3d performance in future games! How can they put "standard code" and "future games" in one sentence since we all know that current games and future games will rely heavyly on driver optimization (especially "the" future game for many years come, namely Doom3). It's all about optimization and vendor specific paths so how can 3dmark03 be a reliable indicator of future games?

All I know now is that I will not use 3dmark to decide whether I'm going to buy an ATI or nvidia graphiccard but I will wait until the next generations games are out (HL2, DX2. Doom3) and then decide which card performs the best... I believe this 3dmark03 bench is as synthetic as it gets...
Morrow is offline   Reply With Quote