Futuremark releases 3DMark03 patch - alleges NVIDIA cheats

[lukar]

Actually the fact is that my Radeon 9700Pro beats GFX 5800ultra and GFX 5900Ultra. The other fact is that Radeon 9800Pro beat **** out GFX 5900Ultra.

http://www.hothardware.com/hh_files/...mb_gffx5900upd(2).shtml

Radeon 9800Pro 256Mb 5747

Radeon 9700Pro (My score) 5379 with patch, with no patch 5460

GFX 5900 Ultra 4886

[Nv40]

Originally posted by PreservedSwine
Because it is a benchmark, NOT a game. Two cards excecuting the same intstruction sets.

The benchmark is inefficient by design, as it is meant to stress the hardware capabilities of existing hardware. It may re-draw some scenes multiple times, in an effort to fully tax the GPU. If one card doesn't do the re-draws, it has not completed the benchmark efectivley, and has a strong fpa advatage over the others.

Now, if that takes case in a game, who cares? No one- it's an optimization, taking advantage of a poorly coded game.

the benchmark is not inefficient by purpose.. what is inefficient is
-> their programmers.. because you get lower frame rates than Doom3 ,in some of their test ,and the image quality is FAR LOWER than Doom3.

3dmark claims to be an accurate representation of the games of the future.. Obviously that simply marketing BS!.
they need to sell the program ,so they cheat gamers by saying those lies .

gamers dont know how dangerous can beFuturemark to the industry..
it not only mislead them by a performance they will never see in the real world. but it also force IHV to Optimize their hardware and drivers for
their inefficient programming..

just look at what Nvidia programmers have told..

Benchmarks drive hardware and driver development because high scores are essential. Good benchmarks benefit the consumer - usually because they measure things that games do - or because they are actually a game or application. Bad benchmarks hurt consumers because push technology in the wrong direction. So in the end we spend effort educating people on what's wrong with bad benchmarks or we spend hardware and engineering resources on optimizing for bad benchmarks.

------------------------------------------------

If 3DMark's algorithm was part of a synthetic vertex program benchmark, that might be OK. However, it's not presented as such. It's presented as a gaming benchmark, and it's presented as even being comparable to what Doom3 does, and certainly as being a reasonably representative shadow-volume-employing game. However, it is grossly incomparable to Doom3, and it is definitely a very, very poor way for any game to implement shadow volumes

---------------------------------------
the best way for us to win the benchmark down the road is to put in incredibly convoluted driver logic that -- in short -- detects everything stupid that the app is doing, and reimplements it using a smarter/faster algorithm, all behind the app's back. I'll leave thinking of the possible ways we could do this up to you; suffice it to say that implementing such code will help 3DMark03 and 3DMark03 *only*, at the expense of our time and at the cost of making our driver quite a bit bigger and buggier.

The dumber and more popular the benchmark, the worse the driver benchmarksmanship that is required. The more driver benchmarksmanship, the less valuable the benchmark.

And at least if it was a game, such benchmarksmanship would improve people's gaming experience. People who like Quake 3 engine games (and there are a bunch of those games) have undoubtedly benefited from the driver tuning we and ATI and others have done for the Quake 3 engine. I'm sure the same will be true for UT2003 and for Doom 3, and it's been true for other popular game benchmarks in the past.

But what gamer benefits from benchmarksmanship targeted at 3DMark03?

Bad but popular benchmarks hurt everyone

so there you see... Do you want to see IHV vendors "optimizing" for
a benchmark that will not benefit in any way in your games ?
you have a few 5-10? programmers (MORONS) at Futuremark ,that doesnt have experience in game development ,but only on good looking DEMOS.. but that have many really good MARKETING GUys there ,that now have conviced OEMS ,gamers ,and many sites ,to use the benchmark as the proof of video card
performance in games of the future..

IMO.. the best thing for the gaming industry ,is to stop following and
supporting 3dmark(s) in reviews ,that does not offer any benefits in
the gamer experience ,not now ,and not in the future.

[Ady]

Originally posted by nForceMan
Why people believe a company which is bribed by ati?

are you saying ATi have bribed Nvidia? Bribed them into cheating and making slower hardware? Wow, what an accusation.

[dpollard55]

O.k. I got about half way through this thread before I finally felt the need to chime in here (like my opinion realy matters...). Unfortunately the solution to this problem is to make it so neither company can cheat at all, which means not using a "standard" benchmark like 3dmark. Reviewers should ignore benchmarks and play some actual games, and I mean actualy PLAY the thing don't just run some time demo. The games should be picked at random only moments before the review, and should be different with each review. The only way to cheat in this instance would be to improve gameplay in all games because you wouldn't know which ones were being used in the reviews. We as consumers would get nice honest results this way (most of the time anyway). And just for the record I am still happy with my 5200.

[zakelwe]

IF you want to use 3dmark for benching then use 3dmark2001 as it is more relevant for todays cards and drivers and shows if your rig is cpu limited.

And there's less "optimisations" from Ati and nvidia in their drivers towards it.

3dmark03 still is bugged by nVidia's optimisations even after build 330 unfortunately, ie it does not take them all out.

Regards

Andy

[jAkUp]

the problem is that reviewers will get different scores every single time if they just "play" the game... depending on how many enemies were shown, how many gibs... etc. its always gonna be different. its not an accurate way to show the differences in hardware, unless one video card is significantly slower... but with close video cards.. you will never be able to tell..

[solofly]

Look!

http://www.hothardware.com/hh_files/...x5900upd.shtml

[Pafet]

stop bashing 3Dmark!!
it's ment mainly for comparison (by most of us atleast, I believe) not for predicting thisgame and that!! no one is that lame saying he gets 30fps in that test so every game will work with 30 fps.

[Ady]

Originally posted by zakelwe
IF you want to use 3dmark for benching then use 3dmark2001 as it is more relevant for todays cards and drivers and shows if your rig is cpu limited.

And there's less "optimisations" from Ati and nvidia in their drivers towards it.

3dmark03 still is bugged by nVidia's optimisations even after build 330 unfortunately, ie it does not take them all out.

Regards

Andy

3dmark 2001 is useless for testing the latest high end cards.

Could you please elaborate on your comment about build 330 not taking out all of nvidia's cheats?

[mbvgp]

For a change I agree with what nv40 is saying. I also agree that gt2 and 3 really look substandard and ugly. GT1 looks alright and GT4 also looks fine. In the end 3dmark does not do the job of a synthetic benchmark ( like shadermark ) nor does it represent a game.

IMO this is what most review sites should do,
Compile a list of games to benchmarks. Most of these games will have demo record and playback. So all the reviewer has to do is sit down and play for 5 mins and record a demo of a few maps. Play back these demos and note the fps.
Also people should do some sort of ingame testing like in the nvnews reviews. And give average benchmark numbers using fraps. There will be some error in this case but it will give a true indication of what each cards are capable off.

[jAkUp]

Originally posted by mbvgp
Compile a list of games to benchmarks. Most of these games will have demo record and playback. So all the reviewer has to do is sit down and play for 5 mins and record a demo of a few maps. Play back these demos and note the fps.

now there is a good idea... i know quake3 and unreal have the ingame demo recording...

[mbvgp]

Originally posted by jAkUp
the problem is that reviewers will get different scores every single time if they just "play" the game... depending on how many enemies were shown, how many gibs... etc. its always gonna be different. its not an accurate way to show the differences in hardware, unless one video card is significantly slower... but with close video cards.. you will never be able to tell..

It will be different if done once. But when the review is done over a period of few days(which is what it takes anyways ) and all the scores are averaged out then you should have enough data to give you an accurate inference.
Right now all it seems reviewers to is run a few predefined scripts which wont take that much time. And then spend a few days in writing the review.