Why is NVIDIA against 3DMark03?

[Nutty]

Nutty, what portions of the whitepaper point to it being poorly coded?

Okay here;

When using this kind of stencil shadowing, the developer is left with some options on the implementation.
3DMark03 does as much work as possible in the vertex shaders, since the goal of 3DMark03 is to measure
vertex and pixel shader performance in 3D games. Also, it is expected that many games with similar technology
will have a heavy workload for the CPU doing physics (including collision detection), artificial intelligence and
visibility optimizations for example. It is therefore desirable to perform as much as possible on the graphics card
in order to offload the CPU.
An alternative implementation would be to give some of the graphics tasks to the CPU, and thereby offloading
the graphics card. The skinning could be done on the CPU, which would reduce the amount of vertex shader
tasks. Also, when pre-skinning on the CPU, the characters would not need to be re-skinned for each rendering
pass.

See, they admit that they waste gpu cycles by re-skinning the character _every_ pass. Doom3 engine does not do this, and neither will any of the games using its engine. It's crap.

I think you will se a mystical disapearance of Nutty from this thread now

Nope.

Legion88, I dont distpute that a gpu benchmark, should push the gpu hard with little emphasis on the cpu. But if thats what it is, they should stop calling it a gaming benchmark, and stop implying that their bloated inefficient techniques are going to be used by the game engines of tomorrow.

Doom3 for example is being written to run on a Gf256 upwards. I cant imagine how poorly 3dmark03 would run on that. I barely got any better than a slideshow on a GF4TI.

Originally posted by Sazar
the reason it matters is that these companies have influence... dont forget intel makes boards and intel actually has a far larger slice of the cpu market than amd... dell is a big seller of intel parts exclusively intel desktops)

so these things add up... ie.. give credible weight to the legitimacy of 3dmark03 as a legit benchmark...

its just the way it is...

now when it comes to advertising... yer opening a can of worms slamming other companies in defense (in a manner of speaking) of nvidia

basically what i'm saying is these guys are full of it. it's marketing, and that's all it is. just like nvidia marketing. while i don't disbelieve some of what they(nvidia) say about 3dmark03 is true, it's obvious to anyone and everyone that they are complaining only because they are not the graphics kings.

i mean really, why would a CPU manufacturer endorse a benchmark where the CPU plays such a little role? it just doesn't fit. if anything, Intel wants the CPU to always be important, otherwise they will lose business.

Dell obviously does have something to gain. everyone with DX8 or lower hardware will run 3dmark03 then decide they need DX9 cards and boom, Dell sells more lucrative high profit systems.

honestly, i would never trust a statement made from a company that tries to sell me something. the motivation is in the wrong place. however, i will evaluate statements on my own and discern what truth there may be in them.

so it all comes down to this. yes, the statements "matter" in that your everyday Joe might see statements from big players like Intel, Dell, and ATI supporting 3dmark03 and agree with them. to me, it means nothing.

[PreservedSwine]

Nutty, selective quoting can be misleading, don't you think?

Perhaps you should include the rest of the paragraph you quoted, as it explains exactly why is was done the way it is, and the lack of advantages in off-loading the work to the cpu....Seeing as this is mainly a GRAPHICS card benchmark.... You failed to include it in your quotes......

You also failed to mention the physics engine at work in 3Dmark.

Hardly reason enough to call it crap, even according to your after-thought analysis, don't you think?

[Nutty]

Physics? In a hard-coded demo?

I think you'll find _actual_ playable games require alot more cpu resources for physics, and collision, and 3d sound production that the 3dmark03 demo.

And even then it's still the preferred choice to use the cpu for silhouette finding in the Doom3 engine. And it makes heavy use of physics on the cpu.

[PreservedSwine]

Physics? In a hard-coded demo?

Funny that you have pretended to read the whitepaper, when it's obvious you have not....

To be an accurate benchmark, the Game tests of 3Dm03 are rendered in D3D in real time. "It is important to note that these renderings are not merely animations or a set of recorded events, they are designed to function as 3D games work..."

Besides 3D rendering, typical 3D games include CPU workloads. In 3Dm03, we use calculations to represent this workload. It is important to note that these calculations are being perfromed in real-time to accuratley represent real game usage. Natural movements such as falling characters and crashes in 3Dm03 use real time physics. Havocks Game Dynamics SDK provides the real time physics functionality

You refer to 3Dm03 as a demo, when it's clearly more than that.

EDIT: re-worded a sentance or two as to not be too mean

Also wanted to add, agree w/ some of what is implied here :

I dont distpute that a gpu benchmark, should push the gpu hard with little emphasis on the cpu. But if thats what it is, they should stop calling it a gaming benchmark, and stop implying that their bloated inefficient techniques are going to be used by the game engines of tomorrow.

If you simply wish to argue semantics, so be it.....As if "Gamers" aren't interested in top of the line GPU's performance and technology.....

If anything, the two largest enthusiast groups interested in GPU performance are gamers and game designers.....kind of logical to call it a gamers bench if you ask me, but I can see your point as well. I don't think there is a definitive "right" or "wrong" answer there, and I don't think it worthy of debate.....

As far as "ineffient techniques" are concerned, how did you miss the online publoications that overviewed 3Dm03. Each one decidely disagree w/ Nvidia on this issue. If Nvidia is attacking the skinning techniques, I'd bet thier vertex shaders are weak....

[legion88]

Originally posted by StealthHawk
that's just the problem. everything would be fine and dandy if Futuremark didn't advertise 3dmark03 as "the gamer's benchmark." i don't know about you, but as a gamer, i want a game that depicts some facet of realworld performance. and that means taking into account the CPU.

i don't think anyone is contesting that 3dmark03 is a video intensive benchmark. it is. and the results show that it is very clearly without a doubt. they are contesting that it shouldn't be, since that is not indicative of a realistic gaming experience.

There are three general categories of performance for games. These categories are: 1) graphics, 2) CPU, and 3) impact of sound on CPU performance.

3DMark2003 measures performance in all three categories separately. Thus, as a gamer, I would know if I would require a CPU upgrade or a graphics card upgrade by comparing the scores I received to everyone else that cared to upload their scores to ORB.

Benchmarking UT or Doom3 or any other game does little to determine where the bottleneck is for the inexperience user. How many times did the "question" come up after benchmarking using a favorite game? You know, "do I need to upgrade my CPU or graphics card to get better frame rates?"

You people need to stop treating 3DMark2003 as if it was just a single benchmark test. It is not single benchmark test. Throughout Futuremark/Madonion's history, 3DMark was never a single benchmark test. It is a collection of benchmarks. With the exception of 3DMark2001, 3DMark had separate scores for the graphics portion and CPU portion, and each portion consisted of multiple benchmark tests.

[John Reynolds]

Originally posted by Joe DeFuria
Somebody shoot me...I agree with Roscoe!

Blam!

Hey, wait, I agree with him too. Angels and ministers of grace defend us.

[Spiritwalker]

Originally posted by StealthHawk
compare the performance of GT2 and GT3 with that of the Doom3 alpha.

Just to refresh your memory, Doom]l[ is an OpenGL game NOT a DX9 game so you cannot make the comparison.

That and the fact that JC has specifically coded the game for the nV30.

3Dm3k is a singular bench written to conform to the DXx specs. How a certain card goes about that is up to the IHV.

[Shinri Hikari]

Driver Issues and Hardware bugs, their drivers and card specs are immature which hurts performance and they think 3dmark2k3 is not cutting them any slack for it. If there were any way that they could have avoided this current situation they would gladly take that option.

[kyleb]

Originally posted by Spiritwalker
Just to refresh your memory, Doom]l[ is an OpenGL game NOT a DX9 game so you cannot make the comparison.

well it all come down to the same hardware no mater wich api you use, ganted driver preformace is an important factor. regardless, doom3, while being an alpha, is still intended to proved playable preformace on less than modren hardware. on the otherhand 3dmark as a benchmark that does not need to provide playable framerate works to push things much harder to the point were systems that do well in doom3 will still absolutly choke. heck, no one runs 3dmark with turely playable framerate at this point, at least not all the way though.

[tamattack]

Originally posted by Nutty
It's badly written because it does soo much redundant stuff. Like skinning the characters 3 times!

How is that a bad thing per se? Let me ask you this: if you wanted to benchmark the speed at which your PC could perform 1000 additions, how would you program it?

[1] 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + ... + 1001; or

[2] 1 + 2 (repeat 1000 times)

Obviously [2] is much more redundant than [1], yet it is also much more efficient for this purpose. Personally, I would do [2] because I'm lazy. Are programmers any different? More to the point, is there much of a difference between [1] and [2] above (other than the result of course) if your goal is to benchmark adding speed?

Now apply this line of thought to 3DMark03 and see if you can spot the similarity.

Originally posted by Nutty
No 3d-engine coder in the games industry would do that.

Well duh! Obviously their purposes are completely different. Futuremark wanted to stress the vidcard. Game coders want compatibility with a large install base.

Something to think about.

[Spiritwalker]

Originally posted by kyleb
well it all come down to the same hardware no mater wich api you use, ganted driver preformace is an important factor. regardless, doom3, while being an alpha, is still intended to proved playable preformace on less than modren hardware. on the otherhand 3dmark as a benchmark that does not need to provide playable framerate works to push things much harder to the point were systems that do well in doom3 will still absolutly choke. heck, no one runs 3dmark with turely playable framerate at this point, at least not all the way though.

Just as a point, remember that JC stated when running spec OpenGL (ARB2) code just as the 3Dm3k is using spec DXx code, and both are using their same hardware paths, that the R9700 was twice as fast as the nV30.

So basically the nV30 is faster running spec DX code than it is at OpenGL code and yet they have to rely on guys like JC to specifically code a game to use their hardware. Then they complain that real games should be used to benchmark with. Now with 3Dm3k confirming what JC found with the ARB2 path we know that the 9700 could very well be faster at next gen games (if they follow spec). I find this rather 2 faced.

A benchmark like this is intended to test how the hardware is according to the spec, not the IHVs version of that spec.

If this turns out to be a driver related thing, well then fine, but I believe that it is a hardware thing, judging by the DX9 specific tests, not the game tests.