nV News Forums


nV News Forums (http://www.nvnews.net/vbulletin/index.php)
-   Benchmarking And Overclocking (http://www.nvnews.net/vbulletin/forumdisplay.php?f=24)
-   -   NVIDIA's detailed slam of 3DMark03 (http://www.nvnews.net/vbulletin/showthread.php?t=7455)

Hellbinder 02-11-03 10:40 PM

Here is Nvidias detailed slam of 3dmark03
"3DMark03 combines custom artwork with a custom rendering engine that creates a set of demo scenes that, while pretty, have very little to do with actual games. It is much better termed a demo than a benchmark. The examples included in this report illustrate that 3DMark03 does not represent games, can never be used as a stand-in for games, and should not be used as a gamers’ benchmark."

"Unfortunately, Futuremark chose a flight simulation scene for this test (game 1). This genre of games is not only a small fraction of the game market (approximately 1%), but utilizes a simplistic rendering style common to this genre. Further, the specific scene chosen is a high altitude flight simulation, which is indicative of only a small fraction of that 1%."

"For all intents and purposes game tests 2 and 3 are the same test. They use the same rendering paths and the same feature set. The sole difference in these tests appears to be the artwork. This fact alone raises some questions about breadth of game genres addressed by 3DMark03. --- These two tests attempt to duplicate the “Z-first” rendering style used in the upcoming first-person shooter game, “Doom 3”. They have a “Doom-like” look, but use a bizarre rendering method that is far from Doom 3 or any other known game application."

"Finally, the choice of pixel shaders in game tests 2 and 3 is also odd. These tests use ps1.4 for all the pixel shaders in the scenes. Fallback versions of the pixel shaders are provided in ps1.1 for hardware that doesn’t support ps1.4. Conspicuously absent from these scenes, however, is any ps1.3 pixel shaders. Current DirectX 8.0 (DX8) games, such as Tiger Woods and Unreal Tournament 2003, all use ps1.1 and ps1.3 pixel shaders. Few, if any, are using ps1.4."

"This year’s 3DMark has a new nature scene (game 4). It is intended to represent the new DirectX 9.0 (DX9) applications targeted for release this year. The key issue with this game scene is that it is barely DX9."

"So, where do you find a true gamers’ benchmark? How about running actual games? Most popular games include a benchmark mode for just this purpose. Doom3, Unreal Tournament 2003, and Serious Sam Second Encounter are all far better indicators of current and upcoming game performance."

jAkUp 02-11-03 10:45 PM

lol they comment on how there arent many flight simulator games?? thats why the geforcefx sucks at it?? bwahahaha... thats ridiculous.

Hellbinder 02-11-03 10:48 PM

Its pretty sad, Now they are slamming flight simulators.. :rolleyes:

I could counter every single one of their arguments, but the outrigth absurdity of these comments is readily apparent to any honest person. What is really sad are all the comments i have read at futuremark message boards about how Nvidia has a point...

Just one small comment from me on this, want to see how decietfull they are? look at their comment about PS 1.4 Vs PS 1.2/1.3

1. They name the only two games in exsistance that use PS 1.3 and call it many games. Completely negleting the PS 1.4 games. Like madden 2003, or even the comming Doom-III you need to ask yourself if PS 1.3 was so great, then why isnt it being used by JC on doom-III??? he only singles out PS 1.4 as well.

Ill tell you why. Becuase Nvidia are a bunch of deciefull LIARS. PS 1.2/1.3 dont affect performance AT ALL. You dont get any performance gain over PS 1.1 which is why is does not make any sense to use them. and nearly no one does. Wheras PS 1.4 is a subset of PS 2.0 and basically Cuts the numer of passes needed to render compaitable scenes by 1/2.

Yet another patently Unethical and decietfull move by Nvidia.

Bigus Dickus 02-11-03 10:51 PM

You know, nVidia's points would be considered valid if the hypocrisy wasn't just oozing from every word written. Their past behavior concerning benchmarks makes their claims now completely irrelevant.

jAkUp 02-11-03 10:55 PM

they continue to make excuses on top of lies on top of more excuses... they are digging themselves a grave. i cant belive this is the same nvidia i have used products for the past 3 years... the same nvidia that i was so excited about... i cant even believe my own ears when i say it... but im starting to take a good look at ati now. i cant stand anything or anyone that talks **** and cant back it up. i cant belive i have followed every single announcement from the very beginning when they first announced geforcefx, i waited and waited, they promised the "next big thing" delays, lies, excuses, ****ty product. thank you nvidia... i actually used to like you guys. thank you for making me wait 4 ****ing months on a geforce2mx! the geforcefx was the whole reason i was waiting! im tired of waiting. if i dont hear something from nvidia in the next couple of weeks that isnt b.s., my money goes to ati.

Joe DeFuria 02-11-03 10:57 PM


I seem to distinctly remember some video card company *cough* nVIDIA, *cough*, claim how "anything they do with PS 1.4, we can do with PS 1.1.

Hypocriscy doesn't even begin to describe nVidia's stance on this matter....

Hellbinder 02-11-03 10:58 PM

Another example.. becuase it just pisses me off so much that there are people in the world like these slime.

Last years nature demo even in the Nature test only used PS 1.1 even though it came out after the GF4. meaning they never cried at all that their precious PS 1.2/1.3 (that do nothing noteworthy) were not used. Even though the radeon 8500 got completely blown off and not properly supported in any of teh SE benchmarks.

Oh another thing from the last version. how many games have you seen with a woman ridding a dragon and torching a bunch of little guys??? Or how many games have you seen with a Truck driving through a wasteland getting shot by a giant robot with missles???


These comments from Nvidia are completely Infantile. Its to bad non one wil do anything about it, and there are already an army of Nvidia types chanting this very montra at other sites...

Burn the Flight Sims... Burn the Flight sims... :angel2:

Spiritwalker 02-11-03 11:00 PM

more from www.extremetech.com

The 2003 version of FutureMark Corp.'s 3DMark graphics benchmark was criticized by executives at Nvidia as unrealistic, who added that they had decided not to assist the developers in their beta testing.


FutureMark's "benchmark for gamers," 3DMark, is one of the leading "synthetic" benchmarks to test 3D applications. Testing agencies, such as ExtremeTech, routinely use a mix of application-based and synthetic benchmarks to test a product's real-world and theoretical performance under different constraints.

Nvidia executives said they had tolerated previous 3DMark versions, but that FutureMark's latest effort didn't give an adequate indication of real-world performance. However, FutureMark executives said they had moved their benchmarking suite in the right direction.

"High-end 3D effects are no longer the sole domain of serious gamers, but are becoming ubiquitous in mainstream applications," said Tero Sarkkinen, executive vice president of sales and marketing for FutureMark, in a statement.

"Since the release of the first version of 3DMark in 1998, we've been continually striving to improve our benchmarks to help users maximize their experience and PC performance," Sarkkinen added. "3DMark03 meets this goal by supporting DirectX 9.0 and providing detailed 3D performance measurements for existing systems, while offering challenging, new tests for current, cutting-edge hardware and technologies yet to be released."

In previous versions of 3DMark, like the current version, FutureMark crafted a custom 3D engine to run the tests; part of the 3DMark 2001 code was used to create the game Max Payne. So far, no developer has publicly announced its intention to use the engine in a commercial game, and a white paper developed by FutureMark says that the developer created a lightweight DirectX 9.0 wrapper for the new code base.

That, Nvidia executives said, meant that any assistance that the company would have given FutureMark would have been wasted.

"The reason that we're not all gung ho about it is that (3DMark'03) is not representative of (actual) games, nor is it a good benchmark," said Tony Tamasi, senior director of desktop product management at Nvidia. "That means Nvidia has to expend effort to make sure it runs well on our hardware. All that energy that we spend doesn't benefit the user. None. Zero. All that effort doesn't go to benefit any game, either. That's kind of depressing."

For that reason, Nvidia didn't participate in the beta testing of the 3DMark benchmark, only doing some last-minute driver tweaks, Tamasi said.

"Being a beta partner at all required us to pay money to Mad Onion [the previous name for FutureMark] and that seemed really wrong," Tamasi said. "Don't get me wrong -- that's a tough business. I totally appreciate the quandary that MadOnion is in… But from our perspective it didn't make sense for Nvidia to give them money for something we didn't believe in."

Specifically, Tamasi said he objected that Futuremark apparently chose to emphasize single-textured pixels in the benchmark's four tests, while previous versions had pushed multitexturing. Tamasi also criticized FutureMark's use of older version pixel and vertex shaders, and the benchmark's heavy emphasis on running and rerunning vertex shader operations -- 36 times, by his count.

Another hardware testing website, HardOCP.com, said Tuesday that it would not to use the technology in future tests of graphics cards. However, FutureMark claimed that testing shaders was critical to assessing a product's capabilities.

"Pixel shader programs, unlike their vertex counterparts, require hardware support," the company's white paper states. "Vertex and pixel shaders have become an important part of 3D graphics and are accordingly featured prominently in 3DMark03. One of the game tests, the DirectX 9 showcase, uses 2.0 vertex shaders and 2.0 pixel shaders. All other games tests use 1.1 vertex shaders. The DirectX 8 game tests use 1.4 pixel shaders if available; otherwise they default to 1.1 pixel shaders."

FutureMark allows free downloads of its standard benchmark, but charges $39.95 for individual copies or $250 for a corporate license to a "professional" version.

The lack of support from Nvidia is telling. Jon Peddie Research estimates that approximately 53 million PC graphics devices shipped from nine suppliers in the fourth quarter of 2002, a 13 percent increase over the previous quarter, and a market that was, somewhat surprisingly, controlled by Nvidia. Nvidia led the market with a 32 percent market share, followed closely by integrated-graphics powerhouse Intel, with 28 percent. ATI finished third with a 19 percent share, followed by Via and SIS. Matrox, Trident, Silicon Motion and 3Dlabs each controlled 1 percent of the markt, or less.

So what is Nvidia's choice for benchmarking? "Games," Tamasi said. "Use games."

Hellbinder 02-11-03 11:16 PM

More Bull****.

Everyone knows that the move in future games is AWAY from Multi-texture games, and towards Single texture games with lots of shaders.

NVidia Themselves have been pushing this idea right along woth everyone else. I have been in Countless discussions about this at B3D and other web sites...

digitalwanderer 02-11-03 11:45 PM

What were they supposed to say? "It's a great benchmark, it's just our cards suck at it."?

My question is if nVidia was so against 3dm2k3 for so long, how come they were an active participating partner 'til December of last year?

Is it an odd coincidence they quit around the time of the original FXs first supposed appearance, or am I just being overly conspiracy minded again? :hmm:

jAkUp 02-11-03 11:50 PM

supposly this is the geforcefx performance in 3dmark 2003
check out the pixel shader 2.0 score....


Nv40 02-12-03 12:00 AM


Originally posted by Bigus Dickus
You know, nVidia's points would be considered valid if the hypocrisy wasn't just oozing from every word written. Their past behavior concerning benchmarks makes their claims now completely irrelevant.

for the first time i think i agree with you Biggus.. lol
Nvidia has a few points , sure the use of only PS 1.4
benefit primary the ATI hardware ,hardware optimized
since day 1 ,for that in mind...

if we look at PS1.1/1.3 tests in the old 3dmark..
NVidia wipe the floor of ATI video cards..
but ATI never complained in public the way NVidia
does today... :(

it doesnt look good Nvidia acting like they
are victims in the industry..
NVidia are the only ones responsible for all this mess...
the gamers will care less about 3dmark ,if at least the GeforceFx
performance in today games is indisputable ..but is not.

the radeon9700pro defeated the GeforceFX by a few FPS
in most games (the benchmarks that Nvidia agree to use) :)
when using AA+AF to the max..

this is like Boxing..
if you want to take the crown of a indusputable
for many years champion you need to knockout him
or win EVERy round so easily to avoid any mistake by
the judges :angel2:
about who is the clear winner.

if they want to get the CRown of ATI in
performance they need to release a card (not only killer in technology -like the NV30 is!!!) but killer in AA/AF and
most important killer in SPEED .. not trading back and forth in performace in most benchmarks like it is happening now..

if Nvidia doest believe that with their limited time /or resources
or other problems they cant compete at the top , they still can compete in the price/performance category. just like AMD .

the lack of a 30%-40% faster NV30 (like Nvidia promised)
is the ONLY! problem of Nvidia..

hardcore gamers will not have problem dealing with the dustbuster/with another PCi slot with the lot of heat ,with the a bit inferior AA/AF modes. not IF ,the card were 50% easily more faster than the R300..

and even 3dmark 2003 with all the unfairness and benefits
for ATI ,could be still faster in all test for Nvidia.

but it never happened..

so i really hope that next time NVidia folks
with the NV35?, take the speed crown of ATi , with a devils card ,that wipe the floor of the R300/R350/and competitive with R400 , in every possible
benchmark and unfair test that benefits more ATI hardware.

by the way .. my next card will be the NV35.. ;)

All times are GMT -5. The time now is 03:07 PM.

Powered by vBulletin® Version 3.7.1
Copyright ©2000 - 2015, Jelsoft Enterprises Ltd.
Copyright ©1998 - 2014, nV News.