nV News Forums

 
 

nV News Forums (http://www.nvnews.net/vbulletin/index.php)
-   Benchmarking And Overclocking (http://www.nvnews.net/vbulletin/forumdisplay.php?f=24)
-   -   The ideal review would include... (http://www.nvnews.net/vbulletin/showthread.php?t=199)

pelly 07-30-02 02:10 PM

The Ideal Review...
 
I know Mike discussed this a bit earlier...but I think it still deserves osme attention...

What qualities does the ideal review have?

IE: What games do you want to see and what resolutions? Do you want/need samples of each combo of FSAA and Aniso or are the min and max enough?

Following Mike's lead, I have compiled my first review based upon "real" gaming instead of some pre-determined and structured demo. Using FRAPS, I would make my way to a certain point in the game, measuring fps along the way. Granted, each test will not be exactly the same as the other...However, I feel this method is the most accurate portrayal of how the card reacts in a real-world environment.

I neglected to include any resolution below 1024x768 as I feel the past few generations of video cards are powerful enough to power higher resolutions on nearly any system.

The inclusion of 1024x768 was also important because it represents the typical resolution of 15" and some 17" LCD monitors. With the growing popularity of these monitors, we have to consider them when writing reviews...

Well....I'm expecting some thorough responses...don't let me down!
:D

Smokey 07-31-02 11:46 AM

Here is what I would like to see.

Games no more than 12 months old
resolutions @
1024*768 - 32bit
1280*960 - 32bit
1280*1024 - 32bit
1600*1200 - 32bit

When seeing what new cards can do, 2+4xAA + min of 4x anistopic filtering is a must, and games (if any) that support both DirectX and OpenGL eq. UT but new games, I would like to see the results run on both APIs.

Now just to add a bit more to the games used, I think it would be good if there were games from different genes used, as not everyone just plays FPS ;)

Basiclly benchmarks have to relate to everyone, like using 2 or even three different systems, as not everyone upgades thier hole system all at once, many upgrade thier video card first and could be using a low end cpu. When looking at benchmarks, everyone should be able to say, ok now if I buy this card, this is what I can expect on my pc with game x at certain settings.

Most of todays setting really only reflect systems that have both the latest card(which is bieng reviewed) and the latest cpu, memory, motherboard etc. To me only a very small group of people would have these systems, so only very few could expect similar results on thier pcs if they upgraded thier video to what was being reviewed.


Sorry for the long post and I hope it all makes sense
:confused: but this is what I think alot of reviews are missing.


Smokey

pelly 07-31-02 12:32 PM

I think you are 100% correct....It would be great to have a sample of a few different CPU levels to see how the card scales...Unfortunately, that requires a fair amount of resources ( I know I can only afford 1 system ). As a result, we are left looking at work such as Anandtech's GPU scaling article for answers.

The next wave of reviews should be a breath of fresh-air for most. I know Mike is trying to take a new approach to testing cards and utilizing a wide variety of games...

Any non-1st person shooter games in particular?

Thanks for the reply Smokey!

:D

Smokey 07-31-02 01:09 PM

Here is an idea, if you only have one pc would you be able to underclock it? eg. running an AtlonXP @ 100 fsb?

As for any non fps games, that I cant really help with :( as the only game im playing is The Sum of All Fears, and then raven Shield, but i was thinking of others that play RPGs, flight sims, racing games, RTS games.

Another thing that I forgot to mention earlier, is that when taking screenshots for reviews, the FPS counter should be showing (which is done often on the site :D ) . I hate seeing screenshots with all the bells and whistles and not seeing the FPS counter, I always use the FPS ingame.

Hope this helps


Smokey

pelly 07-31-02 01:17 PM

My next system will be an AMD-based box and I plan on "unlocking" the CPU. This way, I can keep the same FSB more or less and just raise or lower the multiplier. This way, no setting has the advantage of the higher FSB which would skew the results...

Here is a list of what I think should make the cut:
  • 3dMark2001 SE build 330
  • Quake3 Arena v1.31
  • Return to Castle Wolfenstein
  • Jedi Knight II - Jedi Outcast
  • Soldier of Fortune II
  • Warcraft 3
  • Unreal Tournament G.O.T.Y Edition
  • Grand Theft Auto 3
  • Vulpine GLmark

Anything else? Is something there that shouldn't be?

PCarr78 07-31-02 02:04 PM

  • 3dMark2001 SE build 330 - NO, pure BS
  • Quake3 Arena v1.31 - Only 1024x768 and up with everything turned on
  • Return to Castle Wolfenstein - No. Not really stressful.
  • Jedi Knight II - Jedi Outcast - OK
  • Soldier of Fortune II Maybe
  • Warcraft 3 Not THAT stressful
  • Unreal Tournament G.O.T.Y Edition No. Too CPU dependent.
  • Grand Theft Auto 3 Yes. But come up with standard settings (veiwdist, etc)
  • Vulpine GLmark Too old, not stressful

What should be added is:

Flight sim 2002 with everything maxed (stressful as hell, because it's badly programmed.)

CMR2 (Very stressful game, especially with all the effects on full)

PCarr78 07-31-02 02:07 PM

I also forgot to mention commanche 4.

Morrowind too.

"Another thing that I forgot to mention earlier, is that when taking screenshots for reviews, the FPS counter should be showing (which is done often on the site ) . I hate seeing screenshots with all the bells and whistles and not seeing the FPS counter, I always use the FPS ingame." -Good suggestion

Spectral 07-31-02 02:08 PM

3dmark is a very viable benchmark. Anybody who says it isnt is scoring low on it.

You can see an improvment relative to hardware/software configurations and tweaking, and it directly correlates into real world games.

Why, Dan would it be a bad idea to use it as a bench.. It was DESIGNED as a 3d benchmark.

PCarr78 07-31-02 02:11 PM

It's trash

Biased.

And couldn't hope to represent realistic gaming scenarios.

Not to mention it's archaic.

Spectral 07-31-02 02:14 PM

Biased against what? If I made a 3d accelerator myself and benched it on 3dmark the scores would directly relate to how well it performed... So, how is that garbage?

And you mean to tell me that a card that scores a 5,000 compared to a card that scores 11,000 means absolutely nothing?

Couldnt hope to resemble gaming scenarios? I guarantee you the person who scores 11,000 over the person who scores 5,000 will have a better framerate and be able to have higher detail in their, "real world" games. Please, prove me wrong.

pelly 07-31-02 02:25 PM

For better or for worse, 3dMark will always be used as a benchmark. Everyone knows 3dMark and they can see a difference when new hardware is added or tweaks are made. That "magic" number at the end provides a quick and simple estimate of the card's overall performance...

There are definitely more accurate benchmarks out there....

As for WarCraft 3, I can assure you that the game IS stressfull.....though I can't begin to tell you why...I really can't see what is so taxing about this game....though I have seen how it hammers systems when the eye candy is turned on...very strange...

GTA3 should be used w/ view distance set to max....anything lower inhibits the playability of the game...

I'm a bit confused how you can include Q3 but not RTCW....Surely there is more detail in RTCW than in Q3....So seeing it at high res and full eye-candy should stress the card enough...

I'd like to see Morrowind included.....though I am not much of a fan of Comanche 4....

:D

PCarr78 07-31-02 02:34 PM

1) Nvidia optimized -> Biased

2) "And you mean to tell me that a card that scores a 5,000 compared to a card that scores 11,000 means absolutely nothing?" It does mean simething, but it isn't easily quantifiable.

3) "Couldnt hope to resemble gaming scenarios?" Yes. Name three games that use point sprites, for example. And the gaming situations depicted are unrealistic.
It uses maxfx engine, right? Which means unrealistically built 'maps'... Weird polycounts and all. Not to mention that certain level geometries are unobtainable with the mapbuilding techniques that are used. Also, the textures are really lowres (on the environments). Another thing is that there are very few special maps (texture maps) used. No bumpmaps, no refraction maps, etc. Particles play a very little part as well. These are not situations seen in typical games.

4) "I guarantee you the person who scores 11,000 over the person who scores 5,000 will have a better framerate and be able to have higher detail in their, "real world" games. Please, prove me wrong."

Simple. Most feature tests are done exclusively. A pixelshader test has pretty much just the pixel shaders there. The Dot3 test has a handful of polygons, a lightsource or two, and the maps. The pointsprite test has only pointsprites. The vertexshader test uses only the vertex shader engine.

A card may perform EXCELLENTLY on these tests. But in real life scenarios, where any number of the elements of the tests are involved, the card may not do so well.

Lets say I have an Nvidia CorpDanXP 5000+ video card, compared with an ATI SpectralPower Extreme Maxx.

While the CorporalDanXP card, with 120003dmarks, totally owns the Spectralpower card, which has 80003dmarks in the individual tests, the SpectralPower card fares much better under conditions where the elements are mixed because of it's more balanced architecture.

Get the idea? Because the CorporalDanXP card does SO WELL in the individual (let's call them shader) tests, it gets a really high number of 3dmarks. The Spectralpower card, which perfoms fairly badly in the individual shader tests, gets a lower mark.

When entering a game where all the shader techs are mixed, however, the spectralpower card might own the corporaldanxp card.

So, yes. 3dmark is not a good benchmarking tool.


All times are GMT -5. The time now is 08:25 AM.

Powered by vBulletin® Version 3.7.1
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Copyright 1998 - 2014, nV News.