The ideal review would include...

[pelly]

I know Mike discussed this a bit earlier...but I think it still deserves osme attention...

What qualities does the ideal review have?

IE: What games do you want to see and what resolutions? Do you want/need samples of each combo of FSAA and Aniso or are the min and max enough?

Following Mike's lead, I have compiled my first review based upon "real" gaming instead of some pre-determined and structured demo. Using FRAPS, I would make my way to a certain point in the game, measuring fps along the way. Granted, each test will not be exactly the same as the other...However, I feel this method is the most accurate portrayal of how the card reacts in a real-world environment.

I neglected to include any resolution below 1024x768 as I feel the past few generations of video cards are powerful enough to power higher resolutions on nearly any system.

The inclusion of 1024x768 was also important because it represents the typical resolution of 15" and some 17" LCD monitors. With the growing popularity of these monitors, we have to consider them when writing reviews...

Well....I'm expecting some thorough responses...don't let me down!
:D

[Smokey]

Here is what I would like to see.

Games no more than 12 months old
resolutions @
1024*768 - 32bit
1280*960 - 32bit
1280*1024 - 32bit
1600*1200 - 32bit

When seeing what new cards can do, 2+4xAA + min of 4x anistopic filtering is a must, and games (if any) that support both DirectX and OpenGL eq. UT but new games, I would like to see the results run on both APIs.

Now just to add a bit more to the games used, I think it would be good if there were games from different genes used, as not everyone just plays FPS ;)

Basiclly benchmarks have to relate to everyone, like using 2 or even three different systems, as not everyone upgades thier hole system all at once, many upgrade thier video card first and could be using a low end cpu. When looking at benchmarks, everyone should be able to say, ok now if I buy this card, this is what I can expect on my pc with game x at certain settings.

Most of todays setting really only reflect systems that have both the latest card(which is bieng reviewed) and the latest cpu, memory, motherboard etc. To me only a very small group of people would have these systems, so only very few could expect similar results on thier pcs if they upgraded thier video to what was being reviewed.


Sorry for the long post and I hope it all makes sense
:confused: but this is what I think alot of reviews are missing.


Smokey

[pelly]

I think you are 100% correct....It would be great to have a sample of a few different CPU levels to see how the card scales...Unfortunately, that requires a fair amount of resources ( I know I can only afford 1 system ). As a result, we are left looking at work such as Anandtech's GPU scaling article for answers.

The next wave of reviews should be a breath of fresh-air for most. I know Mike is trying to take a new approach to testing cards and utilizing a wide variety of games...

Any non-1st person shooter games in particular?

Thanks for the reply Smokey!

:D

[Smokey]

Here is an idea, if you only have one pc would you be able to underclock it? eg. running an AtlonXP @ 100 fsb?

As for any non fps games, that I cant really help with :( as the only game im playing is The Sum of All Fears, and then raven Shield, but i was thinking of others that play RPGs, flight sims, racing games, RTS games.

Another thing that I forgot to mention earlier, is that when taking screenshots for reviews, the FPS counter should be showing (which is done often on the site :D ) . I hate seeing screenshots with all the bells and whistles and not seeing the FPS counter, I always use the FPS ingame.

Hope this helps


Smokey

[pelly]

My next system will be an AMD-based box and I plan on "unlocking" the CPU. This way, I can keep the same FSB more or less and just raise or lower the multiplier. This way, no setting has the advantage of the higher FSB which would skew the results...

Here is a list of what I think should make the cut:

3dMark2001 SE build 330
Quake3 Arena v1.31
Return to Castle Wolfenstein
Jedi Knight II - Jedi Outcast
Soldier of Fortune II
Warcraft 3
Unreal Tournament G.O.T.Y Edition
Grand Theft Auto 3
Vulpine GLmark


Anything else? Is something there that shouldn't be?

[[Corporal Dan]]

3dMark2001 SE build 330 - NO, pure BS
Quake3 Arena v1.31 - Only 1024x768 and up with everything turned on
Return to Castle Wolfenstein - No. Not really stressful.
Jedi Knight II - Jedi Outcast - OK
Soldier of Fortune II Maybe
Warcraft 3 Not THAT stressful
Unreal Tournament G.O.T.Y Edition No. Too CPU dependent.
Grand Theft Auto 3 Yes. But come up with standard settings (veiwdist, etc)
Vulpine GLmark Too old, not stressful


What should be added is:

Flight sim 2002 with everything maxed (stressful as hell, because it's badly programmed.)

CMR2 (Very stressful game, especially with all the effects on full)

[[Corporal Dan]]

I also forgot to mention commanche 4.

Morrowind too.

"Another thing that I forgot to mention earlier, is that when taking screenshots for reviews, the FPS counter should be showing (which is done often on the site ) . I hate seeing screenshots with all the bells and whistles and not seeing the FPS counter, I always use the FPS ingame." -Good suggestion

[Spectral]

3dmark is a very viable benchmark. Anybody who says it isnt is scoring low on it.

You can see an improvment relative to hardware/software configurations and tweaking, and it directly correlates into real world games.

Why, Dan would it be a bad idea to use it as a bench.. It was DESIGNED as a 3d benchmark.

[[Corporal Dan]]

It's trash

Biased.

And couldn't hope to represent realistic gaming scenarios.

Not to mention it's archaic.

[Spectral]

Biased against what? If I made a 3d accelerator myself and benched it on 3dmark the scores would directly relate to how well it performed... So, how is that garbage?

And you mean to tell me that a card that scores a 5,000 compared to a card that scores 11,000 means absolutely nothing?

Couldnt hope to resemble gaming scenarios? I guarantee you the person who scores 11,000 over the person who scores 5,000 will have a better framerate and be able to have higher detail in their, "real world" games. Please, prove me wrong.

[pelly]

For better or for worse, 3dMark will always be used as a benchmark. Everyone knows 3dMark and they can see a difference when new hardware is added or tweaks are made. That "magic" number at the end provides a quick and simple estimate of the card's overall performance...

There are definitely more accurate benchmarks out there....

As for WarCraft 3, I can assure you that the game IS stressfull.....though I can't begin to tell you why...I really can't see what is so taxing about this game....though I have seen how it hammers systems when the eye candy is turned on...very strange...

GTA3 should be used w/ view distance set to max....anything lower inhibits the playability of the game...

I'm a bit confused how you can include Q3 but not RTCW....Surely there is more detail in RTCW than in Q3....So seeing it at high res and full eye-candy should stress the card enough...

I'd like to see Morrowind included.....though I am not much of a fan of Comanche 4....

:D

[[Corporal Dan]]

1) Nvidia optimized -> Biased

2) "And you mean to tell me that a card that scores a 5,000 compared to a card that scores 11,000 means absolutely nothing?" It does mean simething, but it isn't easily quantifiable.

3) "Couldnt hope to resemble gaming scenarios?" Yes. Name three games that use point sprites, for example. And the gaming situations depicted are unrealistic.
It uses maxfx engine, right? Which means unrealistically built 'maps'... Weird polycounts and all. Not to mention that certain level geometries are unobtainable with the mapbuilding techniques that are used. Also, the textures are really lowres (on the environments). Another thing is that there are very few special maps (texture maps) used. No bumpmaps, no refraction maps, etc. Particles play a very little part as well. These are not situations seen in typical games.

4) "I guarantee you the person who scores 11,000 over the person who scores 5,000 will have a better framerate and be able to have higher detail in their, "real world" games. Please, prove me wrong."

Simple. Most feature tests are done exclusively. A pixelshader test has pretty much just the pixel shaders there. The Dot3 test has a handful of polygons, a lightsource or two, and the maps. The pointsprite test has only pointsprites. The vertexshader test uses only the vertex shader engine.

A card may perform EXCELLENTLY on these tests. But in real life scenarios, where any number of the elements of the tests are involved, the card may not do so well.

Lets say I have an Nvidia CorpDanXP 5000+ video card, compared with an ATI SpectralPower Extreme Maxx.

While the CorporalDanXP card, with 120003dmarks, totally owns the Spectralpower card, which has 80003dmarks in the individual tests, the SpectralPower card fares much better under conditions where the elements are mixed because of it's more balanced architecture.

Get the idea? Because the CorporalDanXP card does SO WELL in the individual (let's call them shader) tests, it gets a really high number of 3dmarks. The Spectralpower card, which perfoms fairly badly in the individual shader tests, gets a lower mark.

When entering a game where all the shader techs are mixed, however, the spectralpower card might own the corporaldanxp card.

So, yes. 3dmark is not a good benchmarking tool.

[[Corporal Dan]]

Originally posted by pelly
For better or for worse, 3dMark will always be used as a benchmark. Everyone knows 3dMark and they can see a difference when new hardware is added or tweaks are made. That "magic" number at the end provides a quick and simple estimate of the card's overall performance...

There are definitely more accurate benchmarks out there....

As for WarCraft 3, I can assure you that the game IS stressfull.....though I can't begin to tell you why...I really can't see what is so taxing about this game....though I have seen how it hammers systems when the eye candy is turned on...very strange...

GTA3 should be used w/ view distance set to max....anything lower inhibits the playability of the game...

I'm a bit confused how you can include Q3 but not RTCW....Surely there is more detail in RTCW than in Q3....So seeing it at high res and full eye-candy should stress the card enough...

I'd like to see Morrowind included.....though I am not much of a fan of Comanche 4....

:D

RTCW actually runs FASTER than q3 on my box.

What we may need to do is make our own q3 stressdemo. The Four demo is good, standardized, but by no means stressful.

GTA3: I play with the visibility at 3 bars and it's quite alright ...

WC3? Dunno. I havent experimented with settings, but it looks like lowres textures + low polycount. Dunno why it's so stressful. I assumed because my box is old, that it was the reason wc3 ran a little slow...

And about JK2, RTCW, SOF2 and Q3...

dont you think thats too many quake3 engine games?

Jk2 is stressful, and EVERYONE knows what a score in a q3demo means. that's why they're good ideas for benchmarks.

I got another idea: Ut2003 beta :)

[Spectral]

How do your examples directly relate to videocards that are out now?

My Ti4600 destroys the Radeon 8500 in 3dmark, and it destroys it in real world games.

The 8500 destroys a GeForce DDR in 3dmark, and it destroys it in real world games.

Where is the real world example of your theory here?
I understand the point you are trying to make. And Pelly pretty much sums up my thoughts... 3dmark is for an overall score... It takes into account many things, such as fillrate, pixel shading, vertex shading, bump mapping, cpu limited, fill rate limited etc etc.

It is way more of an overall benchmark, than a specialized benchmark. So, your examples arent really valid... Perhaps if 3dmark only benchmarked the cards say.. memory bandwith. Then a Parhelia would score 11,000 and a GeForce Ti4600 5,000.. Then you could say it has nothing to do with real gaming scenarios.

And yes, all graphic engines are different. It still is a very valid benchmark.

And dont for one second tell me that if you were given a choice of taking a Videocard that scored 20,000 3dmarks over the card that you currently have that you wouldnt take it... Cause if you say you wouldnt, You are a liar.

[[Corporal Dan]]

Spectral:

Think of tech demos.

While they are impressive, there is no way youll see anything like that in games.

I know for a fact that NVIDIA demos are maxxed out. They use so many polygons and maps, that if you added a few extra polys, it would KILL the framerate.

Think of individual shader tests like this, and youll get an idea why they arent a good perftest

[[Corporal Dan]]

Originally posted by Spectral
How do your examples directly relate to videocards that are out now?

My Ti4600 destroys the Radeon 8500 in 3dmark, and it destroys it in real world games.

The 8500 destroys a GeForce DDR in 3dmark, and it destroys it in real world games.

Where is the real world example of your theory here?
I understand the point you are trying to make. And Pelly pretty much sums up my thoughts... 3dmark is for an overall score... It takes into account many things, such as fillrate, pixel shading, vertex shading, bump mapping, cpu limited, fill rate limited etc etc.

It is way more of an overall benchmark, than a specialized benchmark. So, your examples arent really valid... Perhaps if 3dmark only benchmarked the cards say.. memory bandwith. Then a Parhelia would score 11,000 and a GeForce Ti4600 5,000.. Then you could say it has nothing to do with real gaming scenarios.

And yes, all graphic engines are different. It still is a very valid benchmark.

And dont for one second tell me that if you were given a choice of taking a Videocard that scored 20,000 3dmarks over the card that you currently have that you wouldnt take it... Cause if you say you wouldnt, You are a liar.

Did you even read the post and try to understand the example I gave?

Oh, and my current card is a geforce2, so I would take ANY newer card (except an mx)

[Spectral]

You can bet that Id much rather have a card that could churn out 200fps on their most stressful techdemo than a card that could only churn out 20fps.... Now, I assume that Nvidia tech demos would obviously be optimized for Nvidia cards. But, I have heard that the 9700 destroys the Ti4600 in some of Nvidias tech demos, and scores higher in 3dmark 2k1.

Whether or not the card is STABLE, with high image quality is something that has nothing to do with those benchmarks. That would be a driver issue that ATI would have to sort out, but I can tell you right now.. That if the drivers for the 9700 are stable, I will definetely be getting that card (Thanks to the compusa replacement policy :))

Same goes with 3dmark. Isnt that the point of a benchmark?
Like I said before... I agree with what you are saying to a point, but to say that 3dmark is useless garbage just isnt quite factually right. Those scores do add up to performance on most, if not all 3d engines that are used in games today.

[Spectral]

Did you even read the post and try to understand the example I gave?

Your examples do nothing to validate your claim. Tell me how those examples make 3dmark a piece of garbage. That is entirely the point here.

3dmark always scores the highest real world performing cards highest... And it always has. Which cards score the highest right now? The Ti4600s... which cards have the highest performance in real world games right now? The Ti4600s... Tell me how 3dmark is incorrect then.

When the Radeon 9700 comes out it will have the highest performance in games, and in 3dmark... So, how is 3dmark not a valid comparison? How could you say, "Well, the GeSpectral 10x scores 50k in 3dmark, and the GeCorpDan 10x scores 10k, but those scores mean absolutely nothing and I prefer the CorpDan."


You couldnt say that, because 3dmark benches pretty much every viable piece of the 3d accelerator for DirectX 8.

[[Corporal Dan]]

Check back here tomorrow

I'm too tired to type right now LOL :o

[Spectral]

I am too :) cept Im stuck here at work with nothing better to do but post on messageboards. My job is extremely extremely boring.

[pelly]

I definitely think that we are left to use too many Q3-based games as benchmarks...The trouble is, these are typically the most popular games out there...Nobody wants to see benchmarks from a game they'll never play...

This is the trouble...Though once UT2003 and Doom3 come out...we'll have an easier time b/c they will provide black and white depictions of how a card performs ( very taxing on GPU ).

As far as this whole 3dMark issue...couldn't we just sit back and say:

If card A scores higher than card B is 3dMark, chances are card A will likely score higher than card B in most games.

You are both right in a sense here and all I see is a viscious cycle that could go on forever....Oh well, at least it's amusing to read.

:D

[[Corporal Dan]]

LOL what's ur job?

[Spectral]

Xerox Operations/IS for the northern (Detroit suburbia) Michigan DMC hospitals.... And, its a very boring job.

[druga runda]

Hmmm.

What I would want is.

1. CPU scaling - The processor at 100 FSB / 133 FSB/ max OC FSB.
- for 2 games at least or for 3D mark + a game or two
2. 3Dmark regular - 1024 x768
3. All games benchmarked at 1024 x 768 to give an overview of general performance
4. everything tested @ 32 bit color
5. games :
*Serious Sam II
*Morrowind
*RCTW or Jedi II (just one of them - actually the one that normally gives lower scores overall)
*Grand Prix 4
*some flight sim for that audience
*UT2003
*Warcraft 3
*Giants
* - add in Q3 for good measure with the past

- maybe add a few other genres to see how the games perform in one or the other type of display.

Out of these all pick two again and put them trough the paces - one would be the most advanced - like UT2003 and the other would be Most popular (at least for benchmarking on the other sites) - while that is debatable pick a OGL game if UT2003 is DX - or so, and do from 640 x 480 to 1600x 1200.

Put those two games trough AA tests as well. Perhaps all AA levels @ 800x600 and onwards to 1600 x 1200. - the same two games for CPU scaling at "normal" 1024x768 - or 640 x480 if it shows the change better. -and some CPU scaling + most useful/stressful AA setting

So my methodology - if I was a games tester would be -

1. do regular 3DMark thing
2. Pick two games and do it all on them. One DX and one OGL while trying that one of them is most advanced at the time and the other most popular (but from other stable ;) )

3. Take a bunch of games - even more than the ones that I listed above - that cover most genres - and test the cards only at one - most popular so to speak - setting. To check games compatibilty and relative performance accross genres instead of seeing the same thing with a couple of games over and over again.

4 If you could maybe you can pick an older game that is still popular like CS and do max AA max Anisotropy -and all other settings max, and see what you come out with - or the same with Q3 - because the border has moved now to that engine - assumiong that all the games before can be played at MAX settings with no problems.

That would be a good roundup. One review to bind 'em all one to rule them all. ;)

[pelly]

My first review is out...tell me what you think....

I tried to cover the majority of what we talked about in here....

http://www.nvnews.net/reviews/xfx_gf4_ti4200/page_1.shtml

:D