View Full Version : GFFX benchies at 3DCenter: Conservative Aniso?
Hello everyone,
As everyone know, a few weeks ago 3DCenter released a few nVidia-made benchies of the GFFX:
http://www.3dcenter.org/artikel/2002/11-19_b.php
Among them was the very controversial nature score of 40,6, which Radeon 9700 users were able to beat at identical settings.
And today, I've got a quite interesting and simple theory about why that happened.
The 8X AF nVidia is using there isn't their new "agressive" algorithm. It's the old conservative one.
Now, how am I going to justify my bold claims?
Here's a quote ( bold is my own addition ) from http://www.beyond3d.com/previews/nvidia/nv30launch/index.php?p=3
So for benchmarking, if you want a pure apples to apples then set the sliders on conservative, but if you're a user and you're playing you favourite app you can experiment with those sliders and if you find you enjoy the quality with the slider on aggressive then you're going to get more performance out of it.
Now, nVidia is comparing the GFFX to the GF4. From my understanding, the GF4 *cannot* do that AF adaptive algorithm. Or even if it can, not in hardware, and certainly not as efficiently. Else they wouldn't say it's part of their Intellisample technology which is new for the GFFX. If I'm wrong on this, please correct me.
Now, they say it themselves: for an apple to apple comparaison ( obviously with the NV2x ) , you'd have to use conservative. So, unless they aren't doing an apple to apple comparaison, it's conservative AF, not their new algorithm.
Now, something else actually hints at that.
In the Doom 3 benchmark including the GF4 & R300, they aren't using Aniso or AA. Now, for Nature & UT2003, they're using AA & AF. And for the whole % thing in a lot of applications at the beggining of the page, only AA is used; not AF.
How could you explain that? Well, I've got no idea as to why there's no AF in the % area. But for why there's no AF against the R300...
nVidia is not giving us screenshots of their AF agressive implementation. They probably prefer the reviewers to do that so they aren't called liars if quality is, in practice, worse than what they initially shown.
So, the problem here is that they'd obviously get flamed if they compare one of their adaptive settings to ATI's. Maybe they'll try to say their is better, or maybe they'll only say it's faster because its quality is worse. We don't know that yet :(
And until we do know that, it's not a good idea to release agressive AF numbers.
I'd love to get an official word from nVidia about this, but I doubt we'll get any :( Ah well. Probably not even worth asking.
Uttar
You want AF performance?
Set your AF slider to max with trilinear and your LOD to +15. :)
We don't know that yet
exactly. So we can only speculate on this ,until we see
reviews of the Nv30 in hardware sites.
looking by the comments of Anands ,seems Nvidia has a similar
aniso "technique" ,for the sake of helping reviewers to review apple vs apples comparisons.
the good thing is that Nvidia released benchmarks
in the most important games/engines today that will make
the diference of to BUY or not buy.
like UnrealT2003 and Doom3 .
but i think , we will need to wait to see how the retail
cards reviews ,and see how much diference is possible with
more mature drivers.
Originally posted by K.I.L.E.R
You want AF performance?
Set your AF slider to max with trilinear and your LOD to +15. :)
What? Err, shouldn't you put your AF slider to *minimum* with bilinear and your LOD to +15 if you wanted to beat every 3DMark score in the world?
Or maybe I don't quite understand what you say...
Uttar
Bigus Dickus
12-16-02, 01:56 PM
The chipset manufacturer actually being careful to make sure the benchmarks are showing apples to apples, so that they don't unjustly overstate the performance capabilities of their new hot rod product?
That would be a first.
Originally posted by Bigus Dickus
The chipset manufacturer actually being careful to make sure the benchmarks are showing apples to apples, so that they don't unjustly overstate the performance capabilities of their new hot rod product?
That would be a first.
Eh, agreed.
But maybe they're simply doing that because they still got a slight bug in their AF algorithm, or maybe because they want to impress us all suddently with much better performance than expected?
Or maybe I'm being too optimistic. Bad me! Bad!
Uttar
Joe DeFuria
12-16-02, 03:10 PM
Among them was the very controversial nature score of 40,6, which Radeon 9700 users were able to beat at identical settings.
And today, I've got a quite interesting and simple theory about why that happened.
I've got an even simpler, though perhaps boring theory:
NV30 just isn't any faster than the R-300 with AA and ansio enabled.
;)
Could opengl implementation of AF & AA vs DX implementation have any bearing on this??
- Doom3 (OGL) w/o AF & AA
- Nature (DX) w/AF & AA
I've yet to see a reviewer use performance anisotropic filtering in any benchmarks for the 9700. They purposely use quality (slowest) when comparing it to nVidia. I expect they will start comparing the faster methods upon the release of the FX.
No, No. I said AF performance. Disabling AF isn't the point. :p
Originally posted by Uttar
What? Err, shouldn't you put your AF slider to *minimum* with bilinear and your LOD to +15 if you wanted to beat every 3DMark score in the world?
Or maybe I don't quite understand what you say...
Uttar
StealthHawk
12-16-02, 10:54 PM
Originally posted by ChrisW
I've yet to see a reviewer use performance anisotropic filtering in any benchmarks for the 9700. They purposely use quality (slowest) when comparing it to nVidia. I expect they will start comparing the faster methods upon the release of the FX.
because quality uses Trilinear and speed doesn't. it makes sense to me. it's as apples to apples as you can get when comparing gf4 AF to r9700 AF
Originally posted by StealthHawk
because quality uses Trilinear and speed doesn't. it makes sense to me. it's as apples to apples as you can get when comparing gf4 AF to r9700 AF
I'd have to disagree. I think they should compare the two cards with as close as possible image quality settings, not identical settings. 8x AF on one card does not equal 8x AF on the other. Maybe performance AF on the 9700 looks better than quality in the GF4? Maybe 4xFSAA on the GF4 looks better than 6xFSAA on the 9700?
It could just be that the new AF setting portion of the new drivers aren't quite ready for prime-time, and any numbers they are using are estimations for what they are expecting, but they don't dare use it for real yet.
My impression on the whole AF thing, based on what I've read over the last couple years, is that NV engineers felt that their method was the "technically correct" way to do it. While NV held the performance crown with room to spare, they were able to hold off the marketing pukes who were muttering about the performance hit relative to ATI's method. Then the 9700 came along and changed all that, improving ATI's AF quality to boot, and the marketing pukes got enuf of an upper hand to enforce their will. Hence the new settings NV will be debuting with GFX. It will be interesting to see to what degree, if at all, there are teething problems with that new method and how long they last. Certainly ATI has had more experience and iterations to minimize the tradeoffs between speed and quality with their method(s).
StealthHawk
12-17-02, 04:25 AM
Originally posted by ChrisW
I'd have to disagree. I think they should compare the two cards with as close as possible image quality settings, not identical settings. 8x AF on one card does not equal 8x AF on the other. Maybe performance AF on the 9700 looks better than quality in the GF4? Maybe 4xFSAA on the GF4 looks better than 6xFSAA on the 9700?
i don't understand what you're saying :confused:
of course there cannot be "identical settings" as the implementation of AF on the two cards is different. i proposed that Quality AF was used on R9700 in order to get it as close to gf4 quality AF as possible. so i'm really suggesting the same thing that you are.
i can't quite tell if your last few sentences are sarcastic or not, so i'll just address them as literally as possible. no, 8x Performance r9700 looks like 8x gf4.
r9700 has better absolute AF quality because it can use a higher degree than the gf4 can, ie 16x.
no, 6x FSAA AND 4x FSAA on r9700 look better than 4x FSAA on gf4.
To help illustrate my point, read this review:
http://www.anandtech.com/showdoc.html?i=1683&p=14
Clearly, 16x performance AF on the 9700 has much better image quality than 8x on the GF4 and it is a lot faster than 16x quality. Sure it doesn't use trilinear filtering but the overal image quality is still much higher. But you are saying it's not fair to compare 16x performance on the 9700 to 8xAF on the GF4 even though 16x performance has much better image quality than 8x on the GF4 because it is not using trilinear filtering. I'm saying who cares about what settings you have to use, what matters to the gamer is how many fps you can get at whatever level of image quality.
If the GeForce FX has some 4xFSAA method that looks as good as 6xFSAA on the 9700 then would it be fair to compare 4x FSAA on the 9700 to 4xFSAA on the FX? I would say no. Just like it would not be fair to compare 4x quincunx to 4x FSAA on the 9700 even though they are both set to 4x mode.
I owned a GF3 and I now own and R300. I must say, the AF filtering is much higher as well as the FSAA. :)
StealthHawk
12-17-02, 08:03 AM
Originally posted by ChrisW
Clearly, 16x performance AF on the 9700 has much better image quality than 8x on the GF4 and it is a lot faster than 16x quality. Sure it doesn't use trilinear filtering but the overal image quality is still much higher. But you are saying it's not fair to compare 16x performance on the 9700 to 8xAF on the GF4 even though 16x performance has much better image quality than 8x on the GF4 because it is not using trilinear filtering. I'm saying who cares about what settings you have to use, what matters to the gamer is how many fps you can get at whatever level of image quality.
ok, i see what you're saying now. although i still don't see what using 16x has anything to do with "as close as possible image quality settings, not identical settings."
reviewers should just be comparing the two cards both at 8x if you want as close to an apples to apples comparison as possible. additionally that would help the r9700 speed-wise, since that seems to be your primary complaint.
however, the whole point of using AF is to improve image quality. therefore, when comparing maximum imaqe quality between the cards, it only makes sense to use Quality mode on the r9700. if you want to compare speed in similar situations between cards then as i said, you should be using 8x on both, not 8x on one and 16x on another.
Originally posted by StealthHawk
ok, i see what you're saying now. although i still don't see what using 16x has anything to do with "as close as possible image quality settings, not identical settings.
I think his point is that 16x Performance looks about the same as GF4 8x
Of course, you could also use ATI's 8x Quality to compare. But 16x Performance is significantly *faster* than ATI's 8x Quality! So, if you really wanted to see the maximum efficiency of the R300, you'd have to use ATI 16x Performance.
Now, I wouldn't want to comment on that because I don't know if those image quality are really comparable... I'm really bad at seeing differences between high Aniso settings.
I wonder if the NV30 could decide to use trilinear or bilinear using its adaptive algorithm... Probably not, maybe that's for the NV40? Or maybe it simply will never happen because trilinear will become too cheap for a reason or another...
Uttar
legion88
12-17-02, 05:08 PM
Originally posted by Uttar
I think his point is that 16x Performance looks about the same as GF4 8x
Of course, you could also use ATI's 8x Quality to compare. But 16x Performance is significantly *faster* than ATI's 8x Quality! So, if you really wanted to see the maximum efficiency of the R300, you'd have to use ATI 16x Performance.
Now, I wouldn't want to comment on that because I don't know if those image quality are really comparable... I'm really bad at seeing differences between high Aniso settings.
I wonder if the NV30 could decide to use trilinear or bilinear using its adaptive algorithm... Probably not, maybe that's for the NV40? Or maybe it simply will never happen because trilinear will become too cheap for a reason or another...
Uttar
You are also missing an obvious criticism/drawback to what is being suggested here: Objectivity is being removed.
Using the same settings and having a program measure the performance at those same settings is an objective methodology, void of the reviewers' subjectivity or obvious fanboyism (e.g. remember 3dfx days).
By having the reviewers decide whether the images from two different cards have "equivalent" image quality prior to benchmarking adds subjectivity into the scores. There's no guarantee that other reviews can even duplicate the results.
Benchmark scores are the only objective part of a review (or should I say, it is the only part of the review that is expected to be objective).
Joe DeFuria
12-17-02, 05:27 PM
Using the same settings and having a program measure the performance at those same settings is an objective methodology, void of the reviewers' subjectivity or obvious fanboyism...
Yes and no.
The problem is "using the same settings" for a truly objective apples to apples comparison is not always even possible. A few examples:
1) Benchmark Radeon 8500 4X AA vs. GeForce4 4X AA. Same "settings", but not apples to apples. Radeon does supersampling AA, and GeForce multisampling. Radeon's overall image quality will be better.
2) GeForce4 aniso vs. Radeon 8500 Aniso...just the opposite. GeForce's Aniso will be better quality at the "same setting."
And those are rather "obvious" differences in methodology where many web sites just ignore it, and assume the "same setting" is an apples to apples comparison. There are more subtle differences which will be coming up very soon:
3) GeForce4 / FX 4X AA vs. Radeon 9700 4X AA. Both are multisampling, but the Radeon has a spasely sampled grid, vs. the GeForce ordered grid. Even though they are both multisampling, and both the same "setting", Radeon 9700 quality will be better.
4) GeForceFX 6X AA vs. Radeon 9700 6X AA. GeForce FX will incorporate some sort of supersampling + multisampling hybrid. Radeon is multisampling only. Image quality will be different.
In other words, with different methodologies being implemented at "the same settings", it is sometimes very prudent for the reviewer to make subjective comments: how does image quality compare at the "same settings?" It is also a worthwhile exercise to change settings on each card such that the reviewer feels that the overall image quality is as close as possible...then run the benchmarks for comparison. Yes, this does introduce the chance of "fanboyism" getting into the equation. However, a good reviewer would state exactly what settings were used on each card, provide screen shots for reader comparison, which should hopefully limit that kind of thing....
Bigus Dickus
12-17-02, 06:30 PM
IMO, there's no way to remove the subjective nature of card to card comparisons.
Just becaues driver A says "1.2" and driver B says "1.2" doesn't mean that the result of applying those "equivalent" settings will be anything like equivalent. The reviewer must ascertain any apparent IQ differences between allegedly comparable settings, or at more preferrably provide screenshots for readers/consumers to make that decision for themselves.
IMHO, card to card comparisons with today's framerates should be completely subjective, not the other way around. I've outlined a reviewing method in the past that would make this possible. I wish someone would bit the bullet and adopt it.
Sure, there will be some fanboyism involved and they will never match image quality exactly, but I think they could include some screenshots to show how they tried to match image quality. What I'm worried about is if reviewers insist on benchmarking with exact numbers (4xFSAA 2xAF vs. 4xFSAA 2xAF) then one company will change their drivers to reduce image quality for that setting to increase speed. Then the other company will do the same then eventually both cards will look like crap. :(
I play with whatever settings benefit me.
I even taken Bigus Dickus's advice and spread vasoline all over my monitor to have a blur effect in games.
SavagePaladin
12-17-02, 11:36 PM
To the R300 guy...yeah, you'd know.....just like we would. though there is that fact that each successive release of either company is usually faster.
To this current topic....all reviews are subjective. Theres no getting around that. Especially when you have so many manufacturers selling the cards. Chaintech (not saying they do) could have a really crappy 2d implementation and someone would say 'Geforce image quality sucks' even though amother manufacturer could have GREAT IQ.
I could go out and say Athlon XPs run really hot. Well, sure, they do, but supposedly the Thoroughbred Bs aren't even close to the same.
I look at it this way. If you want objective, look at LOTS of reviews. Find the most intelligent ones and stick with em. (I like Anand in this regard, he always sounds like he knows what hes talking about)
If you want a Nvidia fanboy, look somewhere else. I don't say ATIs driver quality sucks. I say their quality sucks at launch.
And somehow people still get offended. Whatever man.
StealthHawk
12-17-02, 11:55 PM
Originally posted by Uttar
I think his point is that 16x Performance looks about the same as GF4 8x
Of course, you could also use ATI's 8x Quality to compare.
the point being that i don't think they look the same. 16x is superior to 8x in pushing mip map boundaries back. 16x Performance is essentially the same on the r9700 as it was on the r8500. and 16x did a "better" job of AF than 8x on the gf4 in pushing back mipmaps. same should still hold true.
vBulletin® v3.7.1, Copyright ©2000-2012, Jelsoft Enterprises Ltd.