NVIDIA antialiasing mode exploration

I thought it would be interesting to revisit the secret FSAA modes made available with a tweaking program, as they offer much better quality than the ones NVIDIA allows you to select via the control panel aTuner was used to force each FSAA mode. The results speak for themselves, each FSAA mode was correctly enabled, although you maybe notice that the screenshots say that the program says no FSAA is enabled. All screenshots are high quality .png. Screenshots taken with a good old GeForce3 and Detonator 44.03. Screenshots with a "GFFX" suffix succeeding the FSAA mode name were taken on a GeForceFX5900Ultra with Detonator 45.23.

Depending on what card you own, and what driver you use, different FSAA modes will be available to you. GeForce3, GeForce4, or GeForceFX users all have the same FSAA modes available after using tweaker programs, the control panel may or may not offer all modes.

*Important* The 50 series driver(5x.xx) has had some massive changes to FSAA modes. On a GeForceFX, 4x Super Sampling, 4x+9tap, 6x, and 8x have all been removed from the driver. You cannot get any of these modes. The control panel is also misleading. Quincunx has been renamed "2xQ." 12x(also known as 16x in D3D in certain drivers that exposed it in the control panel) is known as "6x" in the control panel. The old 6x is gone. The "real" 8xS is known as "8x" in the control panel. The old 8x is gone, and good riddance. I applaud NVIDIA for replacing old modes which hardly differentiated themselves with higher quality modes. I do not however, condone the misleading naming schemes.

The FSAA modes below use their 40 series driver(4x.xx) naming conventions, as they were taken on 4x.xx drivers

Edge quality(Listed from worst to best)
2x
QCA | QCA GFFX
4x
4x Super Sampling
4xS
6x
8x
4x+9tap | 4x+9tap GFFX
8xS
12x

For comparison, you can find some shots taken on a GeForceFX5900Ultra here. As expected, 2x, QCA, 4x, and 4xS have the same AA quality on a GeForce3 as on a GeForceFX. 6x on a GeForce3 appears to have the same quality as 6xS on a GeForceFX. Ditto for 8x and 8xS. In other words, I would go as far to say these are exactly the same modes, just with different names on different cards. 8xS on a GeForce3 looks noticeably better than 8xS on a GeForceFX, and is clearly not the same mode. 4x+9tap actually looks better on a GeForceFX than it does on a GeForce3. On the GeForce3 shot, you can see some weird anomalies where part of the image is not getting smoothed.

While 6x and 8x provide slightly better IQ at vertical angles, 4xS is superior at horizontal angles. In my opinion, 4xS provides superior edge quality compared to 6x or 8x.

You may notice that 6x and 8x look almost exactly the same. 8xS and 12x are also almost exactly the same. 4x+9tap provides very good edge AA quality, roughly equal to 8xS/12x. Of course, straight edge quality does not tell the whole story. Let's see what each FSAA mode does to a 3d rendered scene to get the whole image quality picture and draw some accurate conclusions.

3d scene quality(Grouped for similarity and worst to best where applicable)
QCA | QCA GFFX
4x+9tap | 4x+9tap GFFX
2x
4x
4xS
6x
8x
12x
4x Super Sampling
8xS

Both QCA and 4x+9tap are blurring the whole scene.
4xS clears up vertical parts of the scene compared to standard 4x or 2x MSAA.
6x and 8x clear up horizontal parts of the scene compared to 2x or 4x. Notice that 8x actually clears up some angles better than 6x does.
12x clears up vertical angles better than 6x or 8x. 12x clears up horizontal angles at least as well as 6x, but not as well as 8x. 12x cleans up vertical angles at least as well as 4xS.
8xS clears up all angles better than or at least as well as 4xS, 6x, or 8x.
4x Sumper Sampling cleans up all angles to the same degree as 8xS.

GeForceFX's QCA appears to be slightly blurrier than GeForce3's QCA. I compared the screenshot on the GeForceFX to the output on my monitor and they were the same. GeForceFX's 4x+9tap is much improved over GeForce3's. Notice that GeForce's 4x+9tap causes some distortions on parts on the image. These distortions do not exist for the GeForceFX.

Conclusion
Well, now you know how each FSAA mode affects not only edge quality, but also scene quality. When factoring in overall quality, straight MS modes are unchanged in their ranking. Other modes fall because they blur the whole scene. And still other modes may gain favor because they clear up textures and allow for finer detail to be seen. It should be noted that QCA supposedly does not blur the scene as much in cards after GeForce3, although it still blurs text. Oddly enough, from what I've seen QCA still still seems to blur on later cards. The claim that QCA stopped blurring the scene came about from evaluating screenshots. But this discrepancy between what is shown on the computer screen and what is shown in screenshots could be due to the post filter that is involved. This analysis is missing a critical component: performance. Image quality is only as good as a game's playability. Performance numbers may be found in this post. Of course, this evaulation answers many questions you may have had about different FSAA modes, it certainly did for me. It also raised a few more questions that may be answered later

If you want to see the sample patterns, download the attachment in this post. You will see that 4xS, 6x, 8x, 8xS, and 12x all are hybrid modes with employ Super Sampling and Multisampling. Hence why these modes clear up textures.

NVIDIA graphic card owners who want different FSAA modes shown here but can't get them through the control panel should try different drivers and/or use a tweaker program like aTuner.

[ragejg]

wow, good read, stealthy... if it wasn't 5:20 AM, I'd have something intelligent to say about that, but all my brain can process right now is "good read"

your scene vs. edge quality comments give me something to think about tho...

way to come up with a fresh topic! I haven't really seen anyone else bringing this up!

[ChrisRay]

You wouldnt happen to have an AA viewer program. Where you can precisely see the pattern and how its done no?

Originally posted by ChrisRay
You wouldnt happen to have an AA viewer program. Where you can precisely see the pattern and how its done no?

Sure, I can test that too.

Ok, I took screenshots of the sample patterns for each FSAA mode. Download the .zip file attached to this post to see them. Each mode is labeled by the file name and is in .png format. The .zip file is less than 5K so file size is defintely not a concern.

[ChrisRay]

Originally posted by StealthHawk
Ok, I took screenshots of the sample patterns for each FSAA mode. Download the .zip file attached to this post to see them. Each mode is labeled by the file name and is in .png format. The .zip file is less than 5K so file size is defintely not a concern.

Excellent thank you. I'm wanting to compare the new modes + Hidden modes to old modes ect

[G6-200]

Nice job StealthHawk!
When I had my Ti-4600 I loved 4xS and especially 8xS modes when playing Morrowind. I am CPU limited in that game anyway so I cranked up the AA.
Makes me wonder WTH nVidia does not expose the RGMS+OGSS modes instead of the crappy ordered grid modes they have.
I suppose since they only work in D3D maybe that's why, they could still have seperate AA controls for D3D and OpenGL though.
I'm really tempted to try out a 5900 to see how playable these modes are on a card with far more bandwidth than my Ti-4600 had.

G6

We should sticky this.

SO IT IS WRITTEN, SO IT IS DONE. (Plus, I want to kill your bandwidth. )

Performance numbers. System tested: 1.33GHz Athlon, 256MB RAM, GeForce3@stock clocks. 3dmark2001 used as it is a good system benchmark.

No FSAA: 6570 (baseline)
2x: 5104 (-22.3%)
QCA: 4641 (-29.4%)
4x: 3399 (-48.3%)
4x+9tap: 2841 (-56.8%)
4xS: 3247 (-50.6%)
4xSS: 2701 (-58.9%)
6x: 2044 (-68.9%)
8x: 1645 (-75.0%)
12x: 1958 (-70.2%)
8xS: 1499 (-77.2%)

edit: Appended to the first post in the thread, "GeForceFX users who want the hidden 4x Super Sampling, 12x, or the "real" 8xS modes can also use them via aTuner."

If you own a GeForceFX I would take a long, hard look at using 12x or the "real" 8xS.

[ChrisRay]

Hey Stealthhawk. What program did you use to get those? I seem to have lost mine.

I am quite curious about my Radeon 8500's different AA mode.

2x 3x 4x 5x 6x and Quality Performance.


Since I finally got my second PC up I could put it back in :P

Originally posted by ChrisRay
Hey Stealthhawk. What program did you use to get those? I seem to have lost mine.

I am quite curious about my Radeon 8500's different AA mode.

2x 3x 4x 5x 6x and Quality Performance.


Since I finally got my second PC up I could put it back in :P

http://www.users.on.net/triforce/d3d...aaviewer-3.zip

[Kruno]

Quit bashing QCX AA. IMO it's the best AA mode ever.

I would take QCX over 16/12/4/6/8/etc...xAA anyday.

You colour blind bastard.