One approach to testing graphics card performance is to use in-game benchmarking, which relies the game engine to play back a recorded demo of actual gameplay. As the demo is played back, without sound and as fast as the computer being tested allows, the benchmark calculates and saves the frame rate as well as the time the frame rate was calculated.

When the benchmark concludes, the collected data is then processed and used to report minimum, maximum, and average frame rate. Other useful statistics can be provided such as the number, or percentage, of frames that fall within a given range of values, similar to the game F.E.A.R. With in-game benchmarking, the deviation from repeated benchmark runs is low and makes it an accurate measure of performance. Since many games do not feature an in-game benchmark, a second approach can be used.

Fraps

The second approach takes more time, but the results and commentary can be of greater value to the reader since they are based on gameplay. The key to gameplay-based benchmarking is to develop a scripted walkthrough that is fairly repeatable. Developing and perfecting a walkthrough requires practice. As with in-game benchmarking, the deviation from run to run needs to be kept to a minimum, which is possible with many games. When the walkthrough is perfected, it can then be used to evaluate gameplay performance. During a walkthrough, frame rate data is captured in real-time by Fraps.

Today's graphics cards are more powerful than ever and need to be pushed to perform at their full potential. Limiting benchmarks to a resolution of 1600x1200 or antialiasing (AA) level of 4X does not provide a true picture of the capabilities of a GPU. My approach to benchmarking has changed over the years and I have turned to reporting a variety of settings that are able to achieve a minimum frame rate near 30 frames per second. Instead of providing one playable setting, multiple playable settings at varying resolutions and levels of antialiasing are provided. The results are shown with bar charts and are ordered from top to bottom by resolution and AA level.

The common benchmark results in the Multi-GPU World Tour 2006 were primarily based on enabling a 4X level of AA and 8X level of anisotropic texture filtering (AF). The number of benchmark results from the twenty games that HardiNFO and NeoSeeker produced at that one setting alone was staggering. Imagine the effort it would take to compare performance, and image quality, using all of the available levels of AA and AF that ATI and NVIDIA provide!

While the texture filtering methods used by ATI and NVIDIA may differ, attempting to perform apples-to-apples comparisons will often result in varying degrees of quality. The same is true for antialiasing, although the 2X and 4X sample patterns that ATI and NVIDIA employ in their current line-up of GPUs are similar. The key benefit of antialiasing is that it makes jagged edges appear smoother.

Antialiasing

The quality of antialiasing is mainly dictated by the method used, such as multisampling, and its sample pattern. A common technique to produce a smoother-looking edge is to slightly alter the color of a pixel by sampling the color of adjacent pixels based on the sampling pattern. The result of antialiasing is illustrated below. Note that the original images were rendered at a resolution of 1024x768 and increased to 2048x1536 using nearest neighbor interpolation, which basically makes each pixel larger.

Non-Antialiased vs. Aliased - (2X Magnified)

No Antialiasing

ATI 4X

NVIDIA 4X

Using the DX9FSAAViewer program, which was developed by Beyond3D forum member Colourless, we are able to examine the sample patterns for the various modes of antialiasing. The first series of patterns are available with a single GPU. A red colored square indicates a sample that applies to geometry, while a green colored square applies to textures. Multiple green colored squares indicate that supersampling is also occurring, such as NVIDIA's 8XS, which is a combination of 4X multisampling and 2X supersampling.

Single-GPU Antialiasing Sample Patterns
ATI 2X	NVIDIA 2X
ATI 4X	NVIDIA 4X
ATI 6X	NVIDIA 8XS

Higher quality methods of antialiasing are possible with two GPUs. ATI refers to their technique as SuperAA while NVIDIA has named theirs SLIAA.

Multi-GPU Antialiasing Sample Patterns
ATI CrossFire 8X	NVIDIA SLI 8X
ATI CrossFire 10X	NVIDIA SLI 16X
ATI CrossFire 12X
ATI CrossFire 14X

The following JavaScript applet demonstrates the effects of the various levels of antialiasing using output from the DX9FSAAViewer program. Clicking the links underneath the image will reveal the output that was produced with the chosen method of antialiasing enabled. Each image is only 1KB in size and you can switch between them in any order for comparisons.

Horizontally Positioned Edge Comparison

ATI Modes:	1X - 2X - 4X - 6X - 8X - 10X - 12X - 14X
NVIDIA Modes:	1X - 2X - 4X - 8XS - SLI 8X - SLI 16X

The effectiveness of antialiasing on a vertically positioned edge is demonstrated below.

Vertical Positioned Edge Comparison

ATI Modes:	1X - 2X - 4X - 6X - 8X - 10X - 12X - 14X
NVIDIA Modes:	1X - 2X - 4X - 8XS - SLI 8X - SLI 16X

Texture Filtering

Texture filtering is an equally important feature that improves image quality and is being covered in Rage3D's article.

Next Page: Low-End System Components