NVIDIA GeForce FX 5950 Ultra Preview - Page 2 Of 8
By Mike Chambers - October 28, 2003
3DMARK03
A key feature of today's graphics cards is their support for Microsoft's DirectX 9 Applications Programming Interface (API). While games that make extensive use of DirectX 9 are just beginning to make an appearance, Futuremark's 3DMark03 debuted in February of 2003. At that time 3DMark03 was one of the few legitimate applications that could test and measure the performance of DirectX 9 features.
3DMark03 Professional Edition Main Menu
The attention that the industry is giving DirectX 9 is increasing as we've seen new synthetic benchmarks such as AquaMark3, RightMark3D, ShaderMark and X2 The Rolling Demo. Although each of these benchmarks measures some aspect of DirectX 9 performance, they differ in their approach, as well as the amount of DirectX 9 related workload they generate. For example RightMark3D and ShaderMark allow for tests that are heavily concentrated on DirectX 9. Aquamark was designed around a game engine and as such obtains the best performance possible in certain tests based on the capabilities of a graphics card.
The first test was run using the default 3DMark03 settings. Note that the overall 3DMark03 result is derived from game tests 1 through 4:
The results from the games tests and vertex and pixel shader tests are in average frames per second.
3DMark03 - 1024x768 - No AA - No AF
According to the 3DMark03 documentation, Game Test 4 uses the DirectX 9 based pixel shader version 2.0 to generate the surface of the lake and the sky, while other effects use prior versions of pixel and vertex shaders. The slight edge of the GeForce 5950 Ultra in Game Test 4 may be reflected in the Pixel Shader 2.0 test results, although a casual examination of image quality could reveal otherwise.
For this series of tests, 4X AA and 8X AF were applied via the respective graphics card driver control panels.
3DMark03 - 1024x768 - 4X AA - 8X AF
Update: November 11, 2003
Futuremark announced the availability of a patch, which brings 3DMark03 up to build 340. According to the patch documentation, "Version 3.4.0 has updated prevention of 3DMark specific optimizations in the graphics drivers. This ensures that the benchmark results you get are genuinely comparable to those of other hardware." Futuremark also states that build 340 is currently the only supported version of 3DMark03.
Although NVIDIA has released updated nForce drivers, I have yet to install them. The system I am currently using is bascially the same configuration that was originally used in this preview. The following table is a comparison of the 3DMark03 benchmark results obtained from build 330 and build 340 using the default settings on the GeForce FX 5950 Ultra.
GeForce FX 5950 Ultra - 3DMark03 Version Comparison
The results I obtained at the default settings mirror those that were published today at Nordic Hardware. Unlike the significant differences in the scores from the GeForce FX 5950 Ultra, Nordic Hardware also reveals that the scores from the Radeon 9800 Pro were similar between the two 3DMark03 builds.
Keep in mind that Futuremark has changed program code in 3DMark03 that they felt were driver specific optimizations. At this time Futuremark has not provided any specific information as to what code was changed and why. We also do not know if NVIDIA will provide an official response in regards to Futuremark's actions.
Update: November 16, 2003
NVIDIA (Europe) provided the first response to the 340 patch, which was later retracted, by stating that 3DMark03 disabled NVIDIA's Unified Compiler. Futuremark's Tero Sarkkinen, Executive Vice President of Sales and Marketing, replied that the patch consisted of changes in the order of some shader instructions or the registers 3DMark03 used. Mr. Sarkkinen contends that a GPU compiler should process the old and the new shader code basically with the same performance as was the case on the Radeon 9800 Pro. Mr. Sarkkinen also mentioned that the GeForce FX's loss in 3DMark03 performance might be attributed to application specific optimizations, which Futuemark prohibits.
Derek Perez, NVIDIA's Director of Public Relations, has issued follow-up statements in this editorial at HardOCP. Mr. Perez contends that patch 340 of 3DMark03 is specifically designed to defeat NVIDIA's Unified Compilier, which evaluates shaders and in some cases substitutes hand tuned shaders, but increasingly simply applies the run-time compiler to generate optimal code. He goes on you say that this is yet another example of how 3DMark03 doesn't behave like a game - as a game developer would never specifically try to make their application run poorly or disable optimizations that produce the correct image while delivering better performance.
IMAGE QUALITY
The professional version of 3DMark03 contains an image capture feature that can be useful for comparing image quality. It also allows the tester access to the reference image that was rendered with the DirectX 9 reference rasterizer. The reference rasterizer is a CPU-only renderer and renders every frame as intended by the DirectX specification. The reference images were generated using the default 3DMark03 settings.
Shown below are results from four image quality tests. Each test contains an image from the GeForce FX 5950 and Radeon 9800 Pro with AA and AF disabled and with 4X AA and 8X AF. Frame 1100 was captured from Game 1, frame 219 from Game 2, frame 1643 from Game 3, and frame 805 from Game 4. There was no particular reason as to why I chose these frames other than they seemed to be interesting to compare image quality with.
Below each game test image is an image that allows a side-by-side comparison to be done. The comparison images are from a particular area in the scene and are double their original size, which was done to make it clearer to see differences between them. The image on the left side is from the GeForce FX 5950 Ultra and the image on the right side is from the Radeon 9800 Pro.
I recommend using the freeware graphic viewer Irfanview to examine and compare the images that are available in this preview. Note that images were not altered in any way other than converting them from their native BMP or TGA format to PNG format.
Below are 2X magnifications of the explosion that appears above the left wing. The image on the right side is from the Radeon 9800 Pro, which is of better quality than the GeForce FX 5950 Ultra and also matches the reference image. However, barring this exception, and the brightness of other smoke on the Radeon 9800 Pro, both full size images are very close to the reference image.
The image below illustrates an area from the ceiling that is located left of center and is being rendered properly on the Radeon 9800 Pro. Further investigation reveals that both the GeForce FX 5950 and Radeon 9800 Pro are very similar in quality to one another although neither come close to matching the lighting in the reference image.
In this scene, you'll find differences in the positioning of flames among other minor variations. A difference can be seen in the level of detail of the club, which was closer to the reference image on the GeForce FX 5950 Ultra. The difference becomes even more apparent when 4X AA and 8X AF are applied to the scene.
With the exception of the area shown below, the images from the GeForce FX 5950 and Radeon 9800 Pro come extremely close to matching the reference image. The comparison image is from a section of the background area located in the center of the scene and is noticeably clearer on the Radeon 9800 Pro.
GeForce FX 5950 Ultra Radeon 9800 Pro
This analysis wasn't intended to be a thorough investigation on image quality. But based on the examples presented on this page, the non-AA/AF image quality generated by the GeForce FX 5950 and Radeon 9800 Pro is similar.
