Today, Futuremark is announcing PCMark04, which is a major update to their PCMark performance analysis software for Windows 2000 and Windows XP based personal computers. PCMark04 was crafted as an application based benchmark that can be used to measure the performance of a variety of tasks that are associated with typical home PC use.
The Benchmarking Cycle
The test results from PCMark04 can be sent to an on-line accessible repository, which allows comparisons to be made against other systems with Futuremark's Online ResultBrowser or ORB. The information from the comparisons can then be used to evaluate the benefits of upgrading system components or buying a more powerful PC.
Since PCMark04 measures the performance associated with today's home PCs, Futuremark has defined the minimum system requirements to run PCMark04 as follows:
Intel or AMD compatible processor running at 1.0GHz
128MB of system memory
60MB of disk space (additional 130MB for HDD tests)
DirectX 7 compliant graphics card
Windows 2000 or Windows XP
DirectX 9.0 or higher runtime installed (download)
PCMark04 consists of two versions - PCMark04 Professional and PCMark04 Business Edition. There is also a limitied functionality version that allows you to execute and view the detailed results from the system tests, which make up the overall PCMark score. This version provides a detailed report of your systems hardware and software and also allows you to access the ORB.
BENCHMARK APPROACH
Unlike 3DMark03, which is a specialized and synthetic measurement of 3D game performance, PCMark04 was designed as general purpose application based benchmark. In order to effectively measure application performance, while keeping the size of the benchmark manageable, Futuremark has incorporated freely available public code to simulate workload.
PCMark04 Tests
PCMark04 makes use of multithreading in certain tests to maximize performance and resource utilization. Multithreading is a feature that is used by operating systems to execute different parts of a program, that are referred to as threads, simultaneously. However, a program needs to be specifically designed so that the threads can run at the same time without interfering with each other.
The tests that have been incorporated into PCMark04 were the result of a collaborative effort from the members of Futuremarks Benchmark Development Program (BDP). Current BDP members include AMD, ATI, Creative Labs, Dell, Gateway, Imagination Technologies, Innovision, Intel, Microsoft, NVIDIA, S3 Graphics, Transmeta, and XGI. Additionally, the following members of media have also been part of the same process: Beyond3D, CNET, and Extremetech.
Futuremark maintains a policy that vendors are not allowed to detect a running PCMark04 instance in any of their drivers or software and use that to reduce the workload or alter the behavior.
PCMARK04 TEST SUITE
PCMark04 provides an overall PCMark score and four component scores. The PCMark score is a measure of the overall performance of a PC and is based on the results of the thirteen tests that make up the system test suite. Three pairs of tests are run multithreaded, while the remaining seven tests are run single threaded. The setup for the default run is shown below.
PCMark04 System Test Suite
Components scores are obtained by running the corresponding component test suites:
CPU - a subset of tests from the system test suite that isolate the performance of the CPU
Memory - a collection of tests designed to measure the performance of the memory subsystem
Graphics - a series of 2D and 3D graphics tests used to measure the performance of the graphics subsystem
Hard Disk - four tests that measure hard disk drive performance (Windows XP only)
A detailed description of all the tests in PCMark04 and how the scores are determined can be found in Futuremark's PCMark04 White Paper (1.7MB PDF) and in the documentation that is provided after installing the software. The documentation details Futuremark's efforts in designing PCMark04 to be representative of measuring application performance under Windows 2000 and Windows XP.
A system with an 800MHz CPU will generally receive an overall PCMark score of around 1000, while a high-end system with a 3.2GHz CPU will reach 5000.
SYSTEM & GRAPHICS RESULTS
I successfully ran the PCMark04 system and graphics test suites on two PCs. System A is a better than mid-range PC running with a 2.2GHz Athlon XP and a high-end graphics card (GeForce FX 5950 Ultra). System B is a low-end PC based on a 1.0GHz Pentium 3 with a low-end graphics card (GeForce2 Ti 200).
System A Specifications
AMD Athlon XP 3000+ @ ~2.2GHz
ASUS A7NX8 Deluxe nForce2 Motherboard
1.0GB Corsair PC2700 DDR Memory
IBM 80GB Deskstar 120GXP 7200RPM ATA-100 Hard Disk Drives (2)
Sony Multiscan E540 CRT Monitor - 21-Inch
NVIDIA GeForce FX 5950 Ultra - 256MB - 475MHz/950MHz
NVIDIA Forceware Graphics Driver Version 52.16
Windows XP Professional with Service Pack 1 / DirectX 9.0b
System B Specifications
Intel Pentium 3 1.0GHz
ASUS CUSL2 i815E Motherboard
384MB Mushkin PC150 Memory
Western Digital 40GB Caviar 7200RPM ATA-100 Hard Disk Drive
Sony Multiscan E500 CRT Monitor - 21-Inch
NVIDIA GeForce2 Ti 200 - 64MB - 175MHz/400MHz
NVIDIA Detonator Graphics Driver Version 44.03
Windows XP Professional with Service Pack 1 / DirectX 9.0b
To comply with Futuremark's minimum requirements for running PCMark03, it was necessary to install Microsoft Media Encoder 9 on both systems and DirectX 9.0b on system B. Let's check out the results of the thirteen tests that make up the system test suite.
PCMark04 System Test Results
I'm not sure why the Web Page Rendering test failed on system B, but that caused the overall PCMark04 score not to be computed. However, the individual test results from system A are 2-3 times greater than those from system B, which would put system B's overall score around 1500. That appears to be a respectable score for a PC that was purchased well over two years ago and continues to be used for such tasks as e-mail, web browsing, instant messaging, file transfer, word processing, and MP3 playback on a daily basis.
PCMark04 Polygon Throughput - 8 Light Sources
Being a web site that caters to NVIDIA based graphics cards, we should certainly check out the 2D and 3D graphics test suite. Note that the Video Playback tests are listed in the documentation, but were not available as a test on either system. However, the Video Playback tests are not factored in the PCMark04 Graphics score.
PCMark04 Graphics Test Results
Although the complexity of the 3D graphics test suite is limited when compared to 3DMark03, the multi-texturing fill rate results can be indicative of performance in fill rate hungry games.
PCMark04 Mutitexturing Fill Rate Test
In the muti-texturing fill rate test, system A outperformed system B by 3.3 times, which is similar to the performance between the two systems in Quake 3.
Quake 3 Demo004 - Maximum Quality
CLOSING STATEMENTS
I did experience a few glitches with PCMark04. Although the physical and logical hard drives on system A were reported by the PCMark04 system information module, the hard drive test failed to run. On system B, PCMark04 crashed while exiting the application. I also installed PCMark04 on a third system and the application repeatedly crashed during the audio conversion test. I have been working with Futuremark on these issues.
As shown in the benchmark results from the Pentium 3, today's PC is more than capable of running your basic Windows applications. In this case, PCMark04 will probably not be a useful tool in swaying the decision of a consumer when it comes to purchasing a PC.
However, PCMark04 will be an effective tool for comparing the performance of specific system components that operate under controlled conditions. For example, when the same system is used to compare different brands of memory or hard drives. PCMark04 will also be useful for system builders who specialize in building custom systems to maximize performance in specific tasks. These tasks include applications that are known to heavily stress a PC, which include audio and video editing and playback.
Overall, Futuremark has done an admirable job in developing PCMark04. I've only touched on a few of its capabilities, but the interface is user-friendly, the tests closely mirror the workload created by actual applications, and the results are detailed and repeatable.
