Since year 2000, Massive Development has been part of Jowood Software Productions. Massive, which is headquartered in Mannheim, Germany, is basically a developing studio and are known for AquaNox and its sequel, AquaNox 2: Revelation. The technology behind the Aquanox series of games is the Krass™ Engine. Of course Massive Development isn't only developing games. AquaMark 2 was their first benchmark based on DirectX 8 technology, which was only released to professional testers and hardware review websites.
This year Massive Development brings us the AquaMark3 benchmark, which makes use of new capabilities in Microsoft's DirectX 9 Application Programming Interface (API). While there are currently a limited number of games that use DirectX 9, AquaMark3 contains new graphics technology that will be included in the upcoming generation of games. During the course of this preview, you will see a heavy use of pixel and vertex shaders, ranging from the older DirectX 8 formats (PS 1.1, 1.4 and VS 1.1) to the newest and more complex versions (PS 2.0 and VS 2.0) available in DirectX 9.
Note that AquaMark3 is based on a real world game engine. What this means is that the developers decided on the shader version to use to accomplish a task. Although a specific effect could be done with PS 2.0 does not mean that PS 2.0 was actually used. The developer made a decision, based on a variety of factors, on the shader version that was used.
PACKAGING AND INSTALLATION
The AquaMark3 interface is well designed and resembles what you would find in an actual game. The interface operates in full screen mode and consists of a menu driven navigation system.
AquaMark3 Splash Screen
Installing AquaMark3 was straightforward. One thing that needs to be taken care of is your license. If you downloaded the free version of AquaMark3 Benchmark, then you do not need to unlock the software. There are two ways to unlock the application. Either copy the attached license file into the AquaMark3 directory or inside the benchmark by selecting the "Register AquaMark3" option.
AquaMark3 Main Menu
REQUIREMENTS
The hardware and software requirements for AquaMark3 are as follows:
Minimum Requirements
System: Windows 98, Windows ME, Windows 2000, Windows XP, DirectX 9
AquaMark3 Professional ($9.95 / €9.95) - All features from Basic
plus:
Detailed feature-settings
Advanced Measurement for single run tests
Additional special features for DX 9 cardtests
Pixelrate measurements
Automated screenshot function
Technical support for AquaMark3 as well as for ARC
AquaMark3 Professional Plus ($19.95 / €19.95) - All features from
Professional license plus:
Automated measurement functionality (multi-run)
Commandline measurement functionality (unlimited number of user specific
settings)
AquaMark3 Commercial ($39.95 / €39.95) - All features from Professional
Plus license plus:
Right to publish results for commercial use
AquaMark3 Commercial Plus ($199.95 / €199.95) - All features
from Commercial Plus license plus:
Ten simultaneous installations per license
FEATURES
AquaMark3 Score Measurement Mode (AquaMark TRISCORE™) -
Available for all license types
This mode runs the benchmark using default settings: 1024x768, 32-Bit Color, No antialiasng, 4X anisotropic filtering and maximum details. Together there are
nine chapters that AquaMark3 goes through. If you have a basic license, this is the mode you have to use in order to perform a
benchmark run. You must complete all nine chapters to get a final score.
After completing the benchmark, a TRISCORE™ will be computed and
you will be able to upload the results to the ARC database. We will talk about ARC™ later on in this preview.
AquaMark3 TRISCORE™
AquaMark3 TRISCORE™ - Computed
Let's explain the TRISCORE™ Measuring System. AquaMark3 final mark
consists of three independent scores: Graphics, CPU and Overall System
Performance. The total score is based on the number of frames that were
rendered during the benchmark run. Note that the graphics score is also infleuenced by the processor and memory. However, the graphics score does not contribute
to the CPU score.
AquaMark3 Benchmark
Outline
Clicking the "More" button will take you to an outline of
benchmarks that were tested along with average Frames Per Second. Here
is a list of chapters that are available in AquaMark3:
Dynamic Occlusion Culling
High Particle Count
Masked Environment Mapping
Large Scale Vegetation Rendering
Large Scale Terrain Rendering
Vertex And Pixel Lightning
3D Volumetric Fog
Complex Multimaterial Shader
Massive Overdraw
Stay tuned for a more detailed overview of each chapter.
Advanced Measurement (single run - not available with Basic license,
and multi run - not available with Professional license)
AquaMark3 Advanced Measurement
AquaMark3 Advanced Measurement
- Options
This feature allows you to configure your most important benchmark settings
such as:
Enable Sound: Off or On
Color Depth: 16 or 32
Resolutions: 640x480, 800x600, 1024x768, 1280x1024 and 1600x1200,
Full Scene Antialiasing Mode: Off, Non-Maskable, 2x, 4x (Depending
on the hardware vendor)
Full Scene Antialiasing Quality: Levels 1, 2, 3 and 4 (Albeit FSAA
Modes can be coupled with FSAA Quality, current DirectX 9 drivers do
not expose that feature. Thus you can only set FSAA Quality with Non-Maskable
FSAA Mode)
Anisotropy Quality: Off (Trilinear Filtering), 2x, 4x and 8x (Depending
on the hardware vendor)
Level of Detail: Very Low, Low, Medium, High and Very High (Those
options are also available through Options->Choose Preset where
you can set predefined settings)
AquaMark3 Advanced Measurement
- Options
Being still in Advanced Measurement section, you will spot few checkboxes
(If you have a Professional Plus license that is). The above benchmark
settings can be tested with one (single pass) value or more (multi pass)
values. For example, ticking off the checkboxes next to Color Depth and
Resolution modes will create a multi pass benchmark where all resolutions
and all color depths will be tested - a very handy feature for people
that do not have a lot of time on their hands.
Automated Measurement (via command line) - not available with Basic
and Professional licenses
Another great addition to AquaMark3 benchmarking tools is the support
of batch processing. You can run batch files after running an Advanced
Measurement benchmark. An output file will be then put in your “My
Documents/AquaMark3” folder. From there you can edit it to your
taste, add jobs and perform multiple benchmarks. In order to start the
benchmark with your own script, you must execute the following command
from Windows Command Prompt:
AquaMark3.exe –bench <batch config file>
Configurable Demo Mode -
Available for all license types
AquaMark3 Custom Detail Effects
Demo mode is basically a cut-scene with real-time music but without
any score computation. It is configurable through Options menu.
Custom Measurement - Not available with Basic
license
AquaMark3 Custom Settings Measurement
This measurement allows you for custom benchmarking with settings specified
in the Options menu.
AquaMark3 Custom General Options
Here you have similar settings that are available in Advanced Measurement
menu. Additionally you can set Vsync, Triple Buffering and Hyper-Threading
Support.
AquaMark3 Custom Lightning Effect
Options
For low-end systems, this is great feature. Setting effects helps improve
the performance.
AquaMark3 Custom Detail Effects
A lot of details can be set within this menu. If you are not familiar
with those options, you can easily load a preset and you are ready to
go.
SPECIAL FEATURES
Supplementary to the above AquaMark3 benchmark features, Massive Development
has implemented three more tests (OVIST™, SVIST™, PIXPM™)
that have yet to be introduced in any benchmark. Although they render
weird looking scenes, they are very significant while comparing images
from two different hardware vendors.
AquaMark3 ASGRA™ (Automated Screen Grabber) - Not available
with Basic license
AquaMark3 Special Features - ASGRA™
Forget FRAPS and other techniques of capturing screenshots. AquaMark3
is equipped with the right tool to show off those lovely images. Whether
you want to compare Antialiasing modes or detailed textures, this is
a great addition to the benchmark. How does it work? It's very simple.
Just go into Special Features menu, set the duration of time (in frames)
between each screenshot, and hit the "Start Screen Capturing" button.
If you feel you need to capture five different frames, command line functionality
comes into the rescue:
This is one of the Special Feature Tests that exposes the heavy use
of Particles. What happens here is the scene is rendered the same way
as in normal benchmark. In other words, OVIST™ uses the same shaders,
but additional post pixel colorization is done to show different
stages of Overdrawing.
AquaMark3 Special Features - OVIST™ In
Action
Here is another screenshot from the OVIST™ test which lays out
a better picture of what is happening. For reference, this is an explosion.
AquaMark
SVIST™ (Shader Visualization Technique, requires DirectX
9 hardware) - Not available with Basic License
AquaMark3 Special Features - SVIST™
Not every polygon is drawn with a pixel shader. The SVIST™ test
is a shader visualization technique that shows how much of a pixel shader
is used in a rendered scene. Just like in OVIST™, post pixel
colorization is applied to create different levels of shader complexity.
Again, the pixels are colored without altering the ordinal shader. It's
a painstaking and slow task, however it comes with a fruitful effect.
AquaMark3 Special Features - SVIST™ In
Action
There are benchmarks out there who i.e. implement trivial PS 2.0
for particles which is insane. Particles are typically rendered without
any PS at all. With SVIST™ this is visualized for you. If you
change the advanced settings you will notice that the SVIST™ visuals
will change, which reflects that the engine uses different code path
for different situations. Besides we are able to tell you with SVIST™ the
exact amount (absolute, relative and in percentage) of pixels drawn
through a specific PS (or even simple texturestages like the particles).
AquaMark3 Special Features -
SVIST™ In Action
SVIST™ is very fine, sophisticated and easy to use feature.
Here is what Alexander Jorias, Managing Director of Massive Development
had
to say about it:
We are convinced that SVIST™ is one of the coolest features
of AM3, however we are wondering if people will get the point about
it because you have to think about it before you see it's beauty.
We hope that there are enough smart guys out there that will tell
the others about it.
AquaMark PIXPM™ (Pixel Performance Measurement) - Not available
with Basic License
Last but not least is the PIXPM™ test. This is a fillrate benchmark
that utilizes pixel shaders. It consists of two separate tests. In the
first pass, an overdraw is created for each frame and number of drawn
pixels are counted. The second pass takes those results and calculates pixels
per second.
AquaMark3 Special Features - PIXPM™
AquaMark3 Special Features - PIXPM™
AquaMark3 Special Features - PIXPM™ In
Action
This is a screenshot from the first pass of PIXPM™. We did not
complete this test since it would take a lot of time which
we did not have. An update will be made in couple of days with the scores.
Even with low resolution and low details PIXPM™ crawled,
and here is why:
Q: What is PIXPM™ ungodly slow?
A: With PIXPM™ you are able to benchmark the real world
fillrate of a graphics card precisely for the 1st time ever! it is
a very
complex task to give a precise fillrate score under real world conditions.
You have to take the overdraw into account. So in the 1st run we
count the sum of all(!) drawn pixels inclusive the overdrawn ones.
this may take 45 minutes on a current state of the art system. In
the second run we do a realtime run with the same engine setup. Now
we are able to calculate the real world fillrate. Like SVIST™ the
PIXPM™ technique is very smart and gives you as a user much
more transparency about the benchmark and your system then ever before.
BENCHMARKING TECHNIQUES
AquaMark3 Benchmark is designed on a state of the art, very sophisticated
Krass™ Engine
which exploits the new DirectX 9 API features:
The DirectX9 API is used completely for all rendering tasks in
the benchmark. Considering the features that are new in DirectX9
compared to the previous version, we use mainly the new shader models
(pixel and vertex shader 2.0). We don't use vendor specific (2.x)
or not-yet-hardware accelerated (3.0) shader versions.
-Ingo Frick, Technical Director of Massive Development
In addition to improved shaders, there is a new Antialiasing setup.
FSAA can now be applied along with its subroutine: FSAA quality level.
Although mixing those settings is not exposed in current drivers, it
is possible with the new DirectX 9. Although AquaMark3 Benchmark is designed
for the current and future hardware, it is still backward compatible
with older graphic cards based on DirectX 8 and DirectX 7 technology.
The Krass™ Engine allows for a lot of application level tweaking,
making it very versatile for all kinds of systems.
Fore reference, AM3 Benchmark can use a combination of 193 vertex shaders
(VS 1.1 and VS 2.0), 39 pixel shaders (PS 1.1, PS 1.4 and PS 2.0) and
use a variety of Floating Point Precisions ranging from FP64 to FP128
(High internal precision used by NVIDIA's NV3x).
AquaMark3 Benchmark takes nine commonly used graphical environments
and incorporates them into chapters (Cut-scenes). Each chapter contains
its own climate, where different technology is applied and exposed
to the observer.
Dynamic Occlusion Culling
AquaMark3 - Dynamic Occlusion
Culling
Active
This is the first chapter of the benchmark. After cruising the canyon,
a large underwater city unveils.
AquaMark3 - Dynamic Occlusion
Culling
Inactive
What Occlusion Culling does is it detects potential objects that need
to be rendered. Each object has to be tested against any blocked / hidden
(occluded) object. Before such object can be rendered, its shape has
to be tested for occlusion in the depth buffer. Objects are then tested
for visibility with regards to the terrain, in our case hills and canyons.
Dynamic Occlusion Culling deactivates when previously blocked objects
become visible.
While this is a CPU based task, it is very eloquent feature for improving
overall GPU performance.
High Particle Count
AquaMark3 - High Particle Count
I'm sure you've been exposed to Particle Effects of some sort. Whether
it was a firing weapon or a burning object. This is a commonly used technique
for creating any sort of smoke, fire, explosion or rain effects (Of course
there are many others, but this is just to give you an idea). In this
benchmark, your video card will be stressed for high particle count.
Such effects are created from simple 2D textured quads with variation
in direction, size and color.
Masked Environment Mapping
AquaMark3 - Masked Environment
Mapping
Environment Mapping (E.M) is an old technique though improved and perfected.
Specular lightning effects and reflections are the main outcomes of this
benchmark. Transparency combined with E.M produces very complex effects,
like those seen on the ships above. It's hard to capture such effect,
but its beauty is heavily visible in AquaMark3 - the real-time benchmark.
Large Scale Vegetation Rendering
AquaMark3 - Large Scale Vegetation
Rendering
AquaMark3 Benchmark uses its own plant rendering system for
underwater environment. Its efficiency can generate huge amounts of plants,
plankton and other sea-type surroundings. On the above screenshot you
can see hundreds of plants growing on the seabed. Each plant is individually animated
for more life-like effect. What Massive Development has utilized for
this chapter is a Hardware Driven Geometry Instancing Concept. As
a result of this technique, only four tasks need to be processed by the
CPU: plants size, position, rotation and color. This chapter delivers
highest polygon count throughout entire AquaMark3 Benchmark.
Large Scale Terrain Rendering
AquaMark3 - Large Scale Terrain
Rendering
Massive Development's Krass™ Engine allows for very broad viewing
distances. Modern graphics cards should have no major problem coping
with this benchmark.
AquaMark3 - Height Map
Quadtree based cLOD System
The terrain is produced with a height map which covers the
entire underwater world. A cLOD system (continuous Level Of Detail) is
then used to extract the geometry from the terrain. Additionally, the
landscape is built on two types of materials: sand and rock - which are
then colored accordingly.
Vertex And Pixel Lightning
AquaMark3 - Vertex And Pixel Lightning
Vertex and Pixel lightning is a very cool technique which adds life-like
looking lightning effects to the gaming environment. This approach calculates
the "x" amount of light (from the source) that needs to reach
a vertex / pixel. Then those vertecies / pixels lit up. As the name implies,
this benchmark stresses both vertex and pixel shaders.
3D Volumetric Fog
AquaMark3 - 3D Volumetric Fog
Volumetric Fog is an atmospheric type effect used to dim objects within
a beam of light. This type of fog effect is restricted. In other words
it must fit within a specified volume. The 3D Volumetric Fog can be seen
throughout the whole AquaMark3 Benchmark. It's a nice addition
to the whole underwater environment.
Complex Multimaterial Shader
AquaMark3 - Complex Multimaterial
Shader
In this benchmark you will see objects where Multimaterial shaders are
applied. Creating sophisticated material shaders gives very realistic
effects. Take a look at the above screenshot.
Massive Overdraw
AquaMark3 - Massive Overdraw
This is the last chapter of the AquaMark3 Benchmark. As mentioned earlier,
particles can be used for creation of fantastic looking explosions. In
this chapter, you will be exposed to just that: smoke, explosions and
missile trails. Your video card will have to render heavy loads of particles
which is the main purpose of this test.
PERFORMANCE TESTS
Test bed
Abit N7F-S 2.0
Barton 2800+ @ 2.08 GHz
TwinX 512MB Corsair 3200LLPT (dual-channel)
PixelView GeForce FX 5900 (400 MHz core / 425 MHz memory)
Beta Detonator 51.75 Drivers
Windows XP with SP1
Here are some results using the beta Detonator 51.75 Drivers. We did
not have enough time to complete more detailed benchmarks with this set,
but I hope this gives you a picture of what is happening. Detonators
44.33 were about 5-10 FPS slower depending on the chapter.
The first result is a TRISCORE benchmark with
default settings: 1024x768x32bit 0AA 4AF, highest quality settings.
Overall performance during this test was satisfying. The rest of
benchmarks were done using the Advanced Measurement feature. That concludes
our
performance section since this is only a benchmark preview and
not a driver / hardware rant.
ARC™ OVERVIEW
ARC™ aka AquaMark Result Comparator
is an online database designed for submission of AquaMark3 Benchmark
scores.
AquaMark3 - ARC™ Query
Page
ARC™ gives you more options and
tools for comparison of your benchmark results than any other online
benchmark database tool. Just have a
look at the many ways you can filter and set comparison options.
It is the
most powerful online database for benchmarks currently available
Searching and comparing results is easier than with any other software
AquaMark3 - ARC™ Result
Page
You will need to register a username in order to use the ARC™system,
or to access the AquaMark forums. You only have to register once, both
systems are integrated for easier access and navigation.
Things you can do inside ARC™:
View / Edit Measurements
Navigate Measurements
Delete Measurements
Compare Measurements
Make your scores available to the public
AquaMark3 - ARC™ Detailed
Comparison
Detailed comparison is also available. It's a three column page that
lists both yours and the comparative measurements.
CHEATING AND CONCLUSION
Statement of Massive Development regarding AquaMark3 / IHV
cooperation & cheating
attempts
As a developer of interactive high-end 3D gaming applications and benchmarks
we obviously have to collect in-depth Know-How about a specific hardware
from it's manufacturer.
This type of cooperation is common practice for all major developers.
It serves the sole purpose to utilise the latest technologies in the
correct way and thus giving the customer the best gaming and performance
experience possible.
You may decide to support a specific hardware platform or a specific
hardware feature by a game application. However this not an option for
any serious benchmarking application.
While developing AquaMark3 we take precautions in a way so that all
IHV's (Independent Hardware Vendors) with a vital interest in AquaMark3
are treated equally. So there is no reason for those vendors to conduct
any cheating attempts on AquaMark3.
Once a benchmark is released there are obviously multiple possibilities
to cheat its results (both for any IHV and any user). We took precautions
to prevent and detect cheating attempts in this post release phase and
during the lifetime of AquaMark3. Unfortunately we can not prevent this
in all cases by the application itself.
The most powerful anti cheating tool will be the AquaMark Result Comparator
(ARC), the online database and the AquaMark3 forum. We will immediately
inform all customers about our evaluation of the latest cheating attempts
(if they should happen).
As a developer of high end 3D gaming applications we will of course
be very specific and accurate when we have to judge if a specific optimization
is of general interest or if it only targets AquaMark3 in a questionable
way.
We are always open to discuss our policy with interested users in this
forum.
Alexander Jorias / Managing Director Massive Development
Ingo Frick / Technical Director Massive Development
Based on information
from Alexander Jorias and Ingo Frick from Massive Development AquaMark3
does not contain any vendor-specific
codepaths. However, this does not mean a driver cannot modify a shader
or lower filtering quality to artificially increase the performance of
a graphics card. There are many "optimizations" that simply
do not depend on vendor specific codepaths, but can be performed through
driver routines.
Although I regret we haven't had enough time to test the Image Quality
with AquaMark3 and Detonator 51.75, you can expect an update
very soon. All I can say right now is that older drivers have a far better
IQ than the upcoming release of Detonators (51.xx). This is certainly
an indication that NVIDIA's "next-generation automatic shader optimizer" does
exist, and will soon kick into high gear. So far it looks like NVIDIA
forgot about their own optimization guidelines:
An optimization must produce the
correct image
An optimization must accelerate more than just a benchmark
An optimization
must not contain a pre-computed state.
AquaMark3 Benchmark is a fantastic tool, not only for an average Joe,
but for people that demand something extra from such an application.
Whether you need an accurate fillrate test, apple to apple comparison
or a blazing fast and reliable DirectX
9 benchmark with backward compatibility, you can't go wrong with AquaMark3.
This is not some predefined synthetic benchmark, it's based on a real
game engine: Krass™ Engine. Utilizing latest DirectX 9 API, developers
had a great time with the new pixel and vertex shader formats:
The most
powerful new feature is the pixel shader 2.0 model, that allows
significantly more
textures
and
complexer
computations
in
one
pass. Also the new higher-level shading language (HLSL) makes life
for a developer much easier, and produces usually as good code as
you write
by hand.
Weighing only ~60 Megabytes and priced at $9.95 for Professional license,
it's simply a steal. Accurate screenshot frame grabber and automated measurement
functionality are only two features that stand out from the bunch.
Other benchmarks can only dream of accurate SVIST™ or
PIXPM™ technologies. Application level settings make this the most
sophisticated and easiest to use benchmark ever.
Will AquaMark3 replace Futuremark's benchmark suit? Wait
until September 15th and see for yourself.
