Join Date: Mar 2003
Originally posted by The Baron
Somebody want to sum it up for me (or post an HTML version) since I'm unable to read PDF files at the moment?
Here it is in cleartext: (I just added newlines for better readability)
3DMark® Optimization Guidelines Q&A
Date: September 26th, 2003.
Q: How will Futuremark enforce these new rules?
A: We have internal tools and resources with which we will test new drivers against the guidelines.
We will maintain a list of drivers we have determined to be in accordance with the guidelines. We will
publish detailed description of this process including actions when violations have been discovered.
Q: When will the enforcing begin?
A: The rules are in effect now. Our objective is to begin full-fledged enforcing at latest on October
31st. The reason is that we want to give all manufacturers time to adjust to the guidelines.
Q: Who has the final say on what is and isn't acceptable?
A: Futuremark has the final say on all matters regarding the validity of 3DMark results.
Q: Are there any “Futuremark Approved” drivers currently available, according to your new
A: No, none of currently available official, or unofficial drivers have been yet tested against the new
guidelines. In other words, there are no “Futuremark Approved” drivers currently available.
Q: Do the new optimization guidelines apply to the whole 3DMark series?
A: The new optimization guidelines apply to all currently supported, and future 3DMark versions.
Wording Used In the Guidelines
Q: What is considered a “hardware error”?
A: It is a Futuremark approved error that an IHV has reported to Futuremark. Basic characteristic of
possibly qualifying hardware error is that the hardware is unable to run 3DMark (e.g. 3DMark does
not start, test run is halted, or the rendering is erroneous). Cases, which do not qualify as hardware
errors are e.g. lack of a specific feature in the hardware or low execution speed of a hardware when
running 3DMark code.
Q: What do you mean by changing the rendering quality?
A: Changing things like the FSAA settings, data and calculation precision or texture filtering quality to
something else than 3DMark instructs to use.
Q: What do you mean by detecting 3DMark directly or indirectly?
A: The drivers may not include routines that in any way identify that 3DMark (or any part of it) is
running, and go into some special mode, instead of running 3DMark like any other 3D application.
Q: What do you mean by utilizing the empirical data of 3DMark?
A: This relates to and highlights one special case to the rule number 2, about detecting 3DMark. The
driver is not allowed to notice that a certain test of 3DMark is running, and then discard or change
some rendering operations, when it is known that the camera never looks that way or does not look
close enough at some object in the default test. This is not allowed, and this kind of optimization
would never work in for example a game, where the camera path is not pre-defined.
Q: What do you mean by an optimization benefiting applications in general?
A: It is easier to think about this as an opposite of application specific optimization. If for example the
2nd rule is broken, and some part of 3DMark is identified and changed, that optimization would not
benefit any other application than 3DMark, and this kind of optimization is not general optimization
and thus it is prohibited.
Q: What do you mean by the resulting rendering being mathematically consistent with that of
the DirectX reference rasterizer?
A: This rule differs from the others by telling how things should be done, instead of how they should
not be done. The resulting image of the optimized rendering must be as close as possible for that
hardware to the rendering of the reference rasterizer. Also, the calculations performed in the
rendering process must be mathematically equivalent to those performed by the reference rasterizer.
Which Scores Are Valid?
Q: Do the new optimization guidelines invalidate the data you have in your
from 3DMark2001 SE and 3DMark03?
A: No, but future official lists and online services (e.g. Hall of Fame, Performance Analyzer, etc.) will
be based on data obtained only by products which completely fulfil these new optimization guidelines.
Q: Will you not allow any other drivers, than the ones you approve?
A: In general, we can only approve WHQL’ed drivers that we have verified in-house, but we reserve
the right to approve non-WHQL drivers in special case-by-case scenarios. However, we still allow
users to submit results to the ORB gained using beta, unofficial, leaked, non-approved & non-WHQL
drivers, but the data collected using these drivers will not be used in any official lists or online
Q: What about previews of new hardware? Usually manufacturers ship unofficial non-WHQL
drivers to be used in previews, and by following your new ground rules, using 3DMark in
previews will become impossible!
A: Using 3DMark in previews is preferred, but only with drivers, which are approved by us. We
presume that manufacturers will want to make sure the validity of their drivers with us before they
ship those to reviewers.
Q: Have you been discussing the new ground rules with all members in the benchmark
development program, and have they all approved them?
A: We have worked with all members in drafting these guidelines and we have solicited feedback
from them throughout the process. We have gotten very strong agreement to these rules from an
overwhelming majority of the members.
Q: Why did it take so long for you to make these new ground rules?
A: Due to the weight that 3DMark carries in the industry, one must very carefully study every
stakeholder’s point of view and their arguments before making conclusions. When you are dealing
with such a large number and big companies, the process just takes time.
Q: Isn't it fairly typical that graphics drivers have these special modes for running the most
popular games, in order to increase the frame rate during game play?
A: Yes, it is typical, but it is not acceptable when running 3DMark. Increasing the frame rate of a
game is fine as long as the developer of the game and/or the gamer accepts the means by which this
is achieved. However, there is no 'in game' frame rate to improve in 3DMark. Any such optimization
manipulates the 3DMark score. Since 3DMark is designed to measure generic DirectX performance,
such special case optimizations are not allowed.
Last edited by Corak; 09-26-03 at 09:18 AM.