Go Back   nV News Forums > Hardware Forums > Benchmarking And Overclocking

Newegg Daily Deals

Reply
 
Thread Tools
Old 09-26-03, 06:55 AM   #1
Nick[FM]
Registered User
 
Join Date: Apr 2003
Posts: 38
Default Optimization Guidelines QA

Hey guys!

Just wanted to let you all know that we just posted the QA regarding our Optimization Guidelines, which can be viewed here (pdf).

Best,

Nick / Futuremark Corp.
Nick[FM] is offline   Reply With Quote
Old 09-26-03, 08:55 AM   #2
ClyssaN
Registered User
 
Join Date: Jan 2003
Posts: 145
Default

I agree in general expect with this one:

Q: When will the enforcing begin?
A: The rules are in effect now. Our objective is to begin full-fledged enforcing at latest on October
31st. The reason is that we want to give all manufacturers time to adjust to the guidelines.

Months have passed, no need to wait this long ..., more and more users are being deceived, pull the drivers that are "cheating" and wait for the manufacturers to released new ones ...
__________________
9700 Pro built by ATI (342/344)
Barton 2500+@2500 (11x227)
A7N8X Delx 2.0
512 Kingston PC3200
WD800JB, Maxtor DM Plus 60
Dual Display -> Samsung 1100DF (21) & Sampo 811 (19)
Antec 430W

ANIMATED ALGORITHMS IN JAVA
Click here to visit
ClyssaN is offline   Reply With Quote
Old 09-26-03, 08:57 AM   #3
The Baron
Guest
 
Posts: n/a
Default

Somebody want to sum it up for me (or post an HTML version) since I'm unable to read PDF files at the moment?
  Reply With Quote
Old 09-26-03, 09:13 AM   #4
Corak
na
 
Join Date: Mar 2003
Posts: 34
Default

Quote:
Originally posted by The Baron
Somebody want to sum it up for me (or post an HTML version) since I'm unable to read PDF files at the moment?
Here it is in cleartext: (I just added newlines for better readability)

Quote:
3DMark® Optimization Guidelines Q&A
Date: September 26th, 2003.



Enforcing


Q: How will Futuremark enforce these new rules?
A: We have internal tools and resources with which we will test new drivers against the guidelines.
We will maintain a list of drivers we have determined to be in accordance with the guidelines. We will
publish detailed description of this process including actions when violations have been discovered.

Q: When will the enforcing begin?
A: The rules are in effect now. Our objective is to begin full-fledged enforcing at latest on October
31st. The reason is that we want to give all manufacturers time to adjust to the guidelines.

Q: Who has the final say on what is and isn't acceptable?
A: Futuremark has the final say on all matters regarding the validity of 3DMark results.

Q: Are there any “Futuremark Approved” drivers currently available, according to your new
optimization guidelines?
A: No, none of currently available official, or unofficial drivers have been yet tested against the new
guidelines. In other words, there are no “Futuremark Approved” drivers currently available.

Q: Do the new optimization guidelines apply to the whole 3DMark series?
A: The new optimization guidelines apply to all currently supported, and future 3DMark versions.




Wording Used In the Guidelines


Q: What is considered a “hardware error”?
A: It is a Futuremark approved error that an IHV has reported to Futuremark. Basic characteristic of
possibly qualifying hardware error is that the hardware is unable to run 3DMark (e.g. 3DMark does
not start, test run is halted, or the rendering is erroneous). Cases, which do not qualify as hardware
errors are e.g. lack of a specific feature in the hardware or low execution speed of a hardware when
running 3DMark code.

Q: What do you mean by changing the rendering quality?
A: Changing things like the FSAA settings, data and calculation precision or texture filtering quality to
something else than 3DMark instructs to use.

Q: What do you mean by detecting 3DMark directly or indirectly?
A: The drivers may not include routines that in any way identify that 3DMark (or any part of it) is
running, and go into some special mode, instead of running 3DMark like any other 3D application.

Q: What do you mean by utilizing the empirical data of 3DMark?
A: This relates to and highlights one special case to the rule number 2, about detecting 3DMark. The
driver is not allowed to notice that a certain test of 3DMark is running, and then discard or change
some rendering operations, when it is known that the camera never looks that way or does not look
close enough at some object in the default test. This is not allowed, and this kind of optimization
would never work in for example a game, where the camera path is not pre-defined.

Q: What do you mean by an optimization benefiting applications in general?
A: It is easier to think about this as an opposite of application specific optimization. If for example the
2nd rule is broken, and some part of 3DMark is identified and changed, that optimization would not
benefit any other application than 3DMark, and this kind of optimization is not general optimization
and thus it is prohibited.

Q: What do you mean by the resulting rendering being mathematically consistent with that of
the DirectX reference rasterizer?
A: This rule differs from the others by telling how things should be done, instead of how they should
not be done. The resulting image of the optimized rendering must be as close as possible for that
hardware to the rendering of the reference rasterizer. Also, the calculations performed in the
rendering process must be mathematically equivalent to those performed by the reference rasterizer.
Which Scores Are Valid?

Q: Do the new optimization guidelines invalidate the data you have in your
database, gathered
from 3DMark2001 SE and 3DMark03?
A: No, but future official lists and online services (e.g. Hall of Fame, Performance Analyzer, etc.) will
be based on data obtained only by products which completely fulfil these new optimization guidelines.

Q: Will you not allow any other drivers, than the ones you approve?
A: In general, we can only approve WHQL’ed drivers that we have verified in-house, but we reserve
the right to approve non-WHQL drivers in special case-by-case scenarios. However, we still allow
users to submit results to the ORB gained using beta, unofficial, leaked, non-approved & non-WHQL
drivers, but the data collected using these drivers will not be used in any official lists or online
services.

Q: What about previews of new hardware? Usually manufacturers ship unofficial non-WHQL
drivers to be used in previews, and by following your new ground rules, using 3DMark in
previews will become impossible!
A: Using 3DMark in previews is preferred, but only with drivers, which are approved by us. We
presume that manufacturers will want to make sure the validity of their drivers with us before they
ship those to reviewers.




General


Q: Have you been discussing the new ground rules with all members in the benchmark
development program, and have they all approved them?
A: We have worked with all members in drafting these guidelines and we have solicited feedback
from them throughout the process. We have gotten very strong agreement to these rules from an
overwhelming majority of the members.

Q: Why did it take so long for you to make these new ground rules?
A: Due to the weight that 3DMark carries in the industry, one must very carefully study every
stakeholder’s point of view and their arguments before making conclusions. When you are dealing
with such a large number and big companies, the process just takes time.

Q: Isn't it fairly typical that graphics drivers have these special modes for running the most
popular games, in order to increase the frame rate during game play?
A: Yes, it is typical, but it is not acceptable when running 3DMark. Increasing the frame rate of a
game is fine as long as the developer of the game and/or the gamer accepts the means by which this
is achieved. However, there is no 'in game' frame rate to improve in 3DMark. Any such optimization
manipulates the 3DMark score. Since 3DMark is designed to measure generic DirectX performance,
such special case optimizations are not allowed.

Last edited by Corak; 09-26-03 at 09:18 AM.
Corak is offline   Reply With Quote
Old 09-26-03, 09:17 AM   #5
The Baron
Guest
 
Posts: n/a
Default

Thanks. Here's hoping that they have the backbone to risk alienating their beta members at times in order to preserve their integrity.
  Reply With Quote
Old 09-26-03, 09:27 AM   #6
Sazar
Sayonara !!!
 
Sazar's Avatar
 
Join Date: Jan 2003
Location: Austin, Texas
Posts: 9,297
Default

Quote:
Originally posted by The Baron
Thanks. Here's hoping that they have the backbone to risk alienating their beta members at times in order to preserve their integrity.
invertebrates do not have backbones...
Sazar is offline   Reply With Quote
Old 09-26-03, 09:31 AM   #7
digitalwanderer
 
digitalwanderer's Avatar
 
Join Date: Jul 2002
Location: Highland, IN USA
Posts: 4,944
Default

Quote:
Originally posted by Sazar
invertebrates do not have backbones...
Hey, you watch SpongeBob too?

I dunno, I like everything that they're saying and if they enforce it I'll be giving them another chance.
__________________
[SIZE=1][I]"It was very important to us that NVIDIA did not know exactly where to aim. As a result they seem to have over-engineered in some aspects creating a power-hungry monster which is going to be very expensive for them to manufacture. We have a beautifully balanced piece of hardware that beats them on pure performance, cost, scalability, future mobile relevance, etc. That's all because they didn't know what to aim at."
-R.Huddy[/I] [/SIZE]
digitalwanderer is offline   Reply With Quote
Old 09-26-03, 09:53 AM   #8
Sazar
Sayonara !!!
 
Sazar's Avatar
 
Join Date: Jan 2003
Location: Austin, Texas
Posts: 9,297
Default

Quote:
Originally posted by digitalwanderer
Hey, you watch SpongeBob too?

I dunno, I like everything that they're saying and if they enforce it I'll be giving them another chance.
only when my gf's baby nephew is over

else.. its south park

I thought their initial white paper was clear enough... if they had stuck by what that document stated.. things may have been a little different now... but whatever..
Sazar is offline   Reply With Quote

Old 09-26-03, 01:27 PM   #9
Nick[FM]
Registered User
 
Join Date: Apr 2003
Posts: 38
Default

Quote:
Originally posted by The Baron
Thanks. Here's hoping that they have the backbone to risk alienating their beta members at times in order to preserve their integrity.
We have been working with our members while drafting the guidelines. They are all aware of what is going on.
Nick[FM] is offline   Reply With Quote
Old 09-29-03, 09:35 AM   #10
The Baron
Guest
 
Posts: n/a
Default

Quote:
Originally posted by worm[FM]
We have been working with our members while drafting the guidelines. They are all aware of what is going on.
Whoops, forgot about this thread.

But, my real question--are you guys going to certify ANY currently available NVIDIA GFFX drivers for 3DMark03?
  Reply With Quote
Old 09-29-03, 09:38 AM   #11
digitalwanderer
 
digitalwanderer's Avatar
 
Join Date: Jul 2002
Location: Highland, IN USA
Posts: 4,944
Default

I think the magic question mark should also be on "Are you going to have seperate code paths for different IHV's hardware?" too.
__________________
[SIZE=1][I]"It was very important to us that NVIDIA did not know exactly where to aim. As a result they seem to have over-engineered in some aspects creating a power-hungry monster which is going to be very expensive for them to manufacture. We have a beautifully balanced piece of hardware that beats them on pure performance, cost, scalability, future mobile relevance, etc. That's all because they didn't know what to aim at."
-R.Huddy[/I] [/SIZE]
digitalwanderer is offline   Reply With Quote
Old 09-29-03, 09:40 AM   #12
The Baron
Guest
 
Posts: n/a
Default

Quote:
Originally posted by digitalwanderer
I think the magic question mark should also be on "Are you going to have seperate code paths for different IHV's hardware?" too.
Not in 3DMark03 they're not. 04? 05? Maybe, but that'd be silly--there are DX standards for a reason.
  Reply With Quote
Reply


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 11:16 PM.


Powered by vBulletin® Version 3.7.1
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Copyright ©1998 - 2014, nV News.