Originally posted by Nutty
The point is, they wouldn't. Unless IQ is being sacrificed. If its not, then wheres the problem?
I mean I dont care that NV is cheating if the IQ stays the same. I really dont.
Because unless you examine a game closely and make comparison, you won't know when, where, and how IQ gets sacrificed, and you won't know what future areas it might break. What might get broken if the developer makes a core patch or introduces new maps, or modders make their own variants on an engine while unaware of different ways IHV's approach it? And considering how many ways IHV's could optimize only for game benchmarks--without compromizing IQ--how can you trust the numbers OR the quality in comparison to the way the actual game will perform? FRAPS is not repeatable enough yet, and not every game allows one to record their own runthroughs to use FRAPS on, so for right now these kinds of worries are entirely valid. How do you get proper analysis of the future (something very important to consider for any smart consumer) off things we can only see now and can't trust their whole process--the important part--and can just see the visual end-results?
Basically, though the ABSOLUTE LAST step may not really matter in this case, all the intermediary ones can have undesirable side-effects, and would just seem to be the wrong paths to reward. It's fragmenting even more an area of the computer world that's hard enough to follow, and that we've spent many years trying to SOLIDIFY.
On Futuremark's current steps, I think it's definitely the best approach to take, since simply labelling a current set of drivers as "cheating" and forcing results from a few drivers back can kill many valid optimizations as well as the invalid ones. If Futuremark is confident they excised everything objectionable to their new rules, then they are free to approve the current drivers under their new patched version. It removes offending paths while keeping valid ones, as well as being the "smoothest" adoption method. The critical steps are the ones that go from here on out. They've been VERY careful in working up their guidelines and parameters so as to leave no wiggle-room, and they've "reset the counter" as it were. How will they respond if nVidia were to patch 52.16 to defeat their current 3DMark2003 build? How will they respond to later infractions from nVidia, ATi, or any other source?
They've taken a solid stand on THEIR desires alone, and made a long, careful progression to this point, but the maintainance will likely be even harder and more critical to ensure they don't have to do this again.