Originally posted by aaronspink
If you look at the reason why Futuremark objected, you will find that it had nothing to do with the mathematical correctness of the shader optimization.
The issue is that the optimization actually was in violation of both #2 ( an optimization must accelerate more than just the benchmark ) and #3 ( an optimization must not rely on any precomputed state ).
#2 is easy. The optimization was specific just to GT4. #3 is more subtle: ATI was using shader replacement which relied on doing a match ( likely using a hash of the original shader ) and replace operation on the original shader. This required a prior knowledge of what the shader was.
Any and all non-algorithmic ( aka bulk shader replacement ) would violate #3 but something like the new instruction order optimizers that both ATI and Nvidia are adding to their drivers would not since the optimizers don't rely on any precomputed state. Now there could still be possibilities of the instruction optimizers violating #2.
The reason why Futuremark and Valve are concerned about the bulk shader replacement is that it hides real issues with the drivers/hardware of a video card that can be exposed with shader updates, etc.
As a side comment. If the hardware sites are serious about this, they may want to take a look at some of the specCPU run rules. While the spec rules are far from perfect, they are closer to it than anything out there.
speaking for myself inc
Well, I don't see how NVIDIA's guidelines would have any bearing on ATI, especially when they were contrived months after
the whole 3dmark03 thing. Futuremark's policy has been that no specific application optimizations are legal. Although we will be hearing an update from them later this week.
I do agree with you though. ATI's driver shader compiler should catch things like that and do automatic optimization, they shouldn't need a special case. Anyway, it looks like CAt3.7's compiler is better than previous versions since shaders got a boost