View Single Post
Old 11-11-03, 06:13 PM   #116
Nutty
Sittin in the Sun
 
Nutty's Avatar
 
Join Date: Jul 2002
Location: United Kingdom
Posts: 1,835
Send a message via MSN to Nutty
Default

Quote:
Exactly right - and if the task is 'run this piece of code, which I wrote in a standard programming language' then the driver/compiler/processor should do just that. It shouldn't decide that 16 or 12 bits is enough precision. That's not what I asked it to do.
No, its what what you asked it to do, but if the end result is the same, does it matter how it got there?

I can something like this in C
Code:
{
     int j=0;
     for(int i=0; i<100; i++)
     {
          j = 2;
     }
}
Now if I ask the compiler to compile that into machine code, it will just strip the whole thing out. Its not what I asked it to do, but it does nothing anyway, so it just removes it.


Quote:
If some enterprising hard drive manufacturer decided that all the reads/writes to a temporary file during a benchmark didn't do any meaningful work (after all, there's nothing left on the drive, is there?) and decided to just skip the whole thing and report 'done', would we call it a cheat, or congratulate them for an aggressive optimization?
Its not exactly the same is it. NV aren't skipping everything. They're just trying to get to the final result a bit easier.

And in actual fact what you said _does_ happen. In write caching, if I write a small file to disk, then immediately delete it, provided their was space in the cache, the file would have never touched the HD due to optimizations brought about by the cache.

Quote:
If I benchmark MP3 encoding on a CPU, is performing the encoding at 128kbps instead of the requested 256kbps acceptable if I can't hear the difference? Personally, I think not.
Again its not the same thing, as the end result is different. Whereas the end result with NV's drivers is not different.

Quote:
It's all about equal work. Who cares if the work being requested is inefficient? Just do the work (all of it) in the most efficient way you can. No sweeping things under the rug. No cutting corners. Just do it.
But what if you cant do it in the most efficient way without changing it? Are you gonna just let your product look like crap, or change it, so the result is the same, but shows your product in a better light.

You see for a race to be fair, they have to have equal capabilities. And this is the problem. One IHV designed the hardware one way, another the other way. I'm not saying this was good on NV's part, it wasn't. But the fact is they're stuck with it. The hardware is out there, and the only thing they can do, is play to it strengths.

What if I write a benchmark that does 100 sin functions in the fragment program. We all know NV's sin function is about 4 times faster, so would that be fair?

Quote:
Is this too much to ask?
Depends what exactly you're asking. If you're asking for a true apples to apples benchmarking system, then probably yes.

I'm a tired devils advocate now.. I'm off to bed.. I'll argue more tomorrow..
Nutty is offline   Reply With Quote