I have (had) the same problem with a AGP FX5200 DVI. Earlier this month I purchased a new Dell FP1905 LCD which supports both VGA and DVI input. I could not get the FX5200 DVI output to work with the FP1905. I thought I had a defective FP1905. I tried everything, but nothing would get the FX5200 and FP1905 combo to output Digital. BTW- VGA worked fine!
So I bit the bullet and purchased a new AGP FX6600. The reason I bought the new FX6600 was to eliminate the Display as the problem. (really I wanted a better graphics card) Voila!!, DVI worked on the new FX6600! I did not change drivers (NV 77.77) or MB BIOS All I did was install the new AGP FX6600 card.