I had the exact same problem 5-6 years ago with a Voodoo Banshee card. If I loaded up a 3D program (Need for Speed 3, at the time, was what I used), then it'd work. If I didn't, it would lock up after about a minute.
This was either Windows 95 or 98, I don't remember for sure.
The problem was that the card had been damaged in shipping. We sent it back to the reseller, who promptly sent us back the same (still broken) card again, so we sent it to Diamond, who replaced it for us. The new one worked fine.
Since the problems are cross-OS, that makes me think it's hardware related all the more.