The solution in post #5 to thread "8762 still locks up like 8756" by marcdw fixes the problem I was having. I just had to disable NvAGP with:
Option "NvAgp" "0"
Turns out the problem was easy to fix, it just took a while to figure it out. Though, I was sure I tried this before without success.