I compiled a custom kernel with agpgart disabled (not even loadable as a module). I have another machine that works, even though agpgart is loadable as a module, but configuration on that machine prevents agpgart loading. Hence, the nvidia module works fine on both. What does the OP get when he runs modprobe agpgart?
Life would be perfect if it wasn't for thisone small glitch....