Are there any clues in /var/log/messages and/or /var/log/Xorg.0.log that might indicate something wrong?
Try removing anything nvidia related from $X11BASE/lib/modules $X11BASE/lib/modules/drivers $X11BASE/lib/modules/extensions.
Then use Fusselbaer's posting
here to install 9631 using the freebsd port.
I have an SMP system (dual athlon MP) and an FX5200 that currently uses the 9746 driver but before that I had the 9631 version. So I don't think SMP is an issue.