Ok, well here we go....
I feel stupid. I had installed the nforce drivers, but forgot about the graphics drivers. I am using an nvidia card, but not the one on the nforce2 board.
I found and installed the kernel and the GLX driver with the graphical installer and everything was fine. Now performance is great.
The thing is, I never was able to find this driver on the nvidia site while looking for drivers - I got it via a link from another site. How can I get to the latest graphical installer for the video card?