VesaFB fixes it on mine.
After rebuilding with VESAFB and then booting the kernel with the vga=9 parameter this totally resolved the problem for me. I don't know why this is, only that doing this re-enabled the virtual terminals where none others seemed to work; i.e. the "vga16 framebuffer" doesn't work, and the default "no framebuffer" doesn't work either. Not a fix, only a workaround, although the vesafb console drivers seem stable enough and there is no big reason not to use them with a nVidia card.