For some reason my system automatically reverted to the vesa driver after the original crash, and I noticed it after I posted the original message. I'm going to try this again.
I attached a new one to this post. The problem just happened again, only this time, I pressed ALT+CTRL+Backspace and then Nvidia driver restarted and its working for now, it seems to always crash on the first attempt.
I noticed this line in my kdm.log and it concerns me:
The XKEYBOARD keymap compiler (xkbcomp) reports:
> Error: Missing KeyNames section in a Keymap file
> Description of Keymap not compiled
Errors from xkbcomp are not fatal to the X server
(EE) Error loading keymap /var/lib/xkb/server-0.xkm
Perhaps that's the problem? If it is I have no idea how to fix it.