What was the fix? (so that others searching can find it
As for the font issue, that sounds like a DPI thing. The fastest way to fix it is to lower the font size in your DE or WM. The issue, I believe, is that the opensource "nv" X driver always reports a DPI of 75x75 (I haven't checked this assumption directly, but keep reading), and when you install the "nvidia" X driver, it reports the real DPI (which is probably closer to 100x100), which it finds by querying the monitor over DDC or EDID or one of those.
The reason I think the "nv" X driver always reports 75x75 is, if you force the "nvidia" driver to report 75x75 (by using a DisplaySize line in your Monitor section), the fonts go back to "normal" size.
The way you calculate the DisplaySize is, take the resolution you run at, and divide each dimension by 75. Then convert from inches to millimeters (mult. by 25.4), and use those numbers in a line like so:
DisplaySize xxx yyy
in your Monitor section. But rather than forcing an incorrect DPI, you'd probably be better off decreasing the font size. Or maybe forcing your distro to use the "vesa" driver instead during installation, that way it'll set up the font sizes closer to the way they should be (AFAIK the vesa driver does detect DPI correctly).