as how i understand it, when nforce kernel drivers are properly installed (no errors whatsoever..) it patches the i_810 drivers for audio. and the nvidia manual asks you to add an alias to i_810 named nvudio. this is what you load to get the sound properly in everything. if you remove nvidia kernel, you'll be left with the unpatched i_810 (i guess) and that might work partially well. so the best solution is to keep the nvidia kernel drivers (else you'll lose lan and USB) and use the patched i_810 drivers.
we just started a dedicated nForce-Linux topic there that has a detailed installation guide.