If the NVdriver kernel module is the problem, then the crashes will most likely continue as long as the NVdriver is being used or loaded. If you load it on every boot, then (again, assuming the problem with kswapd zombieing is the fault of it) it will most likely happen regardless of which driver you're using.
However, I have seen the same problem a while back, and I don't know what was causing it then either. But that was on kernels 2.4.16, 2.4.17, and 2.4.18 (at least... possibly also on 2.4.14). 2.4.19-pre8 doesn't do it any more (my firewall running that kernel has an uptime of like three weeks now), but then again, this firewall isn't running X. And the NVdriver isn't loading.
On my main-use machine, X and the 3123 drivers are loading, and it's using kernel 2.4.19. I haven't seen that problem yet. *crosses fingers*
Perhaps an upgrade to 2.4.19 might be in order, at least for testing?