That's right. You can either fetch a 2.4.20-6 Red Hat kernel source tree or download a stock 2.4.22 kernel from
www.kernel.org. After compiling it and restarting the system you can then uninstall the binary NVIDIA RPM, download the NVIDIA code and try to compile nvnet, using the 2.4.22 source code.