You don't really have to rebuild the kernel. What is happening (as I understand it) is that the build of the driver has dependencies on the kernel source and since you haven't "built" the kernel, the files needed to satisfy those dependencies aren't "made" yet.
If you have the config file that was used to build your kernel, you should be able to do a "make dep" in the kernel source directory and then build your driver. No need to do a full kernel make.
"make dep" actually builds the make files, some include files and makes sym-links in the source directory that are necessary to build modules.
Hope that helps. Cheers