You must first install the kernel source RPM. Installing the source RPM(does not mean downloading) will set up a directory where you will have the kernel source files. It is not the location where the RPM was downloaded.
/usr/src/linux is a common place as the kernel source top dir, be it may be different for your distro.
You need to find out the right top dir for your distro. For that you need to locate where the kernel source RPM installation will put the files.