I've seen this kind of behavior, too
you see it most obviously if you're running an compositing-manager (compiz(-fusion), kwin of kde4, ...) for a very long time with loads of animations (open-, close-animation, etc.)
then the resource usage e.g. will go up from RES: 107m -> 178m
switching to a non-compositing window-manager makes it somewhat better but this isn't an option if you're dependent on compositing,
also re-launching compiz doesn't help
the only "solution" so far seems to be to kill X and/or restart X and unload and reload the nvidia module then it's as fast as from the start
any ideas from the nvidia-developers; AaronP, ... ?
can you reproduce this ?
many thanks in advance