Well, that depends.
I still have a nV1 video card, the first video card/processor from nVidia and love to keep it. I have a drawer just for old stuff!
Donīt want to get rid of some stuff. Also have a 3Dfx Voodoo 1 & 2, a Matrox Millenium, etc! Some of these old stuff can be used to build a old rig and play some good old games but then again, nothing like the new rigs to play new games, right?!?
If you will never use them then get rid of them, otherwise keep some of the old stuff in some lost corner and just like you say, get rid of dust and remember the old days!