Good news, everyone!
You are now reading this in my voice!
It turns out that the thermal adhesive has a tendency to stick to the metal of the heatsinks when you pry them off, rather than the plastic on the board itself, which makes for a very clean removal. For the parts that stick to the board, run over the IC's with some 99% isopropyl alcohol using a cotton pad, let it soak in, and use a toothpick to remove whatever is left. A toothpick is small enough to remove things precisely, hard enough to remove the thermal adhesive, but soft enough not to scratch the IC's or the board. Mine came off almost completely clean.
Good overclocking, Nvnews!