You have to use 4499 (not 4496), IOMMU should be no problem, but K8 NUMA and a NPTL-enabled glibc cause problems... You should also patch the kernel with the patches from amd64.org, or use kernel 2.6.1-rc1.
There are problems with AGPGART on kernel 2.4.x, update to 2.6.0 and fetch the patches at amd64.org (or use kernel 2.6.1-rc1), and do not(!) use a NPTL-enabled glibc or K8 NUMA.
"Computer games don't affect kids; I mean if Pac-Man affected us as kids, we'd all be running around in darkened rooms, munching magic pills and listening to repetitive electronic music."
-- Kristian Wilson, Nintendo, Inc, 1989.