Hello, nice to know I am not alone!
I am experiencing exactly the same problem on my Linux x86-64 Box (AMD CPU).
Happened since I've upgraded the RAM from 3GB to 8GB, and therefor had to compile the GART IOMMU into the kernel.
When playing a 3D game (in my case 'ET') it first runs fine for a few minutes, but then chokes down. I did watch the processes and the game is constantly leaking huge amounts of memory.
I then tried the various driver versions back to 173.14.12, and with every newer driver the memory leak got worse. Also to note is, that the game starts to choke when the leaking process reaches around 4GB of memory. I can kill the process and anything is fine again.
I spend the last whole two months with trying to find an issue in my systeminstallation/configuration
and even tried various kernel commandline options in regard to that...
but the only temporary reasonable solution I've found is to pass mem=3G
to the kernel command line. And the process behaves like it should (stays all the time at nearly same memory amounts).
Obvisiously with the parameter set, the hole GART IOMMU thingie doesn't get loaded. And there is only the SWIOTLB and AGPGART which depends on that. So maybe you know where to look at...
That means for me, I have to reboot my box and interactively add this parameter to grub when I want to play a game, since I don't want to go without all the RAM all the time.
And besides that, there are no other problems here (like that corruption thingie).