That really looks like a driver bug. I think I know one workaround to the problem. For sure you need to send a bug report to firstname.lastname@example.org
I think the problem is related to the "Xrender" extension of X which is used for AA fonts and other stuff. You can try to disable hardware acceleration in the render extension or you can choose to disable it fully.
Play with the following options in your screen or device section of the XF86Config(-4) file:
Perhaps these things help.