Ok, slight update. X doesn't freeze completely. It just lags a lot.
It seems, that the following error codes resemble some kind of time out
[58776.317552] NVRM: Xid (0002:00): 8, Channel 00000003
[58785.317554] NVRM: Xid (0002:00): 8, Channel 00000004
If I am patient enough, I can kill the offending program and X returns to the normal state.