Sounds like some sort of kernel panic. I have found the easiest way to spot those is to either get it to crash while you are at a console window (in which case the dump will pop up on screen) or connect a serial terminal which will also show the dump. You could start the heavy process and then CTRL-ALT-F1 or whatever to get to a console and then wait for it to crash.
If you can get that dump/trace then you will probably be able to see what is causing the problem.
Do the SysRq magic
keys still work? (has to be turned on in your kernel)