I had exactly the same problem, but it was resolved by turning off CONFIG_VGA_ARB option.
This option can be found in "Device driver" > "Graphics support". But its necessary to check "Configure standard kernel features" option in "General setup" to make this option visible.
However, after switching off VGA arbiter VESA framebuffer doesn't work anymore.
