Without framebuffer worked for you? That's odd -- most of the people that I remember reporting issues like this (blank screen or no signal or something upon exiting X) weren't using the framebuffer. Turning it on fixed it for them...
Plus, I use it with FB on, too, without issues.
Hmm... well, I suppose if it worked for you, then it worked for you. But it does sound a little strange.