I backup my dvd's using anyDVD to rip them to .iso's. For Bluray/HDDVD I just rip them to a folder then use eac3to to extract the video, audio, and chapters. Then I remux them together using mkvMerge into a .mkv with audio in .flac and usually a .dts or .ac3 (depending on original source).
And I'm using mediabrowser
for playback. After setting up vista media center to do hardware video decoding on my .mkv's everything is perfect. I used XBMC for a while but the lack of hardware decoding drove me away. I love mediabrowser because of how it integrates into media center.