This is exactly how I felt reading the book. If the innermost story were extracted into a standalone work, it would be excellent, even more so if it were expanded upon. All the outer frame stories were painfully uninteresting to me. I'd suggest finishing the house story and just skipping the rest, as I felt similarly to you and got nothing out of the rest, but really enjoyed the house stuff.