I have the same issue with Quora. Makes me sad every time I go there and find useful info, but feel pushed away. Dropbox also does similar -- in fact they broke all their old direct public links, so that you can use their new "sharing" dialog, so that they can spam the recipient to get a Dropbox account. Sigh.
Worry not, I'm slowly assembling a massive Quora WARC file of all of the posts uncensored by the login wall (for cold storage). The moment they're out of business, a magnet torrent will be provided.
Seems like kind of a paradox. Too much to do, so you'll sleep when you're dead. But sleep deprivation will kill you, so you won't be able to finish all those things that are keeping you busy. :)
I always get eight hours of sleep ;) if a project exceeds my time availability, and cannot be automated, I delegate, even if it requires paying out of my pocket.
Thanks for doing this! There are so many people putting so much effort in writing detailed Quora answers on such a wide range of topics. All of that content disappearing whenever Quora inevitably shuts down/gets acquired would be akin to a library burning down.
Would you mind sharing your spider technology stack. How do you find new urls efficiently? What spider and what storage do you use? Do you append everything in one WARC file and split it if it gets too big?
