When you say "bought a house", you mean "bought a historic mansion", then left it to rot while he lived somewhere else. It rotted to the point where it was eventually torn down.
It was designed by a noted architect for a copper baron. It doesn't matter how old it was, the fact that a famous architect designed it means it's got history associated with it, therefore it's historic.
There's no need to get ageist with your buildings. Yes, Europe has old buildings. That doesn't detract from the fact that buildings in North America, even if they're fewer than 100 years old, can be considered "historic".
Given that, travelling through Palo Alto, one can still see one of the two "tall sticks" after which the town was named, that white men hadn't set eyes on the San Francisco bay until January, 1770, and much of what had been built in the area was destroyed in 1906, an 86 year old structure, particularly with ownership and design significance, can indeed be considered "historic".
Though I am well aware of family residences to which I've some association on other continents approaching 500 years in age.
History and longevity are also not universally revered: in Japan the tradition is to destroy and rebuild temples frequently (every 20 years in the case of the shrines at Naikū and Gekū).
http://www.sfgate.com/bayarea/article/Steve-Jobs-historic-Wo...