More than for "a while". Oil was discovered hundreds, thousands, of years ago. It used to seep out of the ground in many places (See the opening sequence of The Beverly Hillbillies). Only once we started pumping it did all the surface seeps dry up.
Fun fact, it still does, in 2019, seep out of the ground in downtown Los Angeles. Literally on the miracle mile sits the La brea Tarpits. You can see the pond with oil bubbling out of the water. If you're ever in Los Angeles, it is a very memorable thing to visit.
