Right. What happens when a construction crew accidentally cuts the backhaul fiber for the local cellular tower? Or if the autonomous vehicle failure occurs in a tunnel with no cell service?
You'll note that Waymo is very much operating in a limited area. So I think the answer is: they don't offer service in an area that lacks good cell coverage.
Of course, it's still possible that the vehicle will somehow be out of communication when it encounters something it doesn't understand. A cell jammer, say. In which case, it'll do what it will do if it, say, detects engine trouble: it'll pull over and wait.