Well, we are descendants of people who had lots of kids for a very long time. By that logic we should be having lots of kids today, but it's obviously not the case.
The environment we find ourselves in has undergone rapid and drastic change. Many people are poorly adapted to this changed environment. But not everybody.
Their adaptations will spread. And, thus, we shall evolve.