The problem is there's too much variety in the backgrounds of the training set. They don't follow a pattern the way growing a human does.