Broadly speaking the line is: someone in the background, appearing briefly: fine.

Taking photos of specific people in public without their consent: not fine.