For big-ticket items (cars and houses), official systems of title help. For other items, there's not certainty, as with the world in general you achieve relative confidences with the best info you have. (And, buying from reputable sources means you may have a claim against them for recovery of your payment even if the stolen property is returned to its rightful owner.)

How does it work if the insurance has paid out after the theft?

Insurance is a separate contract between the victim and a third party, the insurance company might be the legal owner, or want its claims payment returned.

See elsewhere in this thread, the situation is not as clear as you make it out to be.

I would certainly agree that it's complicated across jurisdictions and circumstances, but your statement -- "according to the law that person is now the rightful owner" -- remains generally false as you wrote it and bad advice in the USA.

Also, the world is a lot larger than just the US.

That is why I specifically said 'most' 'US' jurisdictions. What qualifiers -- jurisdictions and circumstances -- should have been included in your statement to make it true?