Anyone with only mildly better tech than we have now, can see your ecosystem changing over the course of the seasons before you invented fire, let alone built a Dyson swarm. We're just starting to have this capacity already in special cases, though we've not found any sign of an ecosystem, just "boring" diamond rain etc.
A Dyson swarm will keep you safe from any threat smaller than another Dyson swarm — and while you may not be able to "pack it up", you can use one to run to other galaxies… in fact, almost all of them… at close enough to the same time that light cones matter… and get the settlers moving at a significant fraction of the speed of light… and have a lot of redundancy.
A Dyson swarm will keep you safe from any threat smaller than another Dyson swarm — and while you may not be able to "pack it up", you can use one to run to other galaxies… in fact, almost all of them… at close enough to the same time that light cones matter… and get the settlers moving at a significant fraction of the speed of light… and have a lot of redundancy.