Your post is valid from a technical and idealistic standpoint, however when you realize the size of the data sets turned over in the film / TV world in a daily basis, restoring, hashing and verifying files during production schedules is akin to painting the forth bridge - only the bridge has doubled in size by the time you get half way through, and the river keeps rising...
There are lots of companies doing very well in this industry with targeted data management solutions to help alleviate these problems (I'm not sure that IT 'solutions' exist), however these backups aren't your typical database and document dumps. In today's UHD/HDR space you are looking at potentially petabytes of data for a single production - solely getting the data to tape for archive is a full time job for many in the industry, let alone administration of the systems themselves, which often need overhauling and reconfiguring between projects.
Please don't take this as me trying to detract from your post in any way - I agree with you on a great number of points, and we should all strive for ideals in day to day operations as it makes all our respective industries better. As a fairly crude analogy however, the tactician's view of the battlefield is often very different to that of the man in the trenches, and I've been on both sides of the coin. The film and TV space is incredibly dynamic, both in terms of hardware and software evolution, to the point where standardization is having a very hard time keeping up. It's this dynamism which keeps me coming back to work every day, but also contributes quite significantly to my rapidly receding hairline!
> Your post is valid from a technical and idealistic standpoint
You seem to have direct experience in that particular industry, but I disagree that I'm being "idealistic" (often used as a condescending pejorative by people who want to lower standards). I'm managing the risk based on the value of the asset, the risk to it, and the cost of protecting it. In this case, given the extremely high value of the asset, the cost and difficulty of verifying the backup appears worthwhile. The internal company newsletter in my example above is not worth much cost.
> solely getting the data to tape for archive is a full time job for many in the industry, let alone administration of the systems themselves, which often need overhauling and reconfiguring between projects.
Why not hire more personnel? $100K/yr seems like cheap insurance for this asset.
> restoring, hashing and verifying files during production schedules is akin to painting the forth bridge - only the bridge has doubled in size by the time you get half way through, and the river keeps rising...
> you are looking at potentially petabytes of data for a single production
I agree that not all situations allow you to perform a full restore as a test; Amazon, for example, probably can't test a complete restore of all systems. But I'm not talking about this level of safety for all systems; Amazon may test its most valuable, core asset, and regardless there are other ways to verify backups. In this case it seems like they could restore the data, based on the little I know. If the verification is days behind live data or doesn't test every backup, that's no reason to omit it; it still verifies the system, provides feedback on bugs, and reduces the maximum dataloss to a shorter period than infinity.
> I disagree that I'm being "idealistic" (often used as a condescending pejorative by people who want to lower standards)
A poor word choice on my part. It was certainly not meant to come across that way, so apologies there! Agreed that a cost vs risk analysis should be one of the first items on anyone's list, especially given the perceived value of the digital assets in this instance.
There are lots of companies doing very well in this industry with targeted data management solutions to help alleviate these problems (I'm not sure that IT 'solutions' exist), however these backups aren't your typical database and document dumps. In today's UHD/HDR space you are looking at potentially petabytes of data for a single production - solely getting the data to tape for archive is a full time job for many in the industry, let alone administration of the systems themselves, which often need overhauling and reconfiguring between projects.
Please don't take this as me trying to detract from your post in any way - I agree with you on a great number of points, and we should all strive for ideals in day to day operations as it makes all our respective industries better. As a fairly crude analogy however, the tactician's view of the battlefield is often very different to that of the man in the trenches, and I've been on both sides of the coin. The film and TV space is incredibly dynamic, both in terms of hardware and software evolution, to the point where standardization is having a very hard time keeping up. It's this dynamism which keeps me coming back to work every day, but also contributes quite significantly to my rapidly receding hairline!