Buying a ton of parts, carefully assembling them, and having it be your problem when something breaks is simpler than paying Amazon to solve the problem nearly perfectly?
If you know what you're doing, yeah. Running a fileserver should be pretty damn simple for anyone who stylized themselves a "hacker", but you're talking like maintaining file servers is some sort of black magic for which the simple solution is to give up and turn to an outsourced solution where you have little control and no visibility.
Its not. The hard part of storage is to understand the capabilities and limitations of your storage and more importantly how those fit in with your computational needs. You have to do that at amazon on in house. A3 just makes it easy to ignore evaluating their service because you're never actually forced to. Maintaining servers is butter.
That sounds like a terrible data center. If you count a server short circuit with some minor smoke as a fire I guess it's understandable. The server room should have Halon/Inergen or a similar system with smoke detectors.
Building your own solution to beat S3 is certainly viable but at this scale I doubt it.
Buying a ton of parts, carefully assembling them, and having it be your problem when something breaks is simpler than paying Amazon to solve the problem nearly perfectly?