Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I've never understood how people can use EBS in production. The durability numbers they quote are bad and they wave their hands around about increased durability with snapshots, but never quantify what that means.

Hard drives are unreliable and they certainly don't fail independently of one another - but the independence of their failure is much more independent than EBS.

With physical dives and n-parity RAID you drastically reduce the rate of data loss. This is because although failures are often correlated, it's quite unlikely to have permenant failure of 3 drives out of a pool of 7 within 24 hours. It happens, but it is very rare.

With EBS, your 7 volumes might very well be on the same underlying RAID array. So you have no greater durability by building software RAID on top of that. If anything, it potentially decreases durability.

You could utilize snapshots to S3, but is that really a good solution? It seems that deploying onto EBS at any meaningful scale is a recipe for garunteed data-loss. Raid on physical disks isn't a great solution either, and there is no substitute for backups - but at least you can build a 9 disk RaidZ3 array that will experience pool failure so rarely that you can more safely worry about things like memory and data bus corruption.



The increased durability based on snapshots is actually quite simple, and they explain it in various places: if one of the drives in Amazon's RAID fails, they need to bring up a new disk to replace it in the array. When they being up new disks they typically can do this instantaneously, because they really just dynamically page fault the drive from your latest snapshot. However, all dirty data since the last snapshot will have to be copied from the other drive(s). This is a window of time during which your array is exposed to unrecoverable read errors losing data. The less dirty data you have, the smaller this window of time.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: