What if I would implement an S3-backed block storage. Like every 16MB chunk is stored in a separate object. And filesystem driver would download/upload chunks just like it does it with HDD.
And then format this block device with Ext4 or use it in any other way (LVM, encrypted FS, RAID and so on).
What you're describing already exists in Azure Blob Storage (Azure's S3 compete), known as Page Blobs, which is also how you host VM HDDs (VHDs) in Azure, as it allows for page-level (i.e. disk-block-level) reads/writes.
I'm not clued-in to AWS's equivalent, but a quick google suggests it's AWS EBS (Elastic Block Storage), though it doesn't seem to coexist side-by-side with normal S3 storage, and their API for reading/writing blocks looks considerably more complicated than Azure's: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-acce...
Azure blob storage took me by surprise when I had to load a custom VHD. My experience with AWS/EC2/S3 led me to think it would take half a day or longer.
Turns out, there's no transformation layers or anything. You upload the image using a special tool, it saturates your uplink for a few minutes, and before you can brew some more coffee the VM will be booting up.
Their fuse-based filesystem driver even supported CACHED block access ie you would download, cache and read the blocks you needed locally instead of sending every read over the wire. Close to local fs performance for little space
The single FUSE process can coordinate all that with an in-memory mutex.
This thread is in the context of filesystems like ext4, which are completely unsafe for concurrent mounts - if you connect from multiple machines then it will just corrupt horrifically. If your filesystem was glusterfs, moose/lizard, or lustre, then they have ways to handle this (with on-disk locks/semaphores in some other sector).
What if I would implement an S3-backed block storage. Like every 16MB chunk is stored in a separate object. And filesystem driver would download/upload chunks just like it does it with HDD.
And then format this block device with Ext4 or use it in any other way (LVM, encrypted FS, RAID and so on).
Is it usable concept?