local storage + remote replication
this question isn't about Openstack, it's rather general question about
cloud data storing.
It is obvious, that distributed network storage is generally slower than
local storage, while is much more reliable. Keeping in mind, that
reliability of hardware components became better and better with time
(e.g. MTBF of HP DL380G2 is almost 14 years), power systems are reliable
enough (if built with reliability in mind), it is safe to say that the
most possible source of problems is failure of storage itself, not
motherboard or NIC or power supply.
Why not to use kind of caching, quickly saving data on local storage,
ACKing the operation and replicating changed blocks to remote storage in
background during idle time? Having a local RAID1 (with checksums to
allow distinguish between failed and correct copy of the data block) +
remote replication will give a high speed and enough level of
reliability. The only case of temporary slowing down is migration to
another node (not pretty often event?) with gradual back replication
(from remote storage to local, as the VM will request data) which will
last some time from start at new node.
Are there serious concerns against such approach? I will appreciate your
opinion on this topic.
"Vision without Execution is Hallucination." -- Thomas Edison