Skip to main content


Towards comprehensive measurement of consistency guarantees for cloud-hosted data storage services


David Bermbach, Liang Zhao and Sherif Sakr



The CAP theorem and the PACELC model have described the existence of direct trade-o s between consistency and availability as well as consistency and latency in distributed systems. Cloud storage services and NoSQL systems, both optimized for the web with high availability and low latency requirements, hence, typically opt to relax consistency guarantees. In particular, these systems usually o er eventual consistency which guarantees that all replicas will, in the absence of failures and further updates, eventually converge towards a consistent state where all replicas are identical. This, obviously, is a very imprecise description of actual guarantees.

Motivated by the popularity of eventually consistent storage systems, we take the position that a standard consistency benchmark is of great practical value. This paper is intended as a call for action; its goal is to motivate further research on building a standard comprehensive benchmark for quantifying the consistency guarantees of eventually consistent storage systems. We discuss the main challenges and requirements of such a benchmark, and present rst steps towards a comprehensive consistency benchmark for cloud-hosted data storage systems. We evaluate our approach using experiments on both Cassandra and MongoDB.

BibTeX Entry

    booktitle        = {The 5th TPC Technology Conference on Performance Evaluation \& Benchmarking},
    author           = {Bermbach, David and Zhao, Liang and Sakr, Sherif},
    month            = aug,
    year             = {2013},
    title            = {Towards Comprehensive Measurement of Consistency Guarantees for Cloud-Hosted Data Storage Services},
    address          = {Trento, Italy}


Served by Apache on Linux on seL4.