Publication
How is the weather tomorrow?: towards a benchmark for the cloud
Carsten Binnig; Donald Kossmann; Tim Kraska; Simon Loesing
In: Benoît Dageville; Carsten Binnig (Hrsg.). Proceedings of the 2nd International Workshop on Testing Database Systems. International Workshop on Testing Database Systems (DBTest-2009), June 29, Providence, RI, USA, ACM, 2009.
Abstract
Traditionally, the goal of benchmarking a software system is to evaluate its performance under a particular workload for a fixed configuration. The most prominent examples for evaluating transactional database systems as well as other components on top (such as a application-servers or web-servers) are the various TPC benchmarks. In this paper we argue that traditional benchmarks (like the TPC benchmarks) are not sufficient for analyzing the novel cloud services. Moreover, we present some initial ideas how such a new benchmark should look like that fits better to the characteristics of cloud computing (e.g., scalability, pay-per-use and fault-tolerance). The main challenge of such a new benchmark is to make the reported results comparable because different providers offer different services with different capabilities and guarantees.