End-to-End QoS on Shared Clouds for Highly Dynamic, Large-Scale Sensing Data Streams

Abstract : The increasing deployment of sensor network infrastructures has led to large volumes of data becoming available, leading to new challenges in storing, processing and transmitting such data. This is especially true when data from multiple sensors is pre-processed prior to delivery to users. Where such data is processed in-transit (i.e. from data capture to delivery to a user) over a shared distributed computing infrastructure, it is necessary to provide some Quality of Service (QoS) guarantees to each user. We propose an architecture for supporting QoS for multiple concurrent scientific workflow data streams being processed (prior to delivery to a user) over a shared infrastructure. We consider such an infrastructure to be composed of a number of nodes, each of which has multiple processing units and data buffers. We utilize the "token bucket" model for regulating, on a per workflow stream basis, the data injection rate into such a node. We subsequently demonstrate how a streaming pipeline, with intermediate data size variation (inflation/deflation), can be supported and managed using a dynamic control strategy at each node. Such a strategy supports end-to-end QoS with variations in data size between the various nodes involved in the workflow enactment process. \textcopyright 2012 IEEE.
Document type :
Conference papers
Complete list of metadatas

https://hal-univ-pau.archives-ouvertes.fr/hal-01912938
Contributor : Gaelle Chancerel <>
Submitted on : Monday, November 5, 2018 - 5:50:24 PM
Last modification on : Friday, April 26, 2019 - 6:58:02 PM

Links full text

Identifiers

Collections

Citation

Rafael Tolosana-Calasanz, José Ángel Bañares, Cong-Duc Pham, Omer F. Rana. End-to-End QoS on Shared Clouds for Highly Dynamic, Large-Scale Sensing Data Streams. 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2012, Ottawa, Canada, May 13-16, 2012, 2012, Unknown, Unknown Region. pp.904-911, ⟨10.1109/CCGrid.2012.67⟩. ⟨hal-01912938⟩

Share

Metrics

Record views

31