Prediction-based auto-scaling of scientific workflows
| Authors | |
|---|---|
| Publication date | 2011 |
| Host editors |
|
| Book title | Proceedings of the 9th International Workshop on Middleware for Grids, Clouds and e-Science: MGC 2011, co-located with ACM/IFIP/USENIX 12th International Middleware Conference, December 12-16, 2011, Lisbon, Portugal |
| ISBN |
|
| Event | 9th International Workshop on Middleware for Grids, Clouds and e-Science |
| Pages (from-to) | 1 |
| Publisher | New York, NY: ACM |
| Organisations |
|
| Abstract |
In this paper we propose a novel method for auto-scaling data-centric workflow tasks. Scaling is achieved through a prediction mechanism where the input data load on each task within a workflow is used to compute the estimated task execution time. Through load prediction, the framework can take informed decisions on scaling multiple workflow tasks independently to improve overall throughput and reduce workflow bottlenecks. This method was implemented in the WS-VLAM workflow system and with an image analyses workflow we show that this technique achieves faster data processing rates and reduces overall workflow makespan.
|
| Document type | Conference contribution |
| Language | English |
| Published at | https://doi.org/10.1145/2089002.2089003 |
| Downloads |
prediction_scaling_scientific_workflows.pdf
(Final published version)
|
| Permalink to this page | |
