M.W. van Someren
- Analyzing Grid Log Data with Affinity Propagation
- Book title
- Recent trends in applied artificial intelligence: 26th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems (IEA/AIE) Amsterdam, The Netherlands, June 17-21, 2013 Proceedings
- Pages (from-to)
- Number of pages
- New York: Springer
- Lecture Notes in Artificial Intelligence
- Volume | Edition (Serie)
- Document type
- Faculty of Science (FNWI)
- Informatics Institute (IVI)
In this paper we present an unsupervised learning approach to detect meaningful job traffic patterns in Grid log data. Manual anomaly detection on modern Grid environments is troublesome given their increasing complexity, the distributed, dynamic topology of the network and heterogeneity of the jobs being executed. The ability to automatically detect meaningful events with little or no human intervention is therefore desirable. We evaluate our method on a set of log data collected on the Grid. Since we lack a priori knowledge of patterns that can be detected and no labelled data is available, an unsupervised learning method is followed. We cluster jobs executed on the Grid using Affinity Propagation. We try to explain discovered clusters using representative features and we label them with the help of domain experts. Finally, as a further validation step, we construct a classifier for five of the detected clusters and we use it to predict the termination status of unseen jobs.
- go to publisher's site
If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the material inaccessible and/or remove it from the website. Please Ask the Library, or send a letter to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. You will be contacted as soon as possible.