Scalable implementation of dependence clustering in Apache Spark
Ivannikova, E. (2017). Scalable implementation of dependence clustering in Apache Spark. In I. Škrjanc, & S. Blažič (Eds.), EAIS 2017 : Proceedings of the 2017 Evolving and Adaptive Intelligent Systems (EAIS) (pp. 1-6). IEEE. https://doi.org/10.1109/EAIS.2017.7954843
Päivämäärä
2017Tekijänoikeudet
© 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
This article proposes a scalable version of the
Dependence Clustering algorithm which belongs to the class
of spectral clustering methods. The method is implemented
in Apache Spark using GraphX API primitives. Moreover, a
fast approximate diffusion procedure that enables algorithms
of spectral clustering type in Spark environment is introduced.
In addition, the proposed algorithm is benchmarked against
Spectral clustering. Results of applying the method to real-life
data allow concluding that the implementation scales well, yet
demonstrating good performance for densely connected graphs.
Julkaisija
IEEEEmojulkaisun ISBN
978-1-5090-6444-1Konferenssi
Evolving and Adaptive Intelligent SystemsKuuluu julkaisuun
EAIS 2017 : Proceedings of the 2017 Evolving and Adaptive Intelligent Systems (EAIS)ISSN Hae Julkaisufoorumista
2473-4691Asiasanat
Julkaisu tutkimustietojärjestelmässä
https://converis.jyu.fi/converis/portal/detail/Publication/27339946