hh.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Decentralized and Adaptive K-Means Clustering for Non-IID Data using HyperLogLog Counters
RISE SICS, Stockholm, Sweden.
RISE SICS, Stockholm, Sweden.ORCID iD: 0000-0003-4516-7317
Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.ORCID iD: 0000-0002-2859-6155
Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS), CAISR - Center for Applied Intelligent Systems Research.ORCID iD: 0000-0003-3272-4145
Show others and affiliations
2020 (English)In: Advances in Knowledge Discovery and Data Mining: 24th Pacific-Asia Conference, PAKDD 2020, Singapore, May 11–14, 2020, Proceedings, Part I / [ed] Lauw H., Wong RW., Ntoulas A., Lim EP., Ng SK., Pan S., Cham: Springer Nature, 2020, Vol. 12084, p. 343-355Conference paper, Published paper (Refereed)
Abstract [en]

The data shared over the Internet tends to originate from ubiquitous and autonomous sources such as mobile phones, fitness trackers, and IoT devices. Centralized and federated machine learning solutions represent the predominant way of providing smart services for users. However, moving data to central location for analysis causes not onlymany privacy concerns, but also communication overhead. Therefore, incertain situations machine learning models need to be trained in a collaborative and decentralized manner, similar to the way the data is originally generated without requiring any central authority for data or modelaggregation. This paper presents a decentralized and adaptive k-means algorithm that clusters data from multiple sources organized in peer-to-peer networks. Our algorithm allows peers to reach an approximation of the global model without sharing any raw data. Most importantly, we address the challenge of decentralized clustering with skewed non-IID data and asynchronous computations by integrating HyperLogLog counters with k-means algorithm. Furthermore, our clustering algorithm allows nodes to individually determine the number of clusters that fits their local data. Results using synthetic and real-world datasets show that our algorithm outperforms state-of-the-art decentralized k-means algorithms achieving accuracy gain that is up-to 36%. © Springer Nature Switzerland AG 2020.

Place, publisher, year, edition, pages
Cham: Springer Nature, 2020. Vol. 12084, p. 343-355
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349 ; 12084
Keywords [en]
Decentralized Clustering, K-Means, HyperLogLog Counters, Distributed Machine Learning, Decentralized Machine Learning, Non-IID Data
National Category
Computer Systems
Identifiers
URN: urn:nbn:se:hh:diva-42014DOI: 10.1007/978-3-030-47426-3_27ISI: 000716986400027Scopus ID: 2-s2.0-85085735657ISBN: 9783030474256 (print)ISBN: 9783030474263 (electronic)OAI: oai:DiVA.org:hh-42014DiVA, id: diva2:1428859
Conference
The 24th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD2020), Singapore, May 11-14, 2020
Funder
Knowledge Foundation
Note

This research has been conducted within the BIDAF: A Big Data Analytics Framework for a Smart Society (http://bidaf.sics.se/).

Available from: 2020-05-06 Created: 2020-05-06 Last updated: 2023-10-05Bibliographically approved

Open Access in DiVA

decentralized kmeans(1345 kB)170 downloads
File information
File name FULLTEXT01.pdfFile size 1345 kBChecksum SHA-512
97647ba3be9c89e474e71aba8ea915be3786dd12b07a7fb051ac76146d24a6ff708c1a5c5d9fefdb2b0eec650f775acecab0bec0f138e79873889d3e858cec99
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopus

Authority records

Bouguelia, Mohamed-RafikPashami, SepidehNowaczyk, Sławomir

Search in DiVA

By author/editor
Girdzijauskas, SarunasBouguelia, Mohamed-RafikPashami, SepidehNowaczyk, Sławomir
By organisation
CAISR - Center for Applied Intelligent Systems Research
Computer Systems

Search outside of DiVA

GoogleGoogle Scholar
Total: 170 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 160 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf