Issue |
Wuhan Univ. J. Nat. Sci.
Volume 27, Number 6, December 2022
|
|
---|---|---|
Page(s) | 499 - 507 | |
DOI | https://doi.org/10.1051/wujns/2022276499 | |
Published online | 10 January 2023 |
CLC number: TP 399
A Federated Domain Adaptation Algorithm Based on Knowledge Distillation and Contrastive Learning
School of Electrical and Electronic Engineering,Shanghai University of Engineering Science, Shanghai 201600, China
† To whom correspondence should be addressed. E-mail: zjfang@sues.edu.cn
Received:
18
September
2022
Smart manufacturing suffers from the heterogeneity of local data distribution across parties, mutual information silos and lack of privacy protection in the process of industry chain collaboration. To address these problems, we propose a federated domain adaptation algorithm based on knowledge distillation and contrastive learning. Knowledge distillation is used to extract transferable integration knowledge from the different source domains and the quality of the extracted integration knowledge is used to assign reasonable weights to each source domain. A more rational weighted average aggregation is used in the aggregation phase of the center server to optimize the global model, while the local model of the source domain is trained with the help of contrastive learning to constrain the local model optimum towards the global model optimum, mitigating the inherent heterogeneity between local data. Our experiments are conducted on the largest domain adaptation dataset, and the results show that compared with other traditional federated domain adaptation algorithms, the algorithm we proposed trains a more accurate model, requires fewer communication rounds, makes more effective use of imbalanced data in the industrial area, and protects data privacy.
Key words: federated learning / multi-source domain adaptation / knowledge distillation / contrastive learning
Biography: HUANG Fang, female, Master candidate, research direction: federated learning. E-mail: 2431835009@qq.com
Supported by the Scientific and Technological Innovation 2030—Major Project of "New Generation Artificial Intelligence" (2020AAA 0109300)
© Wuhan University 2022
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.