| 431 | 2 | 72 |
| Downloads | Citas | Reads |
In view of the clients′ limited data resources involved in federated learning in edge computing environment, and the problem that it was difficult to further improve the accuracy of edge federated learning algorithm which used hard label knowledge to train the model, an edge federated learning algorithm based on knowledge distillation was proposed. The extraction of soft label information by knowledge distillation could effectively improve the performance of the model, so the knowledge distillation technology was introduced into the model training of federated learning. In each round of federated learning model training process, the client uploaded the model parameters and samples logic values to the edge server, and the server generated the global model and global soft label together and sent them to the client for the next round of learning, so that the client could also get the guidance of global soft label knowledge during local training. At the same time, a dynamic adjustment mechanism was designed for the proportion of soft label knowledge and hard label knowledge in model training, so that the knowledge of both could be reasonably used to guide model training in federated learning. The experimental results verified that the proposed edge federated learning algorithm based on knowledge distillation could effectively improve the accuracy of the model.
[1] 魏明军，闫旭文，纪占林，等.基于CNN与LightGBM的入侵检测研究[J].郑州大学学报(理学版),2023,55(6):35-40.WEI M J,YAN X W,JI Z L,et al.Research on intrusion detection based on CNN and LightGBM[J].Journal of Zhengzhou university (natural science edition),2023,55(6):35-40.
[2] 吴宇鑫，陈知明，李建军.基于半监督深度学习网络的水体分割方法[J].郑州大学学报(理学版),2023,55(6):29-34.WU Y X,CHEN Z M,LI J J.Semi-supervised deep learning network based water body segmentation method[J].Journal of Zhengzhou university (natural science edition),2023,55(6):29-34.
[3] BONAWITZ K,EICHNER H,GRIESKAMP W,et al.Towards federated learning at scale:system design [EB/OL].(2019-03-22) [2023-04-28].https://arxiv.org/abs/1902.01046.
[4] WANG X F,HAN Y W,WANG C Y,et al.In-edge AI:intelligentizing mobile edge computing,caching and communication by federated learning[J].IEEE network,2019,33(5):156-165.
[5] ZHANG C,XIE Y,BAI H,et al.A survey on federated learning[J].Knowledge-based systems,2021,216:106775.
[6] YU B,MAO W J,LV Y H,et al.A survey on federated learning in data mining[J].WIREs data mining and knowledge discovery,2022,12(1):e1443.
[7] SPRAGUE M R,JALALIRAD A,SCAVUZZO M,et al.Asynchronous federated learning for geospatial applications[C]//Joint European Conference on Machine Learning and Knowledge Discovery in Databases.Cham:Springer International Publishing,2019:21-28.
[8] WANG S Q,TUOR T,SALONIDIS T,et al.Adaptive federated learning in resource constrained edge computing systems[J].IEEE journal on selected areas in communications,2019,37(6):1205-1221.
[9] WANG L P,WANG W,LI B.CMFL:mitigating communication overhead for federated learning[C]//2019 IEEE 39th International Conference on Distributed Computing Systems.Piscataway:IEEE Press,2019:954-964.
[10] LI T,SANJABI M,SMITH V.Fair resource allocation in federated learning [EB/OL].(2020-02-14) [2023-04-28].https://arxiv.org/abs/1905.10497v1.
[11] YOSHIDA N,NISHIO T,MORIKURA M,et al.Hybrid-FL for wireless networks:cooperative learning mechanism using non-IID data[C].(2019-05-17)[2023-04-28].https://arxiv.org/pdf/1905.07210V2.pdf.
[12] ZHAO Y,LI M,LAI L Z,et al.Federated learning with non-iid data [EB/OL].(2022-07-21) [2023-04-28].https://arxiv.org/abs/1806.00582.
[13] YUAN P Y,ZHAO X Y,CHANG B F,et al.COPO:a context aware and posterior caching scheme in mobile edge computing[C]//2019 IEEE International Conference on Signal Processing,Communications and Computing.Piscataway:IEEE Press,2019:1-5.
[14] YUAN P Y,CAI Y Y.Contact ratio aware mobile edge computing for content offloading[C]//IEEE International Conference on Parallel and Distributed Systems.Piscataway:IEEE Press,2019:520-524.
[15] LIU J,HUANG J Z,ZHOU Y,et al.From distributed machine learning to federated learning:a survey[J].Knowledge and information systems,2022,64(4):885-917.
[16] HUANG W K,YE M,DU B.Learn from others and be yourself in heterogeneous federated learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE Press,2022:10143-10153.
[17] CHAI Z,FAYYAZ H,FAYYAZ Z,et al.Towards taming the resource and data heterogeneity in federated learning[C]//2019 USENIX Conference on Operational Machine Learning.Berkeley:USENIX Association Press,2019:19-21.
[18] MCMAHAN H B,MOORE E,RAMAGE D,et al.Communication-efficient learning of deep networks from decentralized data[EB/OL].(2016-02-17) [2023-04-28].https://arxiv.org/abs/1602.05629.
[19] LI T,SAHU A K,ZAHEER M,et al.Federated optimization in heterogeneous networks[EB/OL].(2019-04-21) [2023-04-28].https://arxiv.org/abs/1812.06127.
Basic Information:
DOI：10.13705/j.issn.1671-6841.2023158
China Classification Code:TP181
Citation Information:
[1]SHI Ling,HE Changle,CHANG Baofang ,et al.An Edge Federated Learning Algorithm Based on Knowledge Distillation[J].Journal of Zhengzhou University(Natural Science Edition),2025,57(02):44-50.DOI:10.13705/j.issn.1671-6841.2023158.
Fund Information:
国家自然科学基金项目(62072159,U1804164)
2024-03-14
2024-03-14
2024-03-14