Computer Science

CLC number: O177.2

Deep Learning Mixed Hyper-Parameter Optimization Based on Improved Cuckoo Search Algorithm

基于改进布谷鸟搜索算法的深度学习混合超参数优化

¹ School of Computer and Artificial Intelligence, Hefei Normal University, Hefei 230601, Anhui, China
² College of Information Science and Technology, Dalian Maritime University, Dalian 116026, Liaoning, China

Received: 18 June 2024

Abstract

Deep learning algorithm is an effective data mining method and has been used in many fields to solve practical problems. However, the deep learning algorithms often contain some hyper-parameters which may be continuous, integer, or mixed, and are often given based on experience but largely affect the effectiveness of activity recognition. In order to adapt to different hyper-parameter optimization problems, our improved Cuckoo Search(CS) algorithm is proposed to optimize the mixed hyper-parameters in deep learning algorithm. The algorithm optimizes the hyper-parameters in the deep learning model robustly, and intelligently selects the combination of integer type and continuous hyper-parameters that make the model optimal. Then, the mixed hyper-parameter in Convolutional Neural Network (CNN), Long-Short-Term Memory(LSTM) and CNN-LSTM are optimized based on the methodology on the smart home activity recognition datasets. Results show that the methodology can improve the performance of the deep learning model and whether we are experienced or not, we can get a better deep learning model using our method.

摘要

深度学习算法是一种有效的数据挖掘方法，已在许多领域得到应用。然而，深度学习算法通常包含一些超参数，这些超参数可能是连续的、整数的或混合的，并且通常是基于经验给出的，在很大程度上影响了活动识别的有效性。为了适应不同的超参数优化问题，本文提出了改进的布谷鸟搜索(CS)算法来优化深度学习算法中的混合超参数。该算法鲁棒性地优化深度学习模型中的超参数，进而智能选择使模型最优的整型和连续超参数组合。在智能家居活动识别数据集上对CNN、LSTM和CNN-LSTM中的混合超参数进行了优化。结果表明，该方法可以提高深度学习模型的性能。无论是否有经验，都可以使用我们的方法获得更好的深度学习模型。