Before the rise of large machine learning algorithms, most people manually adjusted the super parameters of the model by relying on experience. However, with the increasing complexity of the model, this method obviously cannot meet the needs. This paper mainly studies the theory and practice of super parameter optimization of machine learning algorithm. This thesis proposes a regression-based hyperparameter optimization algorithm that has the same data-based optimization algorithm as the optimization algorithm Bayesian. The optimization algorithm is based on the Gaussian regression process. In addition to being affected by the super parameters of the kernel function in the process of GP regression fitting, the calculation amount of the algorithm will also increase significantly. The experimental results show that, compared to the optimization algorithm, the parameter optimization results of this algorithm are similar to those of the optimization algorithm.

Lijuan Shan. Theory and Practice of Super Parameter Optimization for Machine Learning Algorithm. Machine Learning Theory and Practice (2020), Vol. 1, Issue 2: 37-44. https://doi.org/10.38007/ML.2020.010205.

[1] Yousefi A , Pishvaee M S . A hybrid machine learning-optimization approach to pricing and train formation problem under demand uncertainty. RAIRO - Operations Research, 2020, 56(3):1429-1451. 

[2]  Ramaiah N S , Ahmed S T . An IoT Based Treatment Optimization and Priority Assignment Using Machine Learning. ECS transactions, 2020(1):107. https://doi.org/10.1149/10701.1487ecst

[3] Durand A, Wiesner T, Gardner M A, et al. A machine learning approach for online automated optimization of super-resolution optical microscopy. Nature communications, 2018, 9(1): 1-16. https://doi.org/10.1038/s41467-018-07668-y

[4] Genty G, Salmela L, Dudley J M, et al. Machine learning and applications in ultrafast photonics. Nature Photonics, 2020, 15(2): 91-101. https://doi.org/10.1038/s41566-020-00716-4

[5] Yu M, Yang S, Wu C, et al. Machine learning the Hubbard U parameter in DFT+ U using Bayesian optimization. npj Computational Materials, 2020, 6(1): 1-6. https://doi.org/10.1038/s41524-020-00446-9

[6] Fukami K, Fukagata K, Taira K. Assessment of supervised machine learning methods for fluid flows. Theoretical and Computational Fluid Dynamics, 2020, 34(4): 497-519. https://doi.org/10.1007/s00162-020-00518-y

[7] Owoyele O, Pal P, Vidal Torreira A, et al. Application of an automated machine learning-genetic algorithm (AutoML-GA) coupled with computational fluid dynamics simulations for rapid engine design optimization. International Journal of Engine Research, 2020, 23(9): 1586-1601. https://doi.org/10.1177/14680874211023466

[8] Karthikeyan R, Alli P. Feature selection and parameters optimization of support vector machines based on hybrid glowworm swarm optimization for classification of diabetic retinopathy. Journal of medical systems, 2018, 42(10): 1-11. https://doi.org/10.1007/s10916-018-1055-x

[9] Nain S S, Garg D, Kumar S. Performance evaluation of the WEDM process of aeronautics super alloy. Materials and Manufacturing Processes, 2018, 33(16): 1793-1808. https://doi.org/10.1080/10426914.2018.1476761

[10] Chugh S, Ghosh S, Gulistan A, et al. Machine learning regression approach to the nanophotonic waveguide analyses. Journal of Lightwave Technology, 2019, 37(24): 6080-6089. https://doi.org/10.1109/JLT.2019.2946572

[11] Rittenhouse K J, Vwalika B, Keil A, et al. Improving preterm newborn identification in low-resource settings with machine learning. PLoS One, 2019, 14(2): e0198919. https://doi.org/10.1371/journal.pone.0198919

[12] Berrar D, Lopes P, Dubitzky W. Incorporating domain knowledge in machine learning for soccer outcome prediction. Machine learning, 2019, 108(1): 97-126. https://doi.org/10.1007/s10994-018-5747-8

[13] Kim T, Moon S, Xu K. Information-rich localization microscopy through machine learning. Nature communications, 2019, 10(1): 1-8. https://doi.org/10.1038/s41467-019-10036-z

[14] Klinkowski M, Ksieniewicz P, Jaworski M, et al. Machine learning assisted optimization of dynamic crosstalk-aware spectrally-spatially flexible optical networks. Journal of Lightwave Technology, 2020, 38(7): 1625-1635. https://doi.org/10.1109/JLT.2020.2967087

[15] Culos A, Tsai A S, Stanley N, et al. Integration of mechanistic immunological knowledge into a machine learning pipeline improves predictions. Nature machine intelligence, 2020, 2(10): 619-628. https://doi.org/10.1038/s42256-020-00232-8

[16] Currie G. Intelligent imaging: anatomy of machine learning and deep learning. Journal of nuclear medicine technology, 2019, 47(4): 273-281. https://doi.org/10.2967/jnmt.119.232470

[17]  Tyralis H, Papacharalampous G, Langousis A. Super ensemble learning for daily streamflow forecasting: Large-scale demonstration and comparison with multiple machine learning algorithms. Neural Computing and Applications, 2020, 33(8): 3053-3068. https://doi.org/10.1007/s00521-020-05172-3

[18] Banadkooki F B, Ehteram M, Ahmed A N, et al. Enhancement of groundwater-level prediction using an integrated machine learning model optimized by whale algorithm. Natural resources research, 2020, 29(5): 3233-3252. https://doi.org/10.1007/s11053-020-09634-2