Microstructure classification method based on multi-scale feature fusion

doi:10.13251/j.issn.0254-6051.2025.05.045

Abstract

Abstract: To address the problems of low manual efficiency, susceptibility to subjective factors, and limited automatic classification and recognition categories in traditional microstructure quantitative analysis, a microstructure classification method based on Res2net50 multi-scale feature fusion was proposed. Firstly, an adaptive multi-scale attention enhancement module was introduced to strengthen the correlation between microstructure images at different magnifications, ensuring that the network effectively captures relevant information across scales. Secondly, to strengthen the extraction of texture features, a TripletAttention module was introduced in the residual structure of the last layer of the network, thus enhancing the attention to details. Finally, the multi-scale feature fusion module was used to dynamically adjust the multi-scale feature fusion, so that the network captures more effective features. The experimental results show that the method proposed in the study is able to accurately classify the metallurgical structure of single-phase, two-phase, three-phase, and four-phase in a total of 20 categories, with an accuracy rate of 95.53%. Compared with the original Res2net50 model, the accuracy is improved by 5.28%, the F1-Score is improved by 6.86%, and the recall rate of three-phase and four-phase structure are significantly improved. These results validate the strong discriminative power of the method for complex multi-phase structures.

Key words: microstructure, multi-scale feature fusion, intelligent recognition, multi-scale attention

CLC Number:

TF31

Huang Xiaohong, Zhang Luyue, Song Yue, Zhang Qingjun. Microstructure classification method based on multi-scale feature fusion[J]. Heat Treatment of Metals, 2025, 50(5): 284-292.

References

[1] Li M C, Chen D L, Liu S X, et al. Online learning method based on support vector machine for metallographic image segmentation[J]. Signal Image and Video Processing, 2021, 15: 571-578.
[2] 段献宝, 何惠珍, 李平平, 等. 基于深度卷积神经网络的钢材微观组织分类识别[J]. 铁道车辆, 2022, 60(1): 43-47.
Duan Xianbao, He Huizhen, Li Pingping, et al. Classification and identification of steel microstructure based on deep convolution neural network[J]. Rolling Stock, 2022, 60(1): 43-47.
[3] Azimi S M, Britz D, Engstler M, et al. Advanced steel microstructural classification by deep learning methods[J]. Scientific Reports, 2018, 8(1): 21-28.
[4] Gola J, Webel J, Britz D, et al. Objective microstructure classification by support vector machine (SVM) using a combination of morphological parameters and textural features for low carbon steels[J]. Computational Materials Science, 2019, 160: 186-196.
[5] Tsutsui K, Terasaki H, Uto K, et al. A methodology of steel microstructure recognition using SEM images by machine learning based on textural analysis[J]. Materials Today Communications, 2020, 25(12): 101514.
[6] 孙攀, 石秀东, 何英杰. 基于特征融合和GWO-SVM的铸铁金相组织分类技术研究[J]. 制造业自动化, 2023, 45(4): 34-38.
Sun Pan, Shi Xiudong, He Yingjie. Metallographic classification of cast iron based on feature fusion and GWO-SVM[J]. Manufacturing Automation, 2023, 45(4): 34-38.
[7] 胡名琪, 刘秋明, 陈辉明, 等. 基于改进残差网络与迁移学习的铜合金金相图分类方法[J]. 有色金属科学与工程, 2024, 15(5): 690-699.
Hu Mingqi, Liu Qiuming, Chen Huiming, et al. Classification method of copper alloy metallographic diagram based on improved residual network and transfer learning[J]. Nonferrous Metals Science and Engineering, 2024, 15(5): 690-699.
[8] 张艳飞, 张永志, 公维炜, 等. 基于GoogLeNet Inception V3模型的火电用钢金相组织智能识别[J]. 材料导报, 2024, 38(17): 247-253.
Zhang Yanfei, Zhang Yongzhi, Gong Weiwei, et al. Intelligent recognition of microstructure of steel in thermal power unit based on GoogLeNet Inception V3 model[J]. Materials Reports, 2024, 38(17): 247-253.
[9] 黄学雨, 贺怀宇, 林慧敏, 等. 基于特征聚合的铜合金金相图分类识别方法[J]. 计算机应用, 2023, 43(8): 2593-2601.
Huang Xueyu, He Huaiyu, Lin Huimin, et al. Classification and recognition method of copper alloy metallograph based on feature aggregation[J]. Journal of Computer Applications, 2023, 43(8): 2593-2601.
[10] 李杰, 付冬梅. 金相图像均衡化和多尺度模板二值降噪方法研究[J]. 铸造技术, 2018, 39(11): 2433-2439.
Li Jie, Fu Dongmei. Equilibrium of metallographic image and the research of multiscale template binary noise reduction[J]. Foundry Technology, 2018, 39(11): 2433-2439
[11] 曾培益. 基于增量学习的多尺度钢材微观组织图像分类[J]. 计算机科学, 2024, 51(S1): 302-309.
Zeng Peiyi. Classification of multiscale steel microstructure images based on incremental learning[J]. Computer Science, 2024, 51(S1): 302-309.
[12] Gao S H, Cheng M M, Zhao K, et al. Res2net: A new multi-scale backbone architecture[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 43(2): 652-662.
[13] 朱丽, 王新鹏, 付海涛, 等. 基于注意力机制的细粒度图像分类[J]. 吉林大学学报(理学版), 2023, 61(2): 371-376.
Zhu Li, Wang Xinpeng, Fu Haitao, et al. Fine-grained image classification based on attention mechanism[J]. Journal of Jilin University (Science Edition), 2023, 61(2): 371-376.
[14] 李云红, 郭越, 谢蓉蓉, 等. 注意力机制和多尺度特征融合的细粒度图像分类[J]. 重庆理工大学学报(自然科学), 2024, 38(12): 155-164.
Li Yunhong, Guo Yue, Xie Rongrong, et al. Attentional mechanisms and multiscale feature fusion for fine-grained image classification[J]. Journal of Chongqing University of Technology(Natural Science), 2024, 38(12): 155-164.
[15] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 770-778.
[16] Li Y, Li X, Yang J. Spatial group-wise enhance: Enhancing semantic feature learning in CNN[C]//16th Asian Conference on Computer Vision, Proceedings, Part V. 2022: 687-702.
[17] 陈德阳, 唐智, 何牧耕. 基于OCR-UNet的金属表面缺陷分割[J]. 组合机床与自动化加工技术, 2023(11): 169-173.
Chen Deyang, Tang Zhi, He Mugeng. Segmentation of metal surface defect based on OCR-UNet[J]. Modular Machine Tool and Automatic Manufacturing Technique, 2023(11): 169-173.
[18] Song Y, He Z, Qian H, et al. Vision transformers for single image dehazing[J]. IEEE Transactions on Image Processing, 2023, 32: 1927-1941.
[19] Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 7132-7141.
[20] Misra D, Nalamada T, Arasanipalai A U, et al. Rotate to attend: Convolutional triplet attention module[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2021: 3139-3148.
[21] 张永志, 辛全忠, 王永亮, 等. 基于迁移学习的钢金相组织分类与识别方法的研究[J]. 材料导报, 2021, 35(24): 24152-24157.
Zhang Yongzhi, Xin Quanzhong, Wang Yongliang, et al. Research on classification and recognition method of steel metallographic structure based on transfer learning[J]. Materials Reports, 2021, 35(24): 24152-24157.