Computer Applications and Software

CLC number: TP391.41

An Intelligent Sorting System of Coal-Gangue Based on Machine Vision and Convolutional Neural Network

一种基于机器视觉与卷积神经网络的煤矸智能分拣系统

¹ College of Physics and Electronic Engineering, Huaibei Normal University, Huaibei 235000, Jiangsu, China (淮北师范大学　物理与电子工程学院，安徽　淮北　235000)
² Anhui Province Key Laboratory of Intelligent Computing and Applications, Huaibei Normal University, Huaibei 235000, Jiangsu, China (淮北师范大学　智能计算及应用安徽省重点实验室，安徽　淮北　235000)
³ Internet of Things (Sensing Mine) Research Center, China University of Mining and Technology, Xuzhou 221116, Jiangsu, China (中国矿业大学　物联网（感知矿山）研究中心，江苏　徐州　221116)

^† Corresponding author. E-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.

Received: 25 March 2025

Abstract

Gangue is inevitably mixed with coal during mining and transportation. Currently, the manual sorting and conventional mechanical separation technologies widely adopted in the coal mining industry are plagued by low efficiency, poor identification accuracy, severe environmental pollution, and other drawbacks. This paper proposes a machine vision-based intelligent coal gangue sorting robot system. Firstly, the OpenMV captures images of coal gangue and utilizes the MobileNetV2 0.35 lightweight convolutional neural network to train the FOMO (Faster Objects, More Objects) target detection model in the cloud. This enables prediction and recognition of gangue, along with the acquisition of its center point pixel coordinates. Secondly, the position information of the gangue is sent to the STM32 microcontroller using the serial communication protocol for coordinate system conversion, pose algorithm, and path planning. Finally, the STM32 microcontroller controls the start and stop of the conveyor belt through the working status of the relay. When the relay is absorbed, the conveyor belt stops, and at the same time, the robotic arm grasps the gangue for transfer action, thus realizing the sorting of coal and gangue. The experimental results demonstrate that the cloud-trained FOMO neural network model achieves an F₁ score of 95.5% and a recall of 91.3%, with a test accuracy of 97.56%. The quantified model deployed on OpenMV can accurately identify multiple gangues and output their position information. The success rate of the robotic arm in tracking and sorting gangue reaches 90.13%, and the positioning error of the robotic arm is [9,12.5] mm. This system realizes high-precision identification, positioning, and intelligent sorting of coal and gangue, meeting the basic requirements for gangue sorting in coal mines.

摘要

煤矿开采和运输煤炭过程中通常掺杂矸石，当前煤矿行业常用的人工和传统机械分拣方式普遍存在效率低下、识别精度低、环境污染严重等问题。本文提出了一种基于机器视觉的智能煤矸分拣机器人系统。首先，通过OpenMV采集煤矸图像，利用MobileNetV2 0.35轻量级卷积神经网络在云端训练FOMO(Faster Objects， More Objects)目标检测模型，对矸石进行预测识别并获取中心点像素坐标；其次，采用串口通信的协议方式将矸石位置信息发送到STM32单片机，进行坐标系转换、姿态解算和路径规划；最后，STM32单片机通过继电器的工作状态控制传送带启停，当继电器吸合时传送带停止，同时机械臂对矸石进行抓取转运动作，实现煤和矸石的分拣。实验结果表明：云端训练FOMO神经网络模型得出F₁分数为95.5%，召回率91.3%，模型测试结果准确度为97.56%，部署到OpenMV中可以较为准确地识别出多个矸石并且输出其位置信息，机械臂跟踪分拣矸石的成功率达90.13%，机械臂定位误差在［9，12.5］mm，实现煤和矸石的高精度识别、定位和智能分选，满足煤矿煤矸分拣的基本要求。