Abstract: The recognition of unsound wheat kernels is an important part of wheat quality inspection, and it is also a key indicator to measure wheat quality. Research on the recognition of unsound wheat kernels is of great significance to the correct evaluation of wheat quality. In recent years, researches on the recognition of unsound wheat kernels were mainly to directly optimize classical classification networks, and the problem of unsatisfactory recognition effect was often caused by insufficient wheat training dataset. Aiming at the problem of poor recognition rate due to insufficient recognition data of unsound wheat kernels in practical application scenarios, we designed and implemented a method for identifying unsound wheat kernels based on transfer learning CNN-DBN model. We used a jitter-type automatic feeding tray, a transparent carrier plate and a pair of high-definition industrial cameras to form an automatic wheat image acquisition device to obtain high-definition RGB images of wheat. Using the deep convolutional neural networks VGG-16, VGG-19 and ResNet50 pre-trained on the ImageNet dataset as feature extractors, the extracted wheat features were fused, and then classified by the deep belief network (DBN). We conducted experiments of single model, two-model fusion and three-model fusion, based on the performance of the three types of fusion models on the dataset, and finally chose the three-model fusion scheme with the best performance on the dataset. Meanwhile, the recognition results of each model using the softmax classifier and DBN as the classifier were compared. Finally, the transfer learning VGG-16+VGG-19 +ResNet50-DBN model was selected as the model for this task. Experiments showed that the transfer learning VGG-16+VGG-19+ResNet50-DBN model was used to identify unsound wheat kernels, and its accuracy can reach 91.86%. The pre-trained models we use were trained on the large-scale dataset ImageNet. Because the scale of the ImageNet dataset was very large, these pre-trained models also had good generalization performance for images outside the ImageNet dataset. Therefore, although our dataset was small, using the pre-trained models of the ImageNet dataset as the wheat feature extractor can still achieve a good recognition result. The proposed method not only avoided complicated feature extraction steps, but also improved the problem of unsatisfactory recognition rate due to the small size of the dataset in the recognition of unsound wheat kernels. The feature fusion method made the extracted wheat image information more abundant and comprehensive. The CNN-DBN model combined the advantages of supervised and unsupervised method, and had better classification capabilities for high-dimensional data. This work provides a theoretical support for the recognition of unsound wheat kernels.