Unveiling Complex Scenes: A Deep Belief Network and Semantic Segmentation Approach

Abstract

Scene classification is a meaningful and challenging research field in computer vision due to the wide variety of objects present in a scene, their internal relationships, and inter-class similarities. Thus, complex scene recognition and understanding are needed in various applications, including virtual reality-based scene integration, robotics, autonomous driving, and tourist guide systems. Therefore, a novel scene recognition system that integrates various components to recognize the scenes in complex imagery is developed. Compared to the state-of-art systems, our system combines many significant features for improving the classification accuracy. Initially, the images are acquired and preprocessed. It is worth mentioning that semantic segmentation approaches are powerful as they not only detect objects present in an image but recognize the boundaries of each object. To leverage the effectiveness of semantic segmentation, we propose a modified fuzzy C-means (MFCM) segmentation method that partitions the image into various objects to label the pixels according to different segmented objects. Then, convolutional neural network (CNN) features, the dynamic geometrical (GF), and, blob features (BF) are extracted and fused for further analysis to recognize the scene through a deep belief network (DBN). The latter incorporates a genetic algorithm that optimizes the number of hidden units based on the error rate and the training time. The effectiveness of the proposed system is validated over Pattern Analysis, Statistical Modeling, and Computational Learning Visual Object Classes (PASCAL VOC 2012) and the Microsoft Research Cambridge (MSRC) datasets by achieving 93.30% and 92.53% recognition accuracies respectively.