With the development of smart home, the demand for digital modeling and visualization of indoor space is increasing. To address the demand for indoor display optimization, the study combines WebGL and Unity3D to develop an indoor 3D visualization system. On this basis, in order to effectively annotate the layout relationship of indoor scenes, a coarse-to-fine spatial layout estimation method is proposed, and then the NSGA-II multi-objective genetic algorithm is applied to solve the spatial layout optimization problem of the dormitory with the effective active space utilization rate as the layout optimization index. Experiments show that, relative to other models, the indoor spatial layout estimation model in this paper has advantages in mIoU, MPA and PE global indexes, with results of 78.38%, 87.52%, 7.33% and 79.56%, 87.53%, 8.57% in the LSUN and Hedau datasets, and has good estimation efficiency. The performance of the indoor display optimization model on the dataset is also significantly better than the comparison method, which reflects the superiority of the model in the rationality of indoor space layout. The application of the method can effectively expand the indoor activity area, resulting in an effective activity space utilization rate of 56.3%, which is 1.2% to 4.0% higher than other methods. The proposed method can effectively solve the indoor space layout problem and realize the functional requirements of the 3D visualization system.
