With the increasing complexity of financial markets, traditional analytical tools are overwhelmed by the amount of information available and the ever-changing markets. Quantitative investment utilizes mathematical models and statistical methods to organize past data, which to a certain extent can grasp the timing of trading and avoid investors’ sentimentality and blindness. Convolutional neural networks were initially widely used in the field of computer vision due to their powerful image feature extraction and recognition capabilities, and have gradually gained popularity among financial researchers. The complex nonlinear and potential relationships hidden in financial market sequences are difficult to be described by traditional linear models, and the convolutional neural network can automatically mine the local and global information embedded in the stock price, volume and technical indicators through its multilevelized feature extraction capability. In this paper, we constructed a portfolio strategy based on the combination of convolutional neural network and long and short-term memory network, used multi-dimensional financial market data as input features to train the model, and evaluated the model based on historical backtesting and out-of-sample testing. The model developed in this paper achieves an out-of-sample prediction accuracy of 75.3% for CSI 300 stocks, an annualized return of 26.8%, a Sharpe ratio of 1.84, and an information ratio of 1.28, which are all better than the results of traditional random forest and support vector machine methods; the results of this paper show that deep learning can be widely used in the field of quantitative investment, and to a certain extent, improve portfolio stability and return.