Rocket image target recognition and tracking is an important application field of image target recognition，which is an important support for rocket test launch and flight control，and has great significance for rocket target tracking and attitude analysis and control.Image tracking of rocket target in ascending stage is an important stage of rocket flight measurement and control，but at present，the video image tracking of rocket in ascending stage mainly relies on manual operation of pinion controller to achieve rocket tracking in the image.The tracking image has some phenomena such as tracking lag and picture shaking，and the tracking effect is greatly affected by human factors.Combining full convolution theory with deep learning method，a method of rocket image target recognition and tracking based on full convolution deep neural network was proposed.Images of rocket launch and ascending flight were collected as samples，and a full convolutional network model was constructed and trained.An end-to-end semantic segmentation method was adopted to realize semantic judgment of rocket targets at pixel level on the basis of deep classification network，with good recognition rate and robustness.Based on the recognition of the rocket target，the PTZ control model was established，the high-quality image of the rocket ascent stage was obtained through the intelligent control of the PTZ，and the tracking of the rocket target was realized.