To solve the problems of slow convergence speed and low estimation accuracy of extended Kalman filter (EKF) for the complex nonlinear state estimation in bearing only tracking system, the square root cubature Kalman filter (SRCKF) is introduced. The SRCKF is a kind of sigma points filtering method. The numerical integration method based on cubature rule is used directly to calculate the mean and covariance of the nonlinear random function. The work of computing Jacobian matrix in EKF can be avoided and the computational efficiency is enhanced. Comparing with the traditional CKF, the symmetry and positive semi definiteness of the covariance matrix of state are guaranteed using the SRCKF and the numerical accuracy are improved as well as the robustness. Simulation results show that in the bearing only tracking system, compared with the traditional EKF, the estimation accuracy of SRCKF and UKF are improved. Meanwhile, the run efficiency of SRCKF is faster than UKF.