It is an important problem that the increase of the performance of radar anti-jamming for a long time in radar signal processing. Separation convolutive mixture of source signals is a challeng of blind source separation (BSS). In this paper, the algorithms of convolative mixtare BSS is used to separate the received signals of the radar to the jamming and the useful signal and realize the aim of radar anti-jamming. A new algorithm for convolutive mixture BSS by using the Householder transformation to realize the joint diagonalization of high-order cumulant and operation in the frequency domain is proposed. Using the algorithms of convolutive BSS the received signals of radar systems are separated to the radar signals and the jamming. Simulation results show that the novel method of radar anti-jamming can realize the separating of radar signals and the jamming and have the better performance of separating.
