Covariance matrices usually exhibit specific spectral structures, such as low-rank ones in the case of factor models. In order to exploit this prior knowledge in a robust estimation process, we propose a new regularized version of Tyler's M-estimator of covariance matrix. This estimator is expressed as the minimizer of a robust M -estimating cost function plus a penalty that is unitary invariant (i.e., that only applies on the eigenvalue) that shrinks the estimated spectrum toward a fixed target. The structure of the estimate is expressed through an interpretable fixed-point equation. A majorization-minimization (MM) algorithm is derived to compute this estimator, and the g-convexity of the objective is also discussed. Several simulation studies illustrate the interest of the approach and also explore a method to automatically choose the target spectrum through an auxiliary estimator.