The accurate prediction of the viscosity and shear stress for magnetorheological (MR) fluids provides the basis for the preparation of MR fluids and the design of MR devices, where a proper model is a key. How to quickly choose an appropriate model to describe its rheological properties is important for the application of MR fluids. In this work, MR fluids with different mass fractions are prepared and their apparent viscosities and shear stresses are measured under different magnetic fields and shear rates. The obtained rheological parameters are fitted by the Bingham model, the Herschel–Bulkley model, and the Casson model to evaluate their fitting effects. It is found that the MR fluid can be regarded as a Newtonian fluid only if its mass fraction is less than 30 wt%, and there is no magnetic field. The shear thinning effect is the main cause of the constitutive model errors, and it is more likely to occur for the MR fluids with higher mass fractions but increasing magnetic field strength will inhibit it. However, the shear thinning effect caused by the high-rate shear is even stronger than the inhibition through increasing magnetic field strength. The highest shear yield stress is obtained with the Bingham model, followed by the Carson model, and finally the Herschel–Bulkley model. This work provides a guidance to accurately predict the shear stress of MR fluids through a suitable constitutive model.