To apply abrasive slurry jet (ASJ) in rock drilling calls for in-depth understandings on the mechanism of impingement process. A fluid–structure coupling model that combines multi-fluid model and finite element method was established for a set of numerical investigation. The simulation results show that entrained air contributes to the divergence by successive disturbing on the jet flow. Subjected to the jet impact, the stress field of rock structure is affected by the standoff distance, abrasive particle diameter and crack in rock material. On the other hand, we adopt both first strength criterion and Drucker–Prager criterion for the elastic–plastic rock material to evaluate the stress state of rock under a high-speed jet impact. It is found the onset of rock damage is primarily attributed to a brittle tensile fracture at crack tip and a plastic failure in the crack front, tending to initiate from a smaller-length-scale crack. The flow field and rock stress are characterized to provide a numerical basis for the parametric optimization of ASJ impinging rock as well as other elastic–plastic materials.