Retrogressive failures occur in slopes consisting of sensitive materials such as snow or quick clay. They can be triggered by a small disturbance at the slope toe, but can cause propagated failure spreading miles away. Understanding the physical mechanism and predicting the retrogressive failure process are particularly important. Previous studies have discussed the failure criteria, the soil properties or the method of numerical modeling of retrogressive slope failure. However, little attention has been paid to the microscopic failure mechanism, especially relating to various possible failure patterns. In this study, multiscale modeling is incorporated to study the physical mechanism of different retrogressive failure patterns, including earth flow, flowslide and spread failure, within a unified framework. Utilizing multiscale analysis, we found that earth flow failure is related to the shear failure of granular materials. In contrast, the development of macroscopic shear bands is accompanied by tensile failure. As shear and tension failures are typical failure mechanisms of frictional and cohesive materials, it is deduced that friction and cohesion effects play key roles in different retrogressive failure patterns. Therefore, the distributions of attractive and repulsive contact forces are explored and a novel parameter η is proposed to quantify the interplay between friction and cohesion. Further analysis proves that η can capture the effect of friction and cohesion and distinguish different retrogressive failure patterns. Finally, a spectrum of retrogressive failures for a granular slope is established, in which the failure mechanism is explained by the changeable dominant effect, that is, frictional or cohesive in soil.