An early contamination warning system with deployed water quality sensors is often used to enhance the safety of a water distribution system (WDS). While algorithms have been developed to select an optimal water quality sensor placement strategy (WQSPS) for WDSs, many of them do not account for the influences caused by future uncertainties, such as sensor failures and system changes (e.g., demand variations and configuration/expansion changes in the WDS). To this end, this paper proposes a comprehensive framework to evaluate the robustness of WQSPSs to these possible uncertainties. This is achieved by considering five different performance objectives of WQSPSs as well as possible future demand and typology variations of WDSs under a wide range of sensor failure scenarios. More specifically, an optimization problem is formulated to evaluate the robustness of the WQSPSs, in which an evolutionary-based optimization approach coupled with an efficient data-archive method is used to solve this optimization problem. The framework is demonstrated on two real-world WDSs in China. The results show that: (1) the WQSPS's robustness can be highly dependent on the performance objectives considered, implying that an appropriate objective needs to be carefully selected for each case driven by practical needs, (2) the WDS's demand and configuration changes can have a significant influence on the WQSPS's robustness, in which the solution with more sensors in or close to the affected area is likely to better cope with these system changes, and (3) the proposed framework enables critical sensors to be identified, which can then be targeted for prioritizing maintenance actions.