A detailed study is presented of the spectacular 'Manhattan switching', occurring during hydriding of epitaxial YH_x switchable mirrors. Manhattan switching involves block-wise domain switching, both in an optical and a structural sense, of discrete areas of a switchable mirror between its conductive metallic YH₂ state and its insulating YH₃ state. By means of in situ atomic force microscopy we link this domain-wise switching to the accompanying changes in resistivity and transmission of a switching mirror. We find that, relative to the optical transition and the resistivity, the structural switching is retarded compared to polycrystalline films. Further, the Manhattan effect is governed by the switching properties of the domain boundaries, that form a regular, connected network of ridges. This crucial role of the network can well be explained by assuming local variations of mechanical properties around each ridge. The findings stress the importance of local investigations on switchable mirrors.