This article develops and tests a self-contained railway track monitoring system that fits in existing vehicles without the need for speed and load control. Combining a train-borne laser Doppler vibrometer and axle box accelerometers enables synchronized measurements of train-track response under operational conditions. Utilizing a GPS antenna and video camera, we propose the multisignal processing method to obtain train-track vibrations with train position and speed. Then, we fuse the multiple signals to extract an impact index and a resonance index and further propose an interpretable anomaly detection strategy. We test the system on an operational line at 20-60 km/h under different working conditions and verify the detection results using information from conventional technologies. The impact index peaks near joints and welds, and the resonance index yields a good correlation with the measured track geometry. The developed solution achieves the detection, localization, and quantification of surface and support anomalies in railway tracks.