We theoretically investigate pumping of phonons by the dynamics of a magnetic film into a nonmagnetic contact. The enhanced damping due to the loss of energy and angular momentum shows interference patterns as a function of the resonance frequency and magnetic film thickness that cannot be described by viscous ("Gilbert") damping. The phonon pumping depends on the magnetization direction as well as geometrical and material parameters and is observable, e.g., in thin films of yttrium iron garnet on a thick dielectric substrate.