Series elastic actuators (SEA) with their inherent compliance offer a safe torque source for robots that are interacting with various environments, including humans. These applications have high requirements for the SEA torque controllers, both in the torque response as well as interaction behavior with its environment. To differentiate state of the art torque controllers, this work introduces a unifying theoretical and experimental framework that compares controllers based on their torque transfer behavior, their apparent impedance behavior, and especially the passivity of the apparent impedance (i.e., their interaction stability) as well as their sensitivity to sensor noise. We compare classical SEA control approaches such as cascaded PID controllers and full state feedback controllers with advanced controllers using disturbance observers, acceleration feedback and adaptation rules. Simulations and experiments demonstrate the trade-off between stable interactions, high bandwidths and low noise levels. Based on these trade-offs, an application-specific controller can be designed and tuned, based on desired interaction with the respective environment.