This paper presents a system for cooperatively manipulating large objects between a human and a robot. This physical interaction system is designed to handle, transport, or manipulate large objects of different shapes in cooperation with a human. Unique points are the bi-manual physical cooperation, the sequential characteristic of the cooperation including contact changes, and a novel architecture combining force interaction cues, interactive search-based planning, and online trajectory and motion generation. The resulting system implements a mixed initiative collaboration strategy, deferring to the human when his intentions are unclear, and driving the task once understood. This results in an easy and intuitive human-robot interaction. It is evaluated in simulations and on a bi-manual mobile robot with 32 degrees of freedom.