This work gives a method which can use both motion and shape information for drinking action detection. Drinking action is differ from the other gestures. A complete drinking action duration could be divided into 3 different stages. Motion-based features and shape-based features are complement with each other in each stage. By using this feature, we have defined a fusion strategy that can use both the advantage of the strength of each part at a different stage.