User feedback plays a significant role in helping recommendation systems to make personalized and accurate predictions. Despite the fact that many methods of collecting user feedback have been proposed, little research exists that addresses both the breadth and depth of data collected. In this study, we incorporate personal reflection into traditional crowdsourcing tasks and investigate how it facilitates people's reflection on their usage of YouTube. We present a novel crowdsourcing approach with personal reflection integration based on several design principles, which allows participants to reflect on contextual factors and personal values of using YouTube through guided context recall and evaluations, therefore gathering deeper insights into people's preferences and behaviors on a large scale. We conducted a user study involving 20 participants and explored the insights generated from their textual answers and the role of design principles in the reflection process. This approach successfully enables multiple participants to conduct the study simultaneously, thereby reflecting on their watching behaviors and preferences. Based on the quantitative and qualitative analysis of the findings, we sum up the implications of this approach to provide guidance for the YouTube recommendation system and point to the directions for the design of similar studies in the future.