Dyadic Physical Activity Planning with a Virtual Coach: Using Reinforcement Learning to Select Persuasive Strategies

Abstract

Physical activity is one of the main factors that contribute to reducing the chance of chronic diseases such as cardiovascular disease, obesity, and depression, all while improving an individual’s health in general. While this is the case, the fact still remains that many adults across the world do not reach the minimum recommendations for physical activity. Setting physical activity goals is one of the most common approaches found in both self-help fitness apps, or medical interventions for increasing physical activity. The issue is that goal setting on its own cannot help people become more physically active, since, if they are not committed to reaching a goal, a person will just abandon it instead. Recent literature shows that creating plans for when to perform physical activity has the potential to help people commit to reaching their goal. Being committed to following the plan is therefore important for ultimately reaching the goal, but only creating the plan is not enough, since people might abandon it. Furthermore, dyadic planning, in which a helper aids the person in creating the plan, has produced even better results than individual planning. Thus, the aim of this work is to develop a virtual coach, which plays the role of the helper in dyadic planning and motivates people to commit themselves to following the plan, so that they can reach their physical activity goals. To facilitate this process, the virtual coach, named Jamie, operates based on reinforcement learning, giving it the ability to select the best persuasive strategy to use. It does so by taking into account the person's situation (opinions of the plan and of planning in general), as well as how these opinions might change based on what persuasive strategy the agent chooses to employ. The persuasive strategies considered were: proposing to make changes to the plan, explaining why planning is useful, identifying and dealing with barriers, and showing testimonials from other people who created plans and used them to reach their goals. Through an observational study, data for the reinforcement learning model was gathered, and a model was trained on the data. Analysing the data revealed that the choice of persuasive strategy is not crucial, as all of them had similar effects on the person's situation, and these effects were small to moderate. At the same time, we saw moderate differences when comparing the situation at the beginning and end of the conversation, indicating that the combined effect of multiple persuasive strategies is needed to change a person's situation. We also investigated the effect of including or excluding the person's situation from the model. If the person's situation is disregarded, the resulting model is equivalent to following a fixed order of persuasive strategies and, through simulations, we have shown that it can change about 74% of people's situations into one where they are likely to commit to the plan. When the person's situation is included, the percentage rises to 82%, suggesting that the person's situation is also important to consider. Both of these models have advantages and disadvantages, which are discussed and addressed. Thus, this thesis provides two models for a virtual coach that can hold a persuasive dialogue in the context of dyadic planning for physical activity, which can be used as the basis for systems which target behaviour change.