This research paper proposes a deep learning model to infer segments of speaking intentions using body language captured by a body-worn accelerometer. The objective of the study is to detect instances where individuals exhibit a desire to speak based on their body language cues. The labeling scheme employed is a binary string, with “0” indicating no intention to speak and “1” indicating the presence of an intention to speak on a defined window size of 40 (corresponding to a 2 seconds segment recorded at a frequency of 100 Hz scaled down to 20 binary points per second). In this experiment, a real-life social event dataset was employed, and intensions to speak were manually annotated. A 10-minute segment from the dataset was selected and annotated using the ELAN software. The annotations included two categories: realized intentions, where individuals intended to speak and actually did so, and unrealized intentions, where individuals displayed intentions to speak but did not take their turn. The dataset consisted of 255 segments with realized intentions and 31 segments with unrealized intentions. Additionally, 255 negative samples were included, representing instances where no intention to speak was observed throughout the entire segment. To address the class imbalance inherent in the dataset, the model was evaluated using the Area Under the ROC Curve (AUC) metric using 5- fold cross validation. The model was tested on realized intentions, unrealized intentions, and a combination of both. Its performance was compared against a baseline model, which is assumed to always predict the start of the intention in the middle of the segment. Also, a new model was built that enables classification using varying window sizes as input. The classification task is performed to provide a comparison to another study [1] in which the training segments were predetermined window sizes instead of precisely annotated segments. The results of the study indicate that the deep learning models perform consistently better than the baseline on the segmentation task, and surpasses the performance of the model trained exclusively on window sizes on the classification task. This not only demonstrates the potential of body language as an informative cue for inferring speaking intentions, but also suggests that a supervised learning where intentions are identified with greater precision can lead to a superior outcome.

In this thesis the automatic multimodal detection of social and task cohesion in meetings is studied. The presence of social and task cohesion has positive benefits on employee well-being, creativity and productiveness, and can therefore be used to assess meeting quality. Conversational partners imitate each other's body language and speech characteristics to have smoother interactions, to increase liking, and it can cause a coordination of expectations. We hypothesize that social and task cohesion are therefore positively related to the imitation of both body language and speech characteristics. As the group-level alignment of non-verbal speech behaviour has been previously linked to social and task cohesion in meetings, this thesis investigates the relationship between cohesion in meetings and both motion and posture mimicry from accelerometer and video data. Motion mimicry is described using accelerometer features previously used for the detection of friendly and romantic attraction in pairs. We propose a method to convert these features to group-level descriptors of mimicry. The same quantifications of mimicry are also applied to video-based motion quantifiers that have been previously used to detect team cohesion. Appearance is described using HOG descriptors from densely-sampled feature points that are tracked over time. To our knowledge this is the first-time appearance similarity is used to estimate cohesion. We also test if a multimodal mimicry model performs better than a unimodal mimicry model to investigate if the different forms of mimicry contain complementary information. Our group-level movement mimicry features detect social cohesion with average an area under the ROC-curve of 0.64 for social cohesion, and the accelerometer-based movement mimicry features specifically detect task cohesion with an average AUC of 0.63. The multimodal combination of the different features does improve over the unimodal models with an area under the ROC-curve of 0.68 for social cohesion and 0.65 for task cohesion. This shows both that movement mimicry is an indicator of verbal expressions of cohesion, and that measuring mimicry in different modalities can better model cohesion than a unimodal model. Further experiments are recommended for general appearance feature mimicry, specifically in the method used to measure mimicry, to confirm or deny if these are related to cohesion.