With the increasing popularity of Large Language Models (LLMs), fine-tuning them has become increasingly computationally expensive. Parameter Efficient Fine-Tuning (PEFT) methods like LoRA and Adapters, introduced by Microsoft and Google, respectively, aim to reduce the number of trainable parameters, with the current state-of-the-art combining both methods as LoRA Adapters. This paper introduces Transformer Modules as a PEFT method. These modules utilize Modular Transformer Blocks (MTBs) inserted into a frozen pre-trained model, achieving competitive performance while significantly reducing computation costs. Compared to the current state-of-the-art using GPT-2, BERT, and T5, Transformer Modules further reduced compute time by 39.7\% and training memory by 72.7\%, with a performance cost of 4.5±2.51\% on the GLUE benchmark. Additionally, the paper presents the Transformer Bridge, a continuous vector transformer designed to transfer Transformer Modules across different models. This could enable cross-model fine-tuning, allowing model-agnostic modules, such as an ethics or medical module, to be used across various LLMs without retraining or access to the original dataset. Although the current implementation of the Transformer Bridge did not fully succeed in mapping embedding spaces, analysis of the results suggests that further refinements using traditional model distillation techniques could lead to success in future iterations.

In order to tackle topics such as climate change together with the population, public discourse should be scaled up. This discourse should be mediated as it makes it more likely that people understand each other and change their point of view. To help the mediator with this task, emotion detection can greatly help. Positive emotions can improve communications, while negative emotions cause people to be irrational and irritated. However, since emotions are highly subjective, it can make both predictions and evaluation more difficult.

Still, Large Language Models (LLMs) could be used to detect these subjective emotions using different prompting strategies and labels. The experiment included zero-, one-, fewshot and Chain of Thought (CoT) strategies. The precision was better for the one- and fewshot method compared to zeroshot. The CoT methods also showed an increase in precision, but a decrease in recall. The different labels were hard majority labels, soft labels and hard per annotator labels. In conclusion, providing examples improved the performance of the LLM. The CoT strategies were more precise, but gave a worse general prediction. The hard majority labels allow for more general predictions, where per annotator hard labels capture the perspective of different annotators. Soft labels reflect the subjective nature of the labels by providing probabilities instead of binary classification.

The experiment was done on a small data sample, so it is recommended to try the strategies on a larger data sample. Looking into appropriate evaluations for subjective predictions is also recommended in order to reflect the actual performance better.

This paper investigates the use of Large Language Models (LLMs) for automatic detection of subjective values in argument statements in public discourse. Understanding the underlying values of argument statements could enhance public discussions and potentially lead to better outcomes. The LLM utilization methods tested were zero- and few-shot prompting, as well as chain-of-thought prompts. In order to compare the predictions made by the LLM, a set of ground truth labels was required as an established baseline. For these labels, either single majority labels or multi-value labels were considered, both derived from a set of aggregated human annotations. Results indicated that LLM performance was sub optimal, achieving a maximum weighed F1 score of 0.594 for single-value chain-of-thought predictions. Additionally, current metrics were found inadequate for assessing LLM performance on a highly subjective task such as value detection, evidenced by poor scores in multi-value predictions despite subjective evaluation suggesting otherwise. Furthermore, a last experiment was aimed at capturing a specific annotator’s subjectivity. This yielded inconsistent results, with F1 scores peaking around 0.4, indicating that LLMs are not well-suited for emulating individual human subjectivity.

This study investigates the effectiveness of Large Language Models (LLMs) in identifying and classifying subjective arguments within deliberative discourse. Using data from a Participatory Value Evaluation (PVE) conducted in the Netherlands, this research introduces an annotation strategy for identifying arguments and extracting their premises. Then, the Llama 2 model is used to test three different prompting approaches: zero-shot, one-shot and few-shot. The performance is evaluated using the cosine similarity metric and later enhanced by introducing chain-of-thought prompting. The results show that zero-shot prompting unexpectedly outperforms one-shot and few-shot prompting, due to the LLM overfitting to the examples provided. Chain-of-thought prompting is shown to improve the argument identification task. The subjectivity of the annotation task is reflected by the low averaged pairwise F1 score between annotators, and the considerable variance in the number of data items marked by each annotator as not being arguments. The subjectivity of the task is further highlighted by a pairwise chain-of-thought prompting analysis, which shows that annotators with more similar annotations received more similar LLM responses.

Public deliberations play a crucial role in democratic systems. However, the unstructured nature of deliberations leads to challenges for moderators to analyze the large volume of data produced. This paper aims to solve this challenge by automatically identifying subjective topics behind public discourse by leveraging Large Language Models (LLMs). The study is structured around two core objectives: Identifying Gold Labels and Exploring Subjective Human Labels. The results highlight that fine-tuning the LLaMa-2 model with QLoRa outperforms other methods for Identifying Gold Labels, while the Few-Shot Chain of Thoughts method, enhanced with EmotionPrompt, is particularly effective in capturing subjective variations in human annotations. However, the study also underscores significant limitations, such as the dependency on large, high-quality annotated datasets and the tendency of models to produce hallucinations. These findings highlight the potential of LLMs to identify subjective topics behind public discourse, while also emphasizing the need for further research to address these challenges.

This study evaluates the performance of different sentiment analysis methods in the context of public deliberation, focusing on hard-, soft-, and subjective-label scenarios to answer the research question: ``can a Large Language Model detect subjective sentiment of statements within the context of public deliberation?''. If the answer to this question is affirmative, that is a strong indicator that, with the help of longitudinal studies, sentiment analysis with large language models (LLMs) may be implemented to scale public deliberations by providing support for moderators in such discussions. To answer this question, four prompting methods were tested: zero-shot, few-shot, chain-of-thought (CoT) zero-shot, and CoT few-shot using a Frisian dataset of 50 statements annotated by 5 annotators. The findings indicate that the CoT few-shot method significantly outperforms other methods in all scenarios, that soft-labels outperform their hard equivalent, that the underlying data must be balanced for high performing models, and that capturing the perspective of a specific annotator requires further research. Our study suggests that LLMs may perform best under the supervision, or with the collaboration of a human, due to the multi-faced nature of sentiment.

Moral values influence humans in decision-making. Pluralist moral philosophers argue that human morality can be represented by a finite number of moral values, respecting the differences in moral views. Recent advancements in NLP show that language models retain a discernible level of knowledge in deontological ethics and moral norms of society. However, a model which can only decide either right or wrong cannot fully understand the diverse moral perspectives of humans.

We propose a moral sentence embedding space, which can encompass moral differences, through the state-of-the-art Contrastive Learning framework. We evaluate the moral embedding space both intrinsically and extrinsically via three tasks: classification, moral similarity, and visual analysis. We show that our moral embedding space understands the characteristics of each moral value. Our results also highlight that moral rhetoric is seldom explicit in the text, emphasizing the necessity of additional information such as moral labels.

When making decisions, people are automatically guided by their moral compass. However, AI agents need to be conditioned in order to be steered towards moral behaviour. An environment that can be used to train and test agents is the Jiminy Cricket environment. The Jiminy Cricket environment consists of a set of text-based narrative games, where every action possible is annotated with the morality of that action. However, to create a more morally nuanced agent, we have annotated all of the actions according to the following moral values: Care/harm, fairness/cheating, loyalty/betrayal, authority/subversion, and purity/degradation. To morally condition the agent, we calculate the predicted progress of a potential action and combine it with an oracle to retrieve the moral annotation of the potential action. Using both of these components, the score per generated action is calculated and based on the score the eventual action is chosen. The score can be calculated differently based on the weights assigned to the overall progress and morality, as well as based on the sub-weights assigned to each moral value. Using this environment we pose the question, if we focus on only one moral value, what is the most optimal configuration that can be achieved in order to maximize both progress and morality? From the results we can observe that the lowest relative immorality can be achieved by imposing no moral constraints on the agent. Posing constraints on the agent will lead to a relatively bigger decrease of the completion percentage than to the immorality decrease. One-hot encoding the moral values will reveal which immoral actions are needed to progress in the game, and which immoral actions should to be prevented to lower immorality.

In our everyday life, people interact more and more with agents. However these agents often lack a moral sense and prioritize the accomplishment of the given task. In consequence, agents may unknowingly act immorally. Little research or progress has been done to endow agents with human morality and an internal sense of right and wrong. As of today, agents have a primitive representation of morality often represented as 1 value. In contrast, humans have multiple reasons to judge an action as moral. In hope of creating agents that are imbued with a more complex and human moral, we build upon the Jiminy Cricket environment. This preexisting environment has multiple games with diverse scenarios and the objective is to do the most moral action to maximize the reward

Morality is a fundamental concept that guides humans in the decision-making process. Given the rise of large language models in society, it is necessary to ensure that they adhere to human principles, among which morality is of substantial importance. While research has been done regarding artificial agents behaving morally, current state of the art implementations consider morality to be linear, thus failing to capture its complexity and nuances. To account for this, a multidimensional representation of morality is proposed, each dimension corresponding to a different moral foundation. Then, the performance of three types of artificial agents tasked with choosing actions while playing text-based games is compared and analysed. One type of agent is implemented to only choose the most moral action, without aiming to win the games, another one prioritizes moral actions over game progression, and another strives to win the games while also playing morally. The latter outperforms the others in terms of game progression, while also taking few immoral actions. However, the agent prioritizing morality over progression performs only slightly worse while taking no immoral actions, proving that artificial agents can perform well while also behaving morally.

Nowadays Large Language Models are becoming more and more prevalent in today's society. These models act without a sense of morality however. They only prioritize accomplishing their goal. Currently, little research has been done evaluating these models. The current state of the art Reinforcement Learning models represent morality by a singular scalar value determining the morality of a statement. This way of representing morality is inaccurate as there are multiple features determining how moral a statement is. We leverage knowledge from the Moral Foundations Theory to represent morality in a more accurate way, by using a 5-dimensional vector representing morality features. We implement several different agents in an environment where decisions with possible moral implications need to be made. These agents all use alternative approaches in deciding which action to take. The policies are: always pick the most moral action and always pick the most immoral action. Two other agents have the same aforementioned policy but still give some weight towards game progression. Lastly, we look at an amoral agent which does not look at morality at all. We compare these agents by percent completion of the Infocom game suspect. We find that the agent which does not take morality into account achieves the highest completion rate. Agents which give morality a huge weight almost instantly get stuck in an infinite loop without progression.

This paper evaluates the performance of an automated explainable model, Moral- Strength, to predict morality, or more pre- cisely Moral Foundations Theory (MFT) traits. MFT is a way to represent and divide morality into precise and detailed traits. This evaluation happens in the Jiminy Cricket environment, an environ- ment composed of 25 text-based games. This evaluation helps us estimate the do- main adaptation of MoralStrength, and also its limitations. The explainability of this model helps understand those limitations. We can conclude that MoralStrength is per- forming overall worse than other optimal models and that the domain adaptation to the Jiminy Cricket domain has some cru- cial flaws, but it leads us to think about the explainability/accuracy trade-off and where to draw the line, knowing that explainable models are important for ethical decision- making.

Moral values are often used as guidelines for human behaviour. The ability to identify moral values is important for social and ethical artificial intelligence. We address the difficulties of using contemporary natural language processing (NLP) techniques to classify moral values in texts. As the classification criterion of moral values is subjective, it is often difficult to argue for the existence of a `ground truth' label. In such circumstances, we can learn from the (dis-)agreement among multiple annotators. However, it is expensive to consult everyone, especially when working with crowdsourcing. A way to reduce the annotation cost is to apply active learning, which uses query strategies to choose the data and annotator that are most valuable to consult. Further, to account for subjectivity, we want to ensure the dataset is labelled by a diverse set of annotators. Therefore, we propose an annotator selection method for active learning. When given an unlabelled text, this method selects an annotator that has labelled the least amount of texts that are similar to the given text. The evaluation results show that the method performs better on datasets with balanced annotator distribution.

Moral values are instrumental in understanding people's beliefs and behaviors. Estimating such values from text would facilitate the interaction between humans and computers. To date, no comparison between NLP models for predicting moral values from text exists. This paper addresses this by comparing LSTM and more novel models such as BERT and fastText to evaluate their capabilities for predicting moral values. Twitter Corpus, a collection of 35000 Tweets containing relevant recent political and social events, is chosen for this purpose. The results show that novel solutions outperform long-established ones. BERT is proven to be the best model for this task, but long training times hinder its practicality. By contrast, fastText offers similar performance while being orders of magnitude faster.

Moral values are abstract ideas that ground our judgements towards what is right or wrong. However, with the rapid unfold of moral rhetoric on social media, it becomes increasingly important to place these ideas in a moral frame, contain their harmful effects, and recognise their positive ones. So far, estimating values from opinionated text has posed a challenge due to values' abstract and subjective nature. However, with the latest developments in Natural Language Processing (NLP), we foresee an opportunity to align the study of morality in text with state-of-the-art NLP architectures. Recently published, the Moral Foundations Tweeter Corpus is a milestone in moral classification tasks by offering a dataset that allows for a closer look into how people express moral narratives in social media. In the downstream process of a text classifier, embeddings convert words and sentences into meaningful vectors. Pre-trained on large corpora, they can be fine-tuned, and domain adapted. This study proposes a refinement model, starting from the available dataset, that learns to capture moral information in Sentence-BERT embeddings by applying a state-of-the-art supervised method (triplet loss). We further demonstrate how the refined embeddings improve the accuracy of moral classifiers. Finally, with an improvement of 5% F1-score over models that use pre-trained embeddings, we pave the way towards a generalisable and transferable set of moral embeddings.

Personal moral values represent the motivation behind individuals' actions and opinions. Understanding these values is helpful both in predicting individuals' actions, such as violent protests, and building AI that can better collaborate with humans. Predicting moral values is a challenging problem due to the abstract and subjective essence of moral values. With the help of seven Twitter datasets corresponding to different domains, we train state-of-the-art Natural Language Processing models in predicting moral values. An interesting limitation of the models is that they all suffer from catastrophic forgetting. Catastrophic forgetting is the degree to which models worsen their performance on older data after being trained on new data. We conclude that catastrophic forgetting occurs irrespective of the models being trained and can be mitigated by not only training on new data but by training on a combination of old and new data. This is all possible under one assumption: old data is available. We deliver an evaluation of catastrophic forgetting for each model, explain the differences between the models, and suggest possible future work that can be built upon this research.

Understanding personal values is a crucial aspect that can facilitate the collaboration between AI and humans. Nonetheless, the implementation of collaborative agents in real life greatly depends on the amount of trust that is built in their relationship with people. In order to bridge this gap, more extensive analysis of the explainability of these systems needs to be conducted. We implement LSTM, BERT and FastText, three deep learning models for text classification and compare their interpretability on the task of predicting moral values from opinionated text. The results highlight the different degrees to which the behaviour of the three models can be explained in the context of moral value prediction. Our experiments showed that BERT, current state-of-the-art in natural language processing tasks, achieves the best performance while also providing more interpretable predictions than the other two models.

Moral values play a crucial role in our decision-making process by defining what is right and wrong. With the emergence of political activism and moral discourse on social media, and the latest developments in Natural Language Processing, we are looking at an opportunity to analyze moral values to observe trends as they form. Recent studies have extensively examined the performance of different NLP models for estimating moral values from text, but none of them has tackled the problem of transfer learning. Our study provides a comprehensive look into the cross-domain performance of three state-of-the-art models. We find that BERT, the current most used model in Natural Language Processing, offers the best results. For reproducibility, we publicly release our code on GitHub.