This research revolves around measuring the quality of arguments. High-quality arguments help in improving political discussions, resulting in better decision-making. Wachsmuth et al. developed a taxonomy breaking down argument quality into several dimensions. This work makes use of that taxonomy and combines it with modern NLP models. A cross-dataset examination of argument quality models was conducted. In particular, models were investigated on their generalizability between dimensions. Overall results show that there is no large difference in accuracy and agreement when models predict data of a quality dimension they were trained on, over dimensions they were not trained on. One can conclude that generalizations of argument quality dimensions with language models were not found. Nevertheless, qualitative analysis highlights findings that indicate some generalization to other dimensions.

Moral values influence humans in decision-making. Pluralist moral philosophers argue that human morality can be represented by a finite number of moral values, respecting the differences in moral views. Recent advancements in NLP show that language models retain a discernible level of knowledge in deontological ethics and moral norms of society. However, a model which can only decide either right or wrong cannot fully understand the diverse moral perspectives of humans.

We propose a moral sentence embedding space, which can encompass moral differences, through the state-of-the-art Contrastive Learning framework. We evaluate the moral embedding space both intrinsically and extrinsically via three tasks: classification, moral similarity, and visual analysis. We show that our moral embedding space understands the characteristics of each moral value. Our results also highlight that moral rhetoric is seldom explicit in the text, emphasizing the necessity of additional information such as moral labels.

This paper evaluates the performance of an automated explainable model, Moral- Strength, to predict morality, or more pre- cisely Moral Foundations Theory (MFT) traits. MFT is a way to represent and divide morality into precise and detailed traits. This evaluation happens in the Jiminy Cricket environment, an environ- ment composed of 25 text-based games. This evaluation helps us estimate the do- main adaptation of MoralStrength, and also its limitations. The explainability of this model helps understand those limitations. We can conclude that MoralStrength is per- forming overall worse than other optimal models and that the domain adaptation to the Jiminy Cricket domain has some cru- cial flaws, but it leads us to think about the explainability/accuracy trade-off and where to draw the line, knowing that explainable models are important for ethical decision- making.

In our everyday life, people interact more and more with agents. However these agents often lack a moral sense and prioritize the accomplishment of the given task. In consequence, agents may unknowingly act immorally. Little research or progress has been done to endow agents with human morality and an internal sense of right and wrong. As of today, agents have a primitive representation of morality often represented as 1 value. In contrast, humans have multiple reasons to judge an action as moral. In hope of creating agents that are imbued with a more complex and human moral, we build upon the Jiminy Cricket environment. This preexisting environment has multiple games with diverse scenarios and the objective is to do the most moral action to maximize the reward

Nowadays Large Language Models are becoming more and more prevalent in today's society. These models act without a sense of morality however. They only prioritize accomplishing their goal. Currently, little research has been done evaluating these models. The current state of the art Reinforcement Learning models represent morality by a singular scalar value determining the morality of a statement. This way of representing morality is inaccurate as there are multiple features determining how moral a statement is. We leverage knowledge from the Moral Foundations Theory to represent morality in a more accurate way, by using a 5-dimensional vector representing morality features. We implement several different agents in an environment where decisions with possible moral implications need to be made. These agents all use alternative approaches in deciding which action to take. The policies are: always pick the most moral action and always pick the most immoral action. Two other agents have the same aforementioned policy but still give some weight towards game progression. Lastly, we look at an amoral agent which does not look at morality at all. We compare these agents by percent completion of the Infocom game suspect. We find that the agent which does not take morality into account achieves the highest completion rate. Agents which give morality a huge weight almost instantly get stuck in an infinite loop without progression.

Morality is a fundamental concept that guides humans in the decision-making process. Given the rise of large language models in society, it is necessary to ensure that they adhere to human principles, among which morality is of substantial importance. While research has been done regarding artificial agents behaving morally, current state of the art implementations consider morality to be linear, thus failing to capture its complexity and nuances. To account for this, a multidimensional representation of morality is proposed, each dimension corresponding to a different moral foundation. Then, the performance of three types of artificial agents tasked with choosing actions while playing text-based games is compared and analysed. One type of agent is implemented to only choose the most moral action, without aiming to win the games, another one prioritizes moral actions over game progression, and another strives to win the games while also playing morally. The latter outperforms the others in terms of game progression, while also taking few immoral actions. However, the agent prioritizing morality over progression performs only slightly worse while taking no immoral actions, proving that artificial agents can perform well while also behaving morally.

Moral values are often used as guidelines for human behaviour. The ability to identify moral values is important for social and ethical artificial intelligence. We address the difficulties of using contemporary natural language processing (NLP) techniques to classify moral values in texts. As the classification criterion of moral values is subjective, it is often difficult to argue for the existence of a `ground truth' label. In such circumstances, we can learn from the (dis-)agreement among multiple annotators. However, it is expensive to consult everyone, especially when working with crowdsourcing. A way to reduce the annotation cost is to apply active learning, which uses query strategies to choose the data and annotator that are most valuable to consult. Further, to account for subjectivity, we want to ensure the dataset is labelled by a diverse set of annotators. Therefore, we propose an annotator selection method for active learning. When given an unlabelled text, this method selects an annotator that has labelled the least amount of texts that are similar to the given text. The evaluation results show that the method performs better on datasets with balanced annotator distribution.

Automated negotiation agents can highly benefit from learning their opponent’s preferences. Multiple algorithms have been developed with the two main categories being: heuristic techniques and machine learning techniques. Historically, heuristic techniques have dominated the field, but with the recent development in the field of machine learning, this is no longer true. The main goal of the paper is to compare these two techniques quantitatively using the Pearson correlation of bids. The models that were chosen as the heuristic and machine learning baseline are the Smith and the Perceptron models, respectively. Our results show that the two baselines have similar performance. This leads us to conclude that machine learning algorithms have caught up with their heuristic counterparts. Furthermore, we have also found a statistically significant correlation between the Perceptron model’s accuracy and the seen bid space.

With the prospects of decentralized multi-agent systems becoming more prevalent in daily life, automated negotiation agents have made their place in these collaborative settings. They are an approach to promote communication between the agents in reaching solutions that are better for all involved.

Recent literature has shown great potential in using machine learning, particularly model-free deep reinforcement learning like Proximal Policy Optimization (PPO), to develop more performant automated negotiation strategies. This work focuses on using information from the opponent's sequence of offers in a bilateral negotiation to further improve a baseline PPO agent. This involves extracting and representing information from the opponent's sequence of offers into a state vector with a fixed dimension to modify the input to the agent's policy, and then comparing the utilities this modified agent achieves to the baseline PPO agent. Since there is a large variety of numerical measures to represent a sequence of offers, an ablation study is conducted to investigate the effectiveness of each.

The modified agents consistently reached solutions that had higher social welfare, although the agent's own utility did not improve or diminish significantly in comparison to the base PPO agent.

This paper introduces a strategy for learning opponent parameters in automated negotiation and using them for future negotiation sessions. The goal is to maximize the agent’s utility while being consistent in its performance over various negotiation scenarios. While a number of reinforcement learning approaches in the field have used Q-learning, this paper uses the newer Proximal Policy Optimization algorithm. Machine learning has been used in opponent modeling, classifying opponents, and learning strategies, but there have been few attempts to store and re-use this information. In an experimental setup, it is shown that this approach outperforms a baseline in terms of individual utility.

Recent developments in applying reinforcement learning to cooperative environments, like negotiation, have brought forward an important question: how well can a negotiating agent be trained through self-play? Previous research has seen successful application of self-play to other settings, like the games of chess and Go. This paper explores the usage of self-play within the training of a negotiating agent and determines if it is possible to successfully train an agent purely through self-play. The results of the experimentation show that a training stage using self-play can match or even exceed an approach using a set of training opponents. By using multiple self-play opponents, the average utility can be further improved by introducing more variance during training. In addition, using a combination of both self-play and training opponents leads to a hybrid approach that performs better than either of the two techniques separately.

The domains of the negotiation can vary significantly. It is possible that a domain is very cooperative, where both agents can receive a high utility; the opposite is also possible, where the domain is very competitive and the agents cannot both get a high utility. In the same manner, the agents can have different strategies leading to a complicated problem with no obvious solution.

This research seeks to represent the differences in negotiation domains to improve a machine learning based agent to help the agent generalize these domains. To achieve this several ways of representing the domain have been explored.
First is the shared domain information. With this representation, the agent uses information about the amount of issues, values and possible bids there are. Second is the private domain information, in this representation, the agent uses different calculations to get a view of how favorable the domain is in terms of utility. Last is the derived information, this is the representation where the agent learns about the domain by interaction with the environment or the opposing agent.

From the experiments, a conclusion could be made that a part of these representations had a positive impact on the final utility of the agent. The shared domain information had a considerable improvement over the base agent with the features having a non-negligible impact on the negotiation. The derived information also had a considerable impact on the final outcome.