With the advent of social robots which are designed to be 'social', human-like interactions have become a necessity. It is natural for us to use a plethora of emotions to convey additional information or to make an interaction more engaging. But emotion expression is not commonly associated with robots. Many humanoid robots cannot generate facial expressions to portray various emotions. Studies have shown that robots are multi-modal systems which can employ multiple channels to express an emotion. This thesis explores the non-verbal emotion expression techniques and their expressive capabilities. We found that some emotions are easier to express than others, and a single technique cannot express all the emotions. We chose a few emotions and systematically determined the best technique for each of them.

Social Robotics is an emerging field in Computer Science. Most social robots currently commercially available to buy do not have fast hardware components. As a result, the built-in software has low accuracy and performance with (amongst others) speech and facial recognition and dialogs during social interaction with users. Cloud computation offers state-of-the-art techniques, performance, and accuracy with its massive available computational power, but at extra costs and increased latency.

In this work, we extend and improve a social robot's standard capabilities and performance by making use of cloud computation. This thesis covers an exploration for the trade-offs present when replacing or augmenting built-in robot software with IBM cloud services, based on the humanoid Pepper robot from Softbank.

The approach for this exploration was guided by a hospitality use case demonstrated in the offices of two companies: a Dutch Health Insurer and IBM Netherlands. Two products were developed for this single use case based on different development toolboxes. The first toolbox contains all development software from the robot's manufacturer (the NAOqi toolbox), while the second toolbox makes use of cloud services (the Watson Toolbox). Using the product built with the NAOqi toolbox, we evaluate interactions with real users and obtain baseline data and experiences. After evaluating the second product built with the Watson toolbox, we can compare differences in human-robot interaction quality, robot component quality, development methods, and software engineering complexity and Total Costs of Ownership for both products.

The main findings include an overview of relevant test metrics and test methods for a social robot's component, including acquired data for some components of the Pepper robot. We show possible architectures for a (semi) cloud-based system, and their trade-offs. Evaluations show that the cloud-based system indeed performs better and has higher human-interaction quality compared to the product built with the NAOqi toolbox, yet downsides such as latency and operating costs are present. This is also reflected in the analysis of single components, where specifically Speech-to-Text from the cloud shows a significant increase in performance and capabilities. We show that a mix of toolboxes results in the best working and cheapest social robot when considering Total Cost of Ownership. IBM Cloud pricing structures and operating costs are analyzed for this. Finally, we contribute to the currently available knowledge on this subject with a decision matrix combining all previously mentioned information in a compact form accessible to people not knowledgeable in the hospitality robot or cloud domains. With the matrix, early-development advice decisions for creating a social robot can be formulated using the data gathered in this thesis.

With a broad approach, this research focuses on finding and discussing trade-offs, rather than an in-depth analysis of all components. Providing methods to determine the data as mentioned above, findings, and trade-offs are more important than the actual numbers found in this thesis, as advances in this domain are quick and expected to change often. The end product built using the Watson toolbox is an improvement on multiple levels, yet is still not always able to autonomously and correctly finish all intended interactions. However, its capabilities, performance, and robustness are closer to the level of being used commercially. The techniques we use to extend Pepper's capabilities could be applied to any social robot.

Schools and other educational instances (such as: elementary, primary and secondary schools, schools for applied sciences and universities) are making use of new innovations and technologies to help teachers and students.
After Smart Boards, computers, and tablets, a new field of research arises: can robots in any way help in teaching? During this research an answer to the question "What is the effect of elaborated feedback versus minimal feedback given by a humanoid robot (NAO), on a primary school student solving basic math problems?". Also, two hypothesis are tested: "Feedback provided by NAO will improve the student math test results." and "Feedback improved the affection of the student towards NAO.". We find that lacing and split and add are the most used calculation strategies, and NAO also aims to identify commonly made mistakes such as switching units in bigger number or confusion about symbols or strategies.
An important element of this research is feedback, which concerns information about how we perform in efforts to reach goals. It requires information about the goal, the followed track and the next steps. Feedback seems to be most effective when it is self-regulated and intended for process. Children need tangible and timely feedback, not too early (to prevent overwhelming the student) and not too late. Giving feedback on the result as well as on the track, challenging students and defining a clear road to success are crucial in helping the student succeed.
In this research, NAO will provide this feedback and thereby should help the student succeed in solving math problems.
Two scenarios are designed: an introductory class, to let students touch, feel and experiment with NAO, and a math tutoring session, which will be used during the experiment. We conduct an experiment, testing a feedback group versus a control group, where the feedback group is able to get feedback twice (elaborated feedback), and the control group only hears whether the answer given was wrong or correct. The elaborated feedback consists of detecting a mistake made by the student, and adapt the feedback according to the given answer. After a second try and incorrect answer, the process to the correct answer is explained to the student. Students had to fill in a PANAS form in order to measure affection towards NAO during the sessions. No significant results were found in both the math test and the PANAS scores.
We did however find that students learned to work with NAO very quickly, interaction was robust, although speech recognition needs some improvement.
Interesting to see was that we found contradictions when we compared the qualitative analysis with the quantitative analysis. Statistical analysis showed that there was no significant growth in the skills of a student, but all students mentioned that they felt that they did learn from the session with NAO. Future work could involve conducting the experiment for a longer period of time, to see a better lasting effect of the feedback given. Other work could be done in distinguishing groups according to their level of math, to see if students with a low grade in math have a higher profit from NAO's influence. Also, combining mistakes could lead to more mistakes that can be detected, and thus feedback could be more efficient.

In this thesis a cognitive multi-agent system is developed for the real-time strategy game StarCraft: Brood War using GOAL, a language developed at the Delft University of Technology. StarCraft provides various challenges to GOAL that it has not faced before in terms of complexity and quantity of agents. A connector is required for GOAL to interact with StarCraft: the StarCraft-GOAL Environment Connector. In order to provide context for the development stage, this thesis starts by explaining the basics of StarCraft, GOAL and the connector. The development of the bot that will be created, named ForceBot, takes place over the course of four tournaments that serve as milestones and are used to measure the game performance of ForceBot. As the conditions and participants of these tournaments vary, an additional setup for hosting internal test tournaments is established in order to provide a form of testing that remains consistent over the course of the development. Based on the results of tournaments, it can be concluded that the final game performance of ForceBot is above average with regards to existing StarCraft bots. A total of four aspects of GOAL, the language, tools, connector and computational performance, are analysed based on their usage and performance during development, and suggestions for improvements are made accordingly. Among these four aspects, the computational performance is identified as the primary bottleneck for the game performance of GOAL-based StarCraft bots.

The purpose of this thesis is to design robot behavior that stimulates self-disclosure of introverted and extraverted children during an initial interaction. This was done by designing two styles of robot behavior. One behavior style that was specifically adapted to introverted children and one behavior style that was specifically adapted to extraverted children. 41 children interacted with one of the two styles of robot behavior, using a 2 x 2 between subjects design, with child extraversion (extravert vs. introvert) and robot adaptation (behavior style matched with child’s extraversion level vs. behavior style not matched with child’s extraversion level). No significant effect of child extraversion or robot adaptation were found. However, our design did enable the robot to interact autonomously with children. Our design thus can be a building block for future design iterations.

The energy markets are emerging slowly towards a more decentralized form, e.g. technologies are allowing prosumers to trade energy from their solar house rooftops. But as these markets are governed by people and depend on the behavior of people directly, they are susceptible to challenges from the social, economic and cultural norms around the people. One of the example is the cultural discrimination in energy trading noted by some researchers and practitioners in rural India, where people do not prefer to trade energy and goods with different castes, especially lower castes. This creates an unfair market for the poor and lower caste producers, and thus this thesis aimed at modeling such markets through agent based modeling, and introduce different automation mechanisms, namely local mediation, and bid splitting to prevent the effects of discrimination in energy trading. Local mediation allows introduction of a mediator who prevents from direct trade between people and thus prevents any discrimination, bringing in some sort of anonymity. The other mechanism of bid split allows splitting the bids by producer and consumer into small chunks, such that 2 producers in market with 1 and 100 units of energy to trade would look like 101 producers in market trading 101 units of energy. Evaluating the results of these mechanisms on increase of social welfare, efficiency, market access and reduction of inequality has shown that the bid split helps in reducing the discrimination, and so does the introduction of mediation (where the mediator does not discriminate). The former has much more significant effect on the market access, and the validation of the model has helped derive some policy recommendations for implementer in the energy industry and governance field e.g. on the load management for the devices.

The elicitation of user engagement is one of the current challenges of human-robot interaction, alongside with the identification of appropriate metrics to evaluate users' experience. Several studies focused on strategies aimed at maintaining engagement throughout an interaction, but limited research has been done on how to initiate it by drawing users' attention. The use of social cues in nonverbal human-human communication has been identified as a reliable source of information to determine if a person is engaged. These social cues can be used not only to understand more about human behavior, but also to design robot behaviors that can successfully draw the attention of humans. In this project we look to investigate what are effective techniques to draw attention and elicit initial engagement with a social robot at the entrance of a building. The robot proposed to display these behaviors is the humanoid robot Pepper, from SoftBank Robotics, as it has been specifically designed for human-robot interaction. Initially, the on-board functionalities of the robot are going to be tested. Secondarily, state-of-the-art techniques are going to extend those functionalities to improve Pepper's interactive skills. Eventually, robot behaviors are going to be designed and displayed to participants during an experiment. We aim at understanding the reactions of people to identify the most effective drawing attention behaviors and examine if the encounter with the robot is affected by a novelty effect. Different metrics are proposed to measure both these phenomena in our results. Our system is developed in Python with features extracted from Pepper's Naoqi framework.

Humanoid robots are becoming part of modern societies mainly involving the education and entertainment fields. Therefore, providing techniques that enable a humanoid to autonomously play instruments like the metallophone we enrich its usefulness.

In this work we present an approach for making humanoid robot NAO play the metallophone. Specifically, we provide an application which can be used by everyone owned a NAO robot. In our approach, no markers are necessary to be attached on the robot, making the procedure of playing music, seems natural. We propose an efficient real-time approach for recognizing and estimate the metallophone in 3D space using a single camera. Our vision approach performs well in camera displacements, illumination and aspect changes.

For accurate playing, the robot automatically validates the computed IK configurations based on a vision-based robot control approach and adapts its arm configurations if it is necessary. To achieve this, we rely on the estimated pose of the musical instrument and the detection of the beaters in 2D, based on natural features and then we apply inverse kinematics. Concerning kinematics, we use singularity-robust approach by pre-computing forward kinematics which are used to efficiently solve inverse kinematics (IK) problems.

In our approach we limited on moving only NAO's arms for playing the metallophone. Therefore, we developed a human-robot interaction phase in which the robot uses its perception about the position of the instrument in 3D space and it guides a human in order to place it in a reachable for the robot position. Moreover, in our system we provide a simple way to add songs, which NAO can immediately play.

This thesis project worked towards extending the interactive capabilities of the RoboTutor Nao, by enabling it to answers natural language questions about topics such as computers, robots and the Nao itself. The main questions we focused on were 1) what a system for answering questions in Dutch and intended for elementary school age children should look like, 2) how the interaction with users should be structured for a smooth interaction and 3) what type of experimental studies and metrics are best for evaluating the quality of the QA system and assess its impact on users. We chose an existing English-language QA system and adapted it to work with Dutch content. An interaction manager was developed for handling the interaction between the system and users. Finally, the system was configured so that it could be interacted with via the Nao robot. The system was then evaluated via two experimental studies. First a quantitative evaluation without users was carried out, evaluating the performance of the system using a collection of test questions. Secondly, a field study with users was done. We took the robot to a day care center where children could ask it questions and assessed the effects of these interactive Q\&A sessions. Results from the quantitative evaluation highlighted slow answering times as the main shortcoming, while recall (70%) and precision (50%) scores were more competitive. The field study showed that even though the occasional repetitiveness and lack of speed were picked up on by users, this did little to curb their enthusiasm. Interaction was smooth, users were interested in many topics the system supported and they continued interacting with it despite the occasional mishap. Overall, the results indicate that, with some speedup, the delivered QA system would be a suitable choice for answering questions about the mentioned topics.

A large problem that primary schools face is that the ratio of pupils to teachers is too high, the class sizes are too large and this makes it difficult for a single teacher to have a good oversight of how the development of a given child is going. The aim of Interactive Robotics is to tackle this problem by bringing robots into the classroom to aid teachers. They aim to have a single robot in a classroom that has the ability to teach different lessons and subjects; the RekenRobot being specifically for basic arithmetic. During the research phase, ideas were gathered regarding how to create teaching methods that are motivating and stimulating. For instance, personalisation, humanising the robot and adaptability of the teaching material were desired functions. The software for the RekenRobot was built from scratch, using the programming language GOAL, JavaScript, CSS, HTML and JSP. The original target audience of the project were children between the ages of 6 and 8. Later this was changed to cover different school years: 3-4, 5-6 and 7-8, making use of levels with different degrees of difficulty. The robot can work one-on-one with a child, being able to practice addition, subtraction, times tables and telling time, as well as 2 forms of explanations can be given: making use of a bus and a number line. Using no explanation to rather focus on automation is also an option. The idea of the project was to lay the groundwork for the later development of the RekenRobot, as this will be an ongoing project for Interactive Robotics. The application designed in this project will be adapted to become part of the Interactive Robotics system. The first user tests at primary schools yielded a largely positive result. The children were excited and motivated to work with the product. The system is simple enough to require very little explanation. This project was never meant to realise a product that can be deployed tomorrow, but the result is a very solid basis for further improvements.

Socially-aware robot navigation strives to find efficient methods to autonomously navigate known environments while incorporating social metrics derived from human behavior. Several methods built for navigation on dynamic and uncertain environments have been adapted to resemble human navigation, but fail to appropriately reflect the social characteristics of human decision making.

A reliable solution for this problem is found in model-based methods working as local or global planners. Model-based research on human aware navigation focuses on two specific alternatives. The first option corresponds to models that apply social-psychology and cognitive sciences to create human-like behavior, where the Social Force Model is the predominant approach. The second alternative is related to models that use machine learning to copy human-like characteristics into mathematical models.

The former approach has proven to be efficient in human-aware navigation, but further studies are required to analyze the expansion of this method to more complex human-interactive navigation. In this thesis project, we look to test the Modified Extended Social Force Model (MESFM) to implement a guiding behavior on a humanoid robot. The MESFM incorporates a new force linked to the guided person to maintain a natural distance between both subjects, while generating smooth navigational maneuvers.

In addition, whether caused by sensor failure or occlusion incidents, the event of losing track of the guided person has a good probability of occurring. This scenario is studied by extending the high level control of the robotic system with a searching mode.

The architecture proposed to control the guiding behavior and the searching mode exploits the design of the humanoid robot Pepper, from SoftBank Robotics, to incorporate human-like gestures and increase the interaction between robot and human.

We aim to apply this behavior as an Office-Guide Robot and test this solution to understand the reaction of people involved in the guiding task though subjective and objective metrics. Our system is developed on top of the open-software Robot Operative System (ROS) with features extracted from Pepper's Naoqi framework.