This dissertation aims to explore the use of extended reality (XR) as an approach to developing user experience (UX) for product-service systems. It included eight chapters to explore the research question: “How can designers use extended reality to develop the user experience for product-service systems?”

Chapter 1 introduces three immersive experiences in user experience studies as examples and explains three relevant research topics - Product-service systems, User Experience, and Extended Reality. By reviewing the XR applications in both in design practices and in literature, the author proposed e the aim, the research question, and six sub-questions of this dissertation, followed by the explanation on theoretical backgrounds and research methodologies.

Chapter 2 answers sub-question 1 about the essence of immersive experience from users’ and designers’ viewpoints, thus proposes a user-centered model of immersive experience from literature and case analysis; then the author maps currently available XR platforms concerning the categories of experiences.

Chapter 3 firstly answers the sub-question 2 by reviewing state-of-the-art XR technologies for UX studies; then the author proposes a process to prototype experiences via XR to develop positive experiences for product-service systems.

Chapter 4 investigates three case studies to understand how to ideate concepts via
XR at the early design stage, specifically in conceptualization. In addition, the studies also compare the influence of different viewpoints and ways of interaction on the perception of “being comfortable”.

Chapter 5 examines how to assess experiences via XR across user groups and
concentrates on competence-related experiences. This chapter contains three case studies in the context of true-to-life surgical training where a successful surgery depends both on proficient psycho-motor skills and mature self-management of surgeons. In addition, these studies also observe the influences of proficiency, cultural backgrounds, and technology familiarity on the perception of competencies.

Chapter 6 scrutinizes how to facilitate remote collaboration via XR. This chapter
covers two studies in the context of remote teamwork. Given relatedness as a universal need, these studies focus on the influences of different interfaces, either immersive or non-immersive, on the perception of the co-location, as well as task loads, usability, and presence.

Chapter 7 first reviews the lessons learned from the case studies and then probes how design teams integrate immersive experiences into their practices. Hence, four co-creation studies were developed which are in line with the conceptual process in Chapter 3. Section 7.2 to Section 7.5 focus on designer's intention, designerly thinking, prototyping, and co-design via XR respectively.

Chapter 8 reflects on each sub-question from an overarching perspective, and then summarizes three sets of recommendations for design stakeholders who are interested in integrating immersive experiences in their work. This chapter then envisions a concept of a co-design community via immersion - ‘Design Metaverse’. At the end, the limitations of this work are discussed, as well as future research directions.@en

Understanding humans are the key to developing optimal design solutions for product-service systems. In this sense, the experiential approach is in line but might go beyond typical Human Centered Design (HCD) methods in that it focuses on generating positive experiences that contribute directly to human well-being. Extended Reality (XR) showed the potential to replicate or simulate experience as a whole and gained attention from design communities. XR platforms confused design practitioners due to their fast-advancing amounts and relevant experiences. Hence, this study introduced two surveys on XR platforms to clarify which experiences they could provide and when to implement them into HCD processes. Survey 1 categorized XR platforms according to their key attributes and mapped them into the Experience Matrix. Survey 2 invented two designer personas and a fictional project to analyze barriers and strategies to implement XR platforms into design processes. Eighty-eight XR platforms were categorized into nineteen clusters, where creation and simulation had the highest numbers. Regarding implementing XR in design practices, the cost is still the key concern and there's a trade-off between software cost and assets purchased for different types of designers.

@en

For designing large-scale products like an airplane, engaging end-users in the concept phase is difficult. However, early user evaluation is important to choose the path which fits the user’s needs best. In particular, comfort-related assessments are difficult to conduct with digital models that are shown on a desktop PC application. Digital Human Modelling (DHM) plays a role in postural comfort analysis, while the subjective comfort feedback still largely relied on consulting with end-users. This paper applies a human-centered design process and analyses the advantages and disadvantages of using VR prototypes for involving users during concept design. This study focused on using VR prototypes for concept selection and verification based on comfort assessment with potential end-users. The design process started with an online questionnaire for identifying the quality of the design elements (Step 1 online study). Then, alternative concepts were implemented in VR, and users evaluated these concepts via a VR headset (Step 2 Selection study). Finally, the research team redesigned the final concept and assessed it with potential users via a VR headset (Step 3 Experience study). Every design element contributed positively to the long-haul flight comfort, especially tap-basin height, storage, and facilities. The male and female participants had different preferences on posture, lighting, storage, and facilities. The final prototype showed a significantly higher comfort rate than the original prototypes. The first-person immersion in VR headsets helps to identify the nuances between concepts, thus supports better decision-making via collecting richer and more reliable user feedback to make faster and more satisfying improvements.

@en

BACKGROUND: The Human-Centered Design methodology advocates VR prototyping, as an effective tool to evaluate concepts in a cost-efficient, time-saving way. It is the question of whether it works in the development of a product intended to increase privacy while flying. OBJECTIVE: The current study aims at the application of virtual reality on the evaluation of a new privacy bubble called PRIVA for the passenger cabin. METHODS: An interactive VR was created and aligned with the HTC VIVE headset. 40 participants took part in the experiment as well as in the post-experiment survey. RESULTS: The concept was in overall, desirable as it was perceived to be more private, comfortable, satisfactory, effective, and appealing to participants compared to the current seat experiences. It was also perceived as more satisfactory with regards to the activities. CONCLUSIONS: The VR was effective, although there are limitations, the product seems promising and should be developed further.

@en

The medical care responsibilities are often on the shoulders of nonprofessionals such as captains who are equipped with forty hours of designated training every five years. However, this training is neither enough for the captains to handle medical incidents nor releases their stress during the treatment. Currently, captains have very limited support from a medical expert, only via phone call or email from the Radio Medical Services. Thus, the authors explored that how the two-way augmented reality (AR) can support the collaboration between captains and doctors for a better quality of care. A Human-Centred Design approach is applied in this study, including field study and user testing. The lean user experience method was applied with fast prototyping-testing loops. The main findings are AR played an essential role to boost confidence on the captain's side, and the real value of AR is in supporting medical skills like suturing and abdominal searching. This study serves as a pilot research, thus it was limited by small sample size and qualitative method. Improving the communication between the captains and doctors is key for future studies.@en

The present work introduces a cyclical model which showcases the process of immersion during Immersive Technological Experiences (ITEs) such as Virtual Reality, Augmented Reality and Mixed Reality. This model is based on the identified concepts around immersion and immersive environments across 30 years. The concepts' similarities were used to organize them on a cyclical model by acknowledging the user's presence at the beginning and end of immersive experiences. The proposed model's value relies on its cyclical approach based on a user-centred perspective and having a general overview of the immersion process. The Immersive cycle can serve as a mapping tool for developers and researchers, thanks to the inclusion of guidelines that complements the model. Both of these were used in three different examples of ITEs. Furthermore, the cyclical model could be used as a tool for ideation and conceptualization during the early stages of developing immersive experiences. Nevertheless, it is recognized that this is the first step in developing this model; therefore, it still needs to be validated and improved based on tests with developers, designers and researchers in the field.

@en

Ensuring surgeons are well-trained in various skills is of paramount importance to patient safety. Surgical simulators were introduced to laparoscopy training during the last 2 decades for basic skills training. The main drawback of current simulation-based laparoscopy training is their lack of true representation of the intro-operative experience. To create a complete surgical surrounding, the required amount of resources is demanding. Moreover, organizing immersive training with surgical teams burdens daily clinical routines. High-end virtual reality (VR) headsets bring an opportunity to generate an immersive virtual OR with accessible and affordable expenses. Pilot studies reveal that personalization and localization are key needs of the virtual operating room (VOR). They are therefore key in this study. The focus of this study was to explore the effect of different human factors, such as domain knowledge, culture, and familiarity of VR technologies, on the perception of VOR experience. A human-centered design approach was applied to investigate the presence and usability of a VOR. Sixty-four surgical practitioners joined the study in the Netherlands and India. The surgeons were referred to as “experts” and surgical trainees as “novices.” The VOR system we used is composed of a laparoscopic simulator, a graphic virtual OR surrounding, and an Oculus Rift VR headset. Participants conducted the “complete Lapchol” task with the VOR. Afterward, four questionnaires were used to collect subjective ratings on presence and usability. Participant’s qualitative feedback was collected using a semi-structural interview as the final stage. Results showed the surgical knowledge only affected perceived mental demand when using a VOR. The cultural difference would alter the rating on the majority of items in these questionnaires. VR experience mainly affected the judgment on presence including “quality of interface” and “reversible actions.” The interaction effects between surgical knowledge either with culture difference or with VR experience were obvious. This study demonstrated the influences of cultural differences on the perception of immersion and usability. Integrating immersive technologies such as virtual reality and augmented reality to human-centered design opens a brand new horizon for health care and similar professional training.

@en

Motion tracking software for assessing laparoscopic surgical proficiency has been proven to be effective in differentiating between expert and novice performances. However, with several indices that can be generated from the software, there is no set threshold that can be used to benchmark performances. The aim of this study was to identify the best possible algorithm that can be used to benchmark expert, intermediate and novice performances for objective evaluation of psychomotor skills. 12 video recordings of various surgeons were collected in a blinded fashion. Data from our previous study of 6 experts and 23 novices was also included in the analysis to determine thresholds for performance. Video recording were analyzed both by the Kinovea 0.8.15 software and a blinded expert observer using the CAT form. Multiple algorithms were tested to accurately identify expert and novice performances. ½ L + 13 A + 16 J scoring of path length, average movement and jerk index respectively resulted in identifying 23/24 performances. Comparing the algorithm to CAT assessment yielded in a linear regression coefficient R² of 0.844. The value of motion tracking software in providing objective clinical evaluation and retrospective analysis is evident. Given the prospective use of this tool the algorithm developed in this study proves to be effective in benchmarking performances for psychomotor skills evaluation.

@en

Immersive Virtual Reality (VR) laparoscopy simulation is emerging to enhance the attractiveness and realism of surgical procedural training. This study analyses the usability and presence of a Virtual Operating Room (VOR) setup via user evaluation and sets out the key elements for an immersive environment during a laparoscopic procedural training.In the VOR setup, a VR headset displayed a 360-degree computer-generated Operating Room (OR) around a VR laparoscopic simulator during laparoscopy procedures. Thirty-seven surgeons and surgical trainees performed the complete cholecystectomy task in the VOR. Questionnaires (i.e., Localized Postural Discomfort scale, Questionnaire for Intuitive Use, NASA-Task Load Index, and Presence Questionnaire) followed by a semi-structured interview were used to collect the data.The participants could intuitively adapt to the VOR and were satisfied when performing their tasks (M=3.90, IQR=0.70). The participants, particularly surgical trainees, were highly engaged to accomplish the task. Despite the higher mental workload on four subscales (p < 0.05), the surgical trainees had a lower effort of learning (4 vs 3.33, p < 0.05) compared to surgeons. The participants experienced very slight discomfort in seven body segments (0.59-1.16). In addition, they expected improvements for team interaction and personalized experience within the setup.The VOR showed potential to become a useful tool in providing immersive training during laparoscopy procedure simulation based on the usability and presence noted in the study. Future developments of user interfaces, VOR environment, team interaction and personalization should result in improvements of the system.

@en

Virtual reality (VR) training is widely used in several minimal invasive surgery (MIS) training curricula for procedural training. However, VR training in its current state lack immersive training environments, such as using head-mounted displays that is implemented in military or aviation training and even entertainment. The virtual operating room simulation setup (VORSS) is explored in this study to determine the effectiveness of immersive training in MIS. Twenty-eight surgeons and surgical trainees performed a laparoscopic cholecystectomy on the VORSS comprising of a head-mounted 360-degree realistic OR surrounding on a VR laparoscopic simulator. The VORSS replicated a full setup of instruments and surgical team-members as well as some of the distractions occurring during surgical procedures. Questionnaires were followed by semi-structured interviews to collect the data. Experts and novices found the VORSS to be intuitive and easy to use (p = 0.001). The outcome of the usability test, applying QUESI and NASA-TLX, reflected the usability of the VORSS (p < 0.05), at the cognitive level, which indicates a good sense of immersion and satisfaction, when performing the procedure within VORSS. The need for personalized experience within the setup was strongly noted from most of the participants. The VORSS for procedural training has the potential to become a useful tool to provide immersive training in MIS surgery. Further optimizing of the VORSS realism and introduction of distractors in the OR should result in an improvement of the system.

@en

The usability now serves as a fundamental quality of a computational device, e.g. smartphone. Moreover, the smartphone has firmly embedded into our daily life as an indispensable part, so the context and style that user may interact with them are largely different from a decade ago. Nowadays, testing usability with end user has become a common sense. Thus, how valid a usability evaluation method could assess the ‘extent to which a product can be used by specified users’ (ISO 9241-11) to facilitate software design becomes an interesting question to explore. In this research, three usability evaluation methods are compared. Among these methods, IsoMetrics is a standard questionnaire aiming at offer usability data for summative and formative evaluation; SUMI aims to assess quality of software product from end users perspective; User Model Checklist is a method based on user’s cognition-motor chain in specific tasks. The coverage and amount of usability issues, user’s effort of evaluation and software developer’s feedback on evaluation result are compared under a simulated usability test on SMS function with a smartphone. The result indicate that User Model Checklist could cover 90.4% of the usability issues found by IsoMetrics and SUMI, while 26.3% usability issues found by User Model Checklist could not be covered by IsoMetrics and SUMI. Users put highest effort on accomplish IsoMetrics and lowest effort on User Model Checklist. Moreover, the feedbacks from the developers show that the User Model Checklist requires lower usability knowledge, offers clearer improvement points and supports detailed design better.

@en

Smartphone is nowadays the most prevalent computer system, thus a lot of attention from academia and industries has been put to evaluate its quality of use. However, Smartphone has more complex interaction modes and usage scenarios than PC and laptop. And therefore assessing its quality using a conventional usability evaluation is not sufficient. Meanwhile, the mental load serves as an acknowledged index of effort that operators have put in human-machine interaction, especially under high-demanding context. Mental load contains a set of parameters in multiple dimensions, such as primitive task performance, biological measurement(s) and subjective mental load scale, which assesses the efforts of tasks under a particular environment and operating conditions. Thus, it is suitable for evaluating complex mental work, and may indicate the use of Smartphones. The aim of this paper is to apply a multi-dimensional method to assess the mental load of users, and find out which measurement(s) is the most suitable one to evaluate the efforts for using a smartphone. During this study, the effort on conducting tasks with four difficulty levels were assessed using measurements in three dimensions, which were (1) user performance (task accomplishment and secondary task), (2) subjective rating (NASA-TLX scale) and (3) physiological function (EDA). The values of these measurements were compared across novice, average and skilled users. The results show that: task duration and number of usability error are significantly related with mental load and change with the difficulty level of tasks; in subjective rating, Mental Demand, Effort and Frustration were highly related with mental load.@en

The project 'Factory-in-A-day' aims at reducing the installation time of a new hybrid robot-human production line, from weeks or months that current industrial systems now take, down to one day. The ability to rapidly install (and reconfigure) production lines where robots work alongside humans will strongly reduce operating cost and open a range of new opportunities for industry. In this paper, we explore a method of collaborative fabrication planning with the help of Augmented Reality as part of the concept Augmented Fabrication. In order to plan a new production line, two co-located workers at the factory wear a Microsoft Hololens head-mounted display and thus share a common visual context on the planed position of the robots and the production machines. They are assisted by an external remote expert connected via the Internet who is virtually co-located. We developed three different visualizations of the state of the local collaboration and plan to compare them in a user study.

@en