Operating a complex and dynamic system, such as an agile manufacturing line, is a knowledge-intensive task. It imposes a steep learning curve on novice operators and prompts experienced operators to continuously discover new knowledge, share it, and retain it. In practice, training novices is resource-intensive, and the knowledge discovered by experts is not shared effectively. To tackle these challenges, we developed an AI-powered pervasive system that provides cognitive augmentation to users of complex systems. We present an AI cognitive assistant that provides on-the-job training to novices while acquiring and sharing (tacit) knowledge from experts. Cognitive support is provided as dialectic recommendations for standard work instructions, decision-making, training material, and knowledge acquisition. These recommendations are adjusted to the user and context to minimize interruption and maximize relevance. In this article, we describe how we implemented the cognitive assistant, how it interacts with users, its usage scenarios, and the challenges and opportunities.@en

Technology-enhanced learning systems, specifically multimodal learning technologies, use sensors to collect data from multiple modalities to provide personalized learning support beyond traditional learning settings. However, many studies surrounding such multimodal learning systems mostly focus on technical aspects concerning data collection and exploitation and therefore overlook theoretical and instructional design aspects such as feedback design in multimodal settings. This paper explores multimodal learning systems as a critical part of technology-enhanced learning used for capturing and analyzing the learning process to exploit the collected multimodal data to generate feedback in multimodal settings. By investigating various studies, we aim to reveal the roles of multimodality in technology-enhanced learning across various learning domains. Our scoping review outlines the conceptual landscape of multimodal learning systems, identifies potential gaps, and provides new perspectives on adaptive multimodal system design: intertwining learning data for meaningful insights into learning, designing effective feedback, and implementing them in diverse learning domains.

@en

Applicability testing of constructive computational mechanisms (CCMs) is a new challenge for both the academia and the industry. The overwhelming majority of the existing validation approaches focuses on the internal validity of CCMs (e.g. consistency, bias), while there is a shortage of efficient approaches for assessing the external validity (e.g. applicability, reusability). The objective of this paper is to clarify the concepts and criteria, and to develop an approach for a systematic evaluation of the applicability of a given CCM to cases that were not considered at design time. The approach is adapted from the validation square approach (VSA). The adapted methodology (A-VSA) makes it possible to evaluate CCMs from (a) theoretical structural, (b) empirical structural, (c) theoretical performance, and (d) empirical performance dimensions. Altogether eight indicators are introduced that support the evaluation process. The effectiveness of the A-VSA was confirmed through a case study, in which a specific CCM is considered and the strategy of the A-VSA was operationalized with three completely different application cases. As evidenced by the results, the proposed A-VSA establishes a tight coupling among the enablers embraced by a CCM and the aspects of theoretical and empirical validation, which approves the approach to be an efficient tool for defining the range and/or the extent of applicability. The advantage of the A-VSA is that it offers a way to transfer qualitative applicability evaluation into quantitative applicability assessment, which allows the use of both subjective statements and mathematical modeling in applicability testing. The results of the assessment can guide the adaptation work of a CCM when applied to an out-of-domain application.

@en

Maintaining a complex system, such as a modern production line, is a knowledge-intensive task. Many firms use maintenance reports as a decision support tool. However, reports are often poor quality and tedious to compile. A Conversational User Interface (CUI) could streamline the reporting process by validating the user's input, eliciting more valuable information, and reducing the time needed. In this paper, we use a Technology Probe to explore the potential of a CUI to create instructional maintenance reports. We conducted a between-groups study (N = 24) in which participants had to replace the inner tube of a bicycle tire. One group documented the procedure using a CUI while replacing the inner tube, whereas the other group compiled a paper report afterward. The CUI was enacted by a researcher according to a set of rules. Our results indicate that using a CUI for maintenance reports saves a significant amount of time, is no more cognitively demanding than writing a report, and results in maintenance reports of higher quality.

@en

We present OpenFish: an open source soft robotic fish which is optimized for speed and efficiency. The soft robotic fish uses a combination of an active and passive tail segment to accurately mimic the thunniform swimming mode. Through the implementation of a novel propulsion system that is capable of achieving higher oscillation frequencies with a more sinusoidal waveform, the open source soft robotic fish achieves a top speed of 0.85 m/s . Hereby, it outperforms the previously reported fastest soft robotic fish by 27% . Besides the propulsion system, the optimization of the fish morphology played a crucial role in achieving this speed. In this work, a detailed description of the design, construction and customization of the soft robotic fish is presented. Hereby, we hope this open source design will accelerate future research and developments in soft robotic fish.@en

We present a novel DC motor driven soft robotic fish which is optimized for speed and efficiency based on experimental, numerical and theoretical investigation into oscillating propulsion. Our system achieves speeds up to 0.85 m/s, outperforming the previously reported fastest free swimming soft robotic fish by a significant margin of 27%. A simple and effective wire-driven active body and passive compliant body are used to mimic highly efficient thunniform swimming. The efficient DC motor to drive the system decreases internal losses compared to other soft robotic oscillating propulsion systems which are driven by one or multiple servo motors. The DC motor driven design allows for swimming at higher frequencies. The current design has been tested up to a tailbeat frequency of 5.5 Hz, and can potentially reach much higher frequencies.

@en

Bicycle production has not changed much over the last 100 years, it is still performed mainly by manual labor in mass production. During the global pandemic, the demand for ecologically friendly and customized transport has increased. Hence, customers start to impose the same requirements on bikes as on cars: they want more customized products and short delivery time. This publication describes an approach to transform bicycle manufacturing towards human-robot co-production to enable smaller batch sizes and production on-shoring. We list the challenges of this transformation, our applied methods, and presents preliminary results of the cobot-driven prototypes.

@en

How to visualize recorded production data in Virtual Reality? How to use state of the art Augmented Reality displays that can show robot data? This paper introduces an opensource ICT framework approach for combining Unity-based Mixed Reality applications with robotic production equipment using ROS Industrial. This publication gives details on the implementation and demonstrates the use as a data analysis tool in the context of scientific exchange within the area of Mixed Reality enabled human-robot co-production.

@en

Delivery robots are being deployed on sidewalks, but do we actually know which conflicts we should aim to avoid in the design and creation of these systems? Current approaches to social navigation focus on implementing well-established social norms such as proxemics, but it is dubious if these norms are sufficiently applicable to the context of dynamic interactions between such robots and pedestrians. We argue that, to get rich insight in the actual conflicts, we should confront them within their context. Based on this argument, we outline a new method for user observation that aims to elicit and explore representative social conflicts by ignoring humans: context-confrontation. Our first preliminary observations using this method suggest unexpected and novel conflicts, well outside of what current approaches seem to be focusing on – perhaps we, the designers/engineers of these robotic systems, should get out there a bit more, to find the conflicts that really matter.@en

Voice-enabled assistants, such as Alexa and Google Assistant, are among the fastest-growing information technology applications. Their technological foundation matured over the last years and reached a point where new application areas in challenging business environments become a certainty. Maintenance in manufacturing is one of these areas. This paper presents expectations, requirements, and a concept for a voice-enabled digital intelligent assistant that supports maintenance activities. We identified process monitoring, task execution, reporting, problem-solving, and maintenance planning as the key functional modules for an assistant. Realizing them depends on basic, utility, and maintenance functions. Our discussion states that all fundamental technologies and tools to realize an assistant for maintenance exist, but they have constraints. For instance, Speech-to-Text mechanisms lack transparent and performant solutions, and natural language understanding must rely on small datasets, which is challenging. We argue that continuous improvement and systematic evaluation of an assistant prototype is important to create high-quality training data. Trial-and-error is common because some technologies still mature, and conversation designers lack design patterns for the maintenance domain. Challenges for system adoption include providing an outstanding user experience, handling factory-specific jargon, and the limited availability of easy-to-use data exchange interfaces for machines and business applications. We conclude that further efforts on interoperability, technology stack management, AI-focused change management, and education programs are necessary. Furthermore, the accountability of AI systems is a cost factor for the assistant’s service providers and the client companies in manufacturing – AI insurance services, human-in-the-loop functions, user training, and professional education are actions to address this issue.@en

This paper proposes to use Virtual Reality scenarios to explore the reaction of stakeholders within an innovation process in the context of the introduction of robots working in close collaboration with users. The goal is to design the system upfront in such a way, that it is not perceived as a threat to the worker or his/her job. Within the responsible research and innovation approach, the introduction of new technology needs to be accompanied by a careful investigation of the thoughts and feelings of all stakeholders. Especially workers who are currently not working with robots but their workspace is currently undergoing an Industry 4.0 driven transformation, experience fear, that this new technology will make their jobs redundant. On the other hand, it can be observed, that successful robot interaction processes, on the one hand, increase the overall productivity, but also can enhance human well-being. The feeling of 'teamwork' with the artificial intelligence entity can develop to be equally positive and motivating. To be able to design future workspaces which will result in a 'teamwork' perception instead of the 'fear' perception, the use of VR can be applied.

@en

When mobile urban robots will share the sidewalk with people, the resulting interactions can cause unexpected undesirable outcomes to emerge – from people running away scared to people deliberately teasing and harassing such systems. How can we design such AI systems to aptly handle the unexpected? Directly anticipating and/or detecting these kinds of situations will inherently be unreliable; they are unexpected, after all. And yet, there exists a very clear signal for social slip-ups: the emotional response of people. We thus argue that such systems need to be imbued with a capacity to interpret the socio-emotional reactions to their own behavior.@en

Cyber-physical systems (CPSs) have sophisticated control mechanisms that help achieve optimal system operations and services. These mechanisms, imply considering multiple signal inputs in parallel, to timely respond to varying working conditions. Despite the advantages that control mechanisms convey, they bring new challenges in terms of failure prevention. The compensatory action the control exerts cause a fault masking effect, hampering fault diagnosis. Likewise, the multiple information inputs CPSs have to process can affect the timely system response to faults. This article proposes a failure prognosis method, which combines time series-based forecasting methods with statistically based classification techniques in order to investigate system degradation and failure forming on system levels. This method utilizes a new approach based on the concept of the system operation mode (SOM) that offers a novel perspective for health management that allows monitoring the system behavior, through the frequency and duration of SOMs. Validation of this method was conducted by systematically injecting faults in a cyber-physical greenhouse testbed. The obtained results demonstrate that the degradation and fault forming process can be monitored by analyzing the changes of the frequency and duration of SOMs. These indicators made possible to estimate the time to failure caused by various failures in the conducted experiments.

@en

Nautical traffic management in The Netherlands is shifting from local traffic control to corridor traffic management. Current traffic management systems do not sufficiently support operators in perceptual and cognitive process to interpret and understand the large amounts of information needed for corridor traffic management. Newly developed user interface concepts aim to overcome deficiencies of current interface designs that insufficiently support situation awareness assessment. The effects of these new user interfaces, however, are insufficiently known due to the intricate relations between situation awareness, task performance, and workload. The objective of this study is to evaluate the effects of the three previously developed user interface concepts on operators’ situation awareness, task performance, and workload to gain better insights into the benefits and limitations of the user interface design concepts. The effects were tested in a simulator environment. The results show that user interface features of an integrated user interface allowed operators to apply more effective information processing, which resulted in better task performance. Features of a context-dependent adaptable user interface triggered proactive behavior of operators, which resulted in better task performance for tasks in which operators require insight into future activities of the elements in the environment.

@en

Hazard-intense applications of cyber-physical systems (CPSs), such as the evacuation of a building on fire, require personalized informing on the basis of a real-time assessment of dynamic context. In this paper, a context-dependent message construction mechanism (CD-MCM) is proposed with the objectives (i) to inform people about the emergence and development of a situation unsafe for them, and (ii) to instruct them what they have to do according to an adaptively computed action plan. To achieve this, the concepts of 'situation' and 'impact indicator' have been introduced in order to facilitate the computation of personalized action plans and to send messages about the level of danger and the requested actions. In both activities, the inferred implications of situations are used as the basis of informing the involved people. The messages are adapted to actual situations. In addition, the concept of 'relevance indicator' was utilized to assess the significance of the standing situations for the concerned people in a quasi-real-time manner. The level of danger was evaluated for each person by totaling the values of the situation-related relevance indicators. This was also used to select the proper message templates from the predefined alternatives in the process of message construction. The personalized messages were generated based on the chosen message template and the various message components describing concerned situations or providing instructions. The CD-MCM was validated in a simulated indoor fire evacuation guiding application. In the practical evaluation of the quality of the generated messages, a sample of test people was involved. The results of the evaluation show that the messages generated by the proposed CD-MCM lead to more effective messaging about the personal context and the expected actions than the messages constructed by using static context information only. The reason is that the proposed template-based message construction mechanism facilitates the appropriateness as well as the articulation of the contents of context-sensitive personalized messages.

@en

Malfunction or breakdown of certain mission critical systems (MCSs) may cause losses of life, damage the environments, and/or lead to high costs. Therefore, recognition of emerging failures and preventive maintenance are essential for reliable operation of MCSs. There is a practical approach for identifying and forecasting failures based on the indicators obtained from real life processes. We aim to develop means for performing active failure diagnosis and forecasting based on monitoring statistical changes of generic signal features in the specific operation modes of the system. In this paper, we present a new approach for identifying emerging failures based on their manifestations in system signals. Our approach benefits from the dynamic management of the system operation modes and from simultaneous processing and characterization of multiple heterogeneous signal sources. It improves the reliability of failure diagnosis and forecasting by investigating system performance in various operation modes, includes reasoning about failures and forming of failures using a failure indicator matrix which is composed of statistical deviation of signal characteristics between normal and failed operations, and implements a failure indicator concept that can be used as a plug and play failure diagnosis and failure forecasting feature of cyber-physical systems. We demonstrate that our method can automate failure diagnosis in the MCSs and lend the MCSs to the development of decision support systems for preventive maintenance.

@en

Malfunction or breakdown of certain mission critical systems (MCSs) may cause losses of life, damage the environments, and/or lead to high costs. Therefore, recognition of emerging failures and preventive maintenance are essential for reliable operation of MCSs. There is a practical approach for identifying and forecasting failures based on the indicators obtained from real life processes. We aim to develop means for performing active failure diagnosis and forecasting based on monitoring statistical changes of generic signal features in the specific operation modes of the system. In this paper, we present a new approach for identifying emerging failures based on their manifestations in system signals. Our approach benefits from the dynamic management of the system operation modes and from simultaneous processing and characterization of multiple heterogeneous signal sources. It improves the reliability of failure diagnosis and forecasting by investigating system performance in various operation modes, includes reasoning about failures and forming of failures using a failure indicator matrix which is composed of statistical deviation of signal characteristics between normal and failed operations, and implements a failure indicator concept that can be used as a plug and play failure diagnosis and failure forecasting feature of cyber-physical systems. We demonstrate that our method can automate failure diagnosis in the MCSs and lend the MCSs to the development of decision support systems for preventive maintenance.@en

Smart CPSs (S-CPSs) have been evolving beyond what was identified by the traditional definitions of CPSs. The objective of our research is to investigate the concepts and implementations of reasoning processes for S-CPSs, and more specifically, the frameworks proposed for the fuzzy front end of their reasoning mechanisms. The objectives of the paper are: (i) to analyze the framework concepts and implementations of CPS, (ii) to review the literature concerning system-level reasoning and its enablers from the points of view of the processed knowledge, building awareness, reasoning mechanisms, decision making, and adaptation. Our findings are: (i) awareness and adaptation behaviors are considered as system-level smartness of S-CPSs that are not achieved by traditional design approaches; (ii) model-based and composability approaches insufficiently support the development of reasoning mechanisms for S-CPSs; (iii) frameworks for development of reasoning in S-CPS should support compositional design. Based on the conclusions above, we argue that coping with the challenges of compositionality requires both software-level integration and holistic fusion of knowledge by means of semantic transformations. This entails the need for a multi aspect framework that is able to capture at least conceptual, functional, architectural, informational, interoperation, and behavioral aspects. It needs further investigation if a compositionality enabling framework should appear in the form of a meta-framework (abstract) or in the form of a semantically integrated (concrete) framework.

@en