Real-world applications of Artificial Intelligence (AI) in architecture have been explored more recently at Technical University (TU) Delft by integrating AI in Design-to-Robotic-Production-Assembly and -Operation (D2RPA&O) methods. These embed robotics into building processes and buildings by linking computational design with robotic construction and/ or operation of building components and buildings. This paper presents two case studies in which AI-supported D2RA is implemented in a multidisciplinary approach that requires the integration of research domains such as architecture, robotics, computer and material science.@en

In Industry 5.0, cognitive robots and workers will engage in evolving and reciprocal relations, which we call worker-robot relationships (WRRs). To enable evidence-based work futures with workers, we must co-develop WRRs and understand their impact on work, workers, management, and society. To this end, we posit that the HRI field should work beyond disciplines and include value-driven and plural perspectives through transdisciplinary research done with and for workers. However, WRRs and transdisciplinarity pose unique technical, design, and methodological challenges yet to be explored. We propose a workshop to engage the HRI community working on Industry 5.0, aiming at 1) taking stock of current WRR-related challenges in relevant disciplines, 2) collectively kick-off the exploration of a joint research agenda, 3) preliminary examining if and how transdisciplinarity could help the HRI community, and 4) start discussing how to deal with such complex knowledge integration in practice.

@en

Humans can effortlessly grasp various objects when the fingers are in direct physical interaction with the object. However, the same actions become complicated when grasping has to be performed via a teleoperated remote robot due to a lack of direct contact and reduced sensory information. Having a fully autonomous remote robot can eliminate the problem of lack of proper feedback to the human operator, nevertheless, it also prevents human control over the remote robot's grasping actions. In this paper, we propose a semi-autonomous controller for a teleoperated robot grasping where the human operator controls the grasping aperture while the robot controls the impedance of the gripper. When the operator grasps an object with the remote robot, the semi-autonomous controller maintains the grip force by adjusting stiffness. The developed stiffness adjustment approach derives from the concept of grip force safety margin, which is the central regulation principle humans use to maintain a light grasp yet prevent object slippage. To detect incipient slippage, we use a tactile sensor that captures the local deformations due to the contact and interprets them to determine the proximity to the object's slip. To validate the proposed method, we performed experiments on a teleoperation system composed of Force Dimension sigma.7 haptic interface and a KUKA LBR iiwa collaborative robot equipped with a custom-built gripper. The results show that the proposed controller is robust to external perturbations while it adapts to the operator's commands to prevent grasped object slippage.@en

In this research, we propose a novel method to estimate and monitor rotator cuff tendon strains during active robotic-assisted rehabilitation. This is a significant step forward from our previous work which estimated these tendon strains during passive exercises (i.e., no muscle activity). Physiotherapists adopt a cautious approach to rehabilitation treatment to prevent (re-) injury given the limited available information about the shoulder's internal condition. By leveraging a robotic device and a musculoskeletal model, our approach provides quantitative information on the risk of re-injury by monitoring the strains of the rotator cuff tendons during shoulder movements with the application of external loads. Muscle strains depend on the shoulder kinematic state and muscle activations, which makes it crucial to obtain physiologically realistic joint kinematics to estimate real-time muscle function. To obtain the strains, we utilize our muscle redundancy solver that incorporates constraints on model accelerations, the glenohumeral joint reaction forces, and active muscle dynamics. Using this algorithm along with force and pose data from a collaborative robotic arm, we demonstrate the ability to update our tendon strain estimates based on muscle activation during robotic-assisted rehabilitation exercises. The findings of our research pave the way for establishing improved therapy that considers the risk of injury to individual muscles and explores a broader and more personalized range of motion. By providing physiotherapists with valuable quantitative information on rotator cuff tendon strains, our method empowers them to optimize rehabilitation protocols and deliver more personalized and effective care. In addition, the system and method presented here comprise a tool capable of offering new insights into the relationship between the rotator cuff muscles, external forces, and shoulder kinematics.@en

Skill propagation among robots without human involvement can be crucial in quickly spreading new physical skills to many robots. In this respect, it is a good alternative to pure reinforcement learning, which can be time-consuming, or learning from human demonstration, which requires human involvement. In the latter case, there may not be enough humans to quickly spread skills to many robots. However, propagation among robots without direct human supervision can result in robotic skills mutating from the original source. This can be beneficial when better skills might emerge or when a new skill is obtained to be used for other similar tasks. However, it can also be dangerous in terms of task execution safety. This letter studies the mutation of a robotic skill when it is propagated from one robot to another during a physically collaborative task. We chose the collaborative sawing task as a study case since it involves complex two-agent physical interaction/coordination and because its periodic nature can facilitate repetitive learning. The study employs periodic Dynamic Movement Primitives and Locally Weight Regression to encode and learn the motion and impedance required to execute the task. To explore what influences mutation, we varied several control and environment conditions such as the maximum stiffness, robot base position, friction coefficient of the sawed object, and movement period. The results showed that the skill varied over propagation steps and we identified several key aspects of mutation such as movement length, movement offset, and trajectory shape. Based on the results we identified possible benefits (skill mutations useful for different settings or different tasks, and energy efficiency) and dangers (high forces and skill mutations becoming useless for the original task) of the mutation.

@en

The complexity of the human shoulder girdle enables the large mobility of the upper extremity, but also introduces instability of the glenohumeral (GH) joint. Shoulder movements are generated by coordinating large superficial and deeper stabilizing muscles spanning numerous degrees-of-freedom. How shoulder muscles are coordinated to stabilize the movement of the GH joint remains widely unknown. Musculoskeletal simulations are powerful tools to gain insights into the actions of individual muscles and particularly of those that are difficult to measure. In this study, we analyze how enforcement of GH joint stability in a musculoskeletal model affects the estimates of individual muscle activity during shoulder movements. To estimate both muscle activity and GH stability from recorded shoulder movements, we developed a Rapid Muscle Redundancy (RMR) solver to include constraints on joint reaction forces (JRFs) from a musculoskeletal model. The RMR solver yields muscle activations and joint forces by minimizing the weighted sum of squared-activations, while matching experimental motion. We implemented three new features: first, computed muscle forces include active and passive fiber contributions; second, muscle activation rates are enforced to be physiological, and third, JRFs are efficiently formulated as linear functions of activations. Muscle activity from the RMR solver without GH stability was not different from the computed muscle control (CMC) algorithm and electromyography of superficial muscles. The efficiency of the solver enabled us to test over 3600 trials sampled within the uncertainty of the experimental movements to test the differences in muscle activity with and without GH joint stability enforced. We found that enforcing GH stability significantly increases the estimated activity of the rotator cuff muscles but not of most superficial muscles. Therefore, a comparison of shoulder model muscle activity to EMG measurements of superficial muscles alone is insufficient to validate the activity of rotator cuff muscles estimated from musculoskeletal models.@en

While half of all construction tasks can be fully automated the other half relies to a certain degree on human support. This paper presents a Computer Vision (CV) and Human–Robot Interaction/Collaboration (HRI/C) supported Design-to-Robotic-Assembly (D2RA) approach that links computational design with robotic assembly. This multidisciplinary approach has been tested on a case study focusing on urban furniture and involving experts from respective disciplines and students.@en

In this work, we propose a method for monitoring and managing rotator-cuff (RC) tendon strains in human-robot collaborative physical therapy for shoulder rehabilitation. We integrate a high-resolution biomechanical model with a collaborative industrial robot arm and an impedance controller to provide feedback to a human subject, therapist or both, which prevents the subject from entering unsafe poses during rehabilitation. The biomechanical model estimates RC tendon strain as a function of human shoulder configuration, muscle activation and applied external forces. Subject- and injury-specific data are model estimates of strain that compose strain maps, which capture the relationship between the RC strains and movement of the shoulder degrees of freedom (DoF). High-strain regions of the strain map are identified as unsafe zones by clustering and ellipse fitting to smoothly demarcate these zones. These unsafe areas, which reflect increased risks of (re-)injury, are used to define parameters of an impedance controller and reference pose for real-time biomechanical safety control. Using strain maps we demonstrate both safe patient-led movements and teleoperated movements that prevent the subject from entering unsafe zones. In the teleoperated case, the physical therapist leads the patient remotely using a haptic device. The proposed method has the potential to improve the safety, range of motion, and volume of activity that a patient receives through robot-mediated physical therapy. We validated our approach using three experiments that demonstrate shoulder joint torques of less than 1 Nm during free motion with larger torques occurring only when the subject was asked to actively push into the unsafe boundary or, in the case of teleoperation, to resist the physical therapist.@en

Objective: Robotic endoscopes have the potential to dramatically improve endoscopy procedures, however current attempts remain limited due to mobility and sensing challenges and have yet to offer the full capabilities of traditional tools. Endoscopic intervention (e.g., biopsy) for robotic systems remains an understudied problem and must be addressed prior to clinical adoption. This paper presents an autonomous intervention technique onboard a Robotic Endoscope Platform (REP) using endoscopy forceps, an auto-feeding mechanism, and positional feedback. Methods: A workspace model is established for estimating tool position while a Structure from Motion (SfM) approach is used for target-polyp position estimation with the onboard camera and positional sensor. Utilizing this data, a visual system for controlling the REP position and forceps extension is developed and tested within multiple anatomical environments. Results: The workspace model demonstrates accuracy of 5.5% while the target-polyp estimates are within 5 mm of absolute error. This successful experiment requires only 15 seconds once the polyp has been located, with a success rate of 43% using a 1 cm polyp, 67% for a 2 cm polyp, and 81% for a 3 cm polyp. Conclusion: Workspace modeling and visual sensing techniques allow for autonomous endoscopic intervention and demonstrate the potential for similar strategies to be used onboard mobile robotic endoscopic devices. Significance: To the authors' knowledge this is the first attempt at automating the task of colonoscopy intervention onboard a mobile robot. While the REP is not sized for actual procedures, these techniques are translatable to devices suitable for in vivo application.

@en

In this work, we explore using computational musculoskeletal modeling to equip an industrial collaborative robot with awareness of the internal state of a patient to safely deliver physical therapy. A major concern of robot-mediated physical therapy is that robots may unwittingly injure patients. For patients with shoulder injuries this typically means the risk of tearing a rotator-cuff muscle tendon. Risk of reinjury hampers both human and robot therapists and it is the main reason for conservative physical therapy. Advances in human musculoskeletal modeling, however, can equip robots with additional perception of potential reinjury risks. While the ultimate goal is to improve the safety, range-of-motion and activity that patients receive through robot-mediated therapy, the aim of this letter is to develop and test a framework that enables the robot to understand the state of the patient and to execute physical therapy movements that demonstrate low injury risk and achieve a large range-of-motion in human subjects. We build on prior work in human-robot interaction via impedance control, but take robot awareness of the human to the next level by including and manipulating a musculoskeletal model in parallel to the patient. Taking the most common shoulder impairments (i.e., rotator-cuff tears) as an example, we demonstrate planned, model-based trajectories that minimize strain in these muscles and corresponding robot-mediated movements on healthy subjects. Our experiments suggest that musculoskeletal awareness is a promising approach to plan and deliver therapeutic movements that are safe and effective via an industrial robot.

@en

With significant progress being made toward improving endoscope technology such as capsule endoscopy and robotic endoscopy, the development of advanced strategies for manipulating, controlling, and more generally, easing the accessibility of these devices for physicians is an important next step. This article presents an autonomous navigation strategy for use in endoscopy, utilizing a state-dependent region estimation approach to allow for multimodal control design. This region estimator is evaluated for its accuracy in predicting yaw angle of the camera relative to the lumen center, and for estimating the location of the camera based on overall haustra morphology within the colon. To assess the utility of this region estimator, multimodal control is used to allow for autonomous navigation of the Endoculus, a robotic capsule endoscope, within a benchtop, to-scale, simulated colon. The estimation approach is presented and tested, demonstrating successful tracking of fixed velocity rotations at speeds up to 40^circ/s and allowing for curve anticipation approximately 10 cm before entering a curved section of the simulator. Finally, the multimodal control strategy utilizing this estimator is tested within the simulator over a variety of anatomic configurations. This strategy proves successful for navigation in both straight sections of this simulator and in tightly curved sections as small as 8 cm radius of curvature, with average velocities reaching 2.61 cm/s in straight sections and 0.99 cm/s in curved sections.

@en