4D-scans of dynamic deformable human body parts help researchers have a better understanding of spatiotemporal features. However, reconstructing 4D-scans utilizing multiple asynchronous cameras encounters two main challenges: 1) finding dynamic correspondences among different frames captured by each camera at the timestamps of the camera in terms of dynamic feature recognition, and 2) reconstructing 3D-shapes from the combined point clouds captured by different cameras at asynchronous timestamps in terms of multi-view fusion. Here, we introduce a generic framework able to 1) find and align dynamic features in the 3D-scans captured by each camera using the nonrigid-iterative-closest-farthestpoints algorithm; 2) synchronize scans captured by asynchronous cameras through a novel ADGC-LSTMbased-network capable of aligning 3D-scans captured by different cameras to the timeline of a specific camera; and 3) register a high-quality template to synchronized scans at each timestamp to form a highquality 3D-mesh model using a non-rigid registration method. With a newly developed 4D-foot-scanner, we validate the framework and create the first open-access data-set, namely the 4D-feet. It includes 4Dshapes (15 fps) of the right and left feet of 58 participants (116 feet including 5147 3D-frames), covering significant phases of the gait cycle. The results demonstrate the effectiveness of the proposed framework, especially in synchronizing asynchronous 4D-scans.

@en

Preserving features or local shape characteristics of a mesh using conventional non-rigid registration methods is always difficult, as the preservation and deformation are competing with each other. The challenge is to find a balance between these two terms in the process of the registration, especially in presence of artefacts in the mesh. We present a non-rigid Iterative Closest Points (ICP) algorithm which addresses the challenge as a control problem. An adaptive feedback control scheme with global asymptotic stability is derived to control the stiffness ratio for maximum feature preservation and minimum mesh quality loss during the registration process. A cost function is formulated with the distance term and the stiffness term where the initial stiffness ratio value is defined by an Adaptive Neuro-Fuzzy Inference System (ANFIS)-based predictor regarding the source mesh and the target mesh topology, and the distance between the correspondences. During the registration process, the stiffness ratio of each vertex is continuously adjusted by the intrinsic information, represented by shape descriptors, of the surrounding surface as well as the steps in the registration process. Besides, the estimated process-dependent stiffness ratios are used as dynamic weights for establishing the correspondences in each step of the registration. Experiments on simple geometric shapes as well as 3D scanning datasets indicated that the proposed approach outperforms current methodologies, especially for the regions where features are not eminent and/or there exist interferences between/among features, due to its ability to embed the inherent properties of the surface in the process of the mesh registration.

@en

Personalized designs bring significant added value to the products and the users. However, they also pose challenges on the product design process. For instance, for products for personalized fit ,each may differ subject to each user’s body shape and preference. Presently, there exist knowledge and methods which suppor designing personalized products/services ,with sample applications in the fields of medical products, shoes, clothing industry, etc. Meanwhile, the major steps in these methods are manual or semi-automated, thus designing Ultra Personalized Products and Services(UPPS) can be a tedious and time-consuming task. Furthermore ,the design process is usually not optimized and most applications are employing ad-hoc approaches. Designers need a systematic approach to designing UPPS...@en

Preserving features of a surface as characteristic local shape properties captured e.g. by curvature, during non-rigid registration is always difficult where finding meaningful correspondences, assuring the robustness and the convergence of the algorithm while maintaining the quality of mesh are often challenges due to the high degrees of freedom and the sensitivity to features of the source surface. In this paper, we present a non-rigid registration method utilizing a newly defined semi-curvature, which is inspired by the definition of the Gaussian curvature. In the procedure of establishing the correspondences, for each point on the source surface, a corresponding point on the target surface is selected using a dynamic weighted criterion defined on the distance and the semi-curvature. We reformulate the cost function as a combination of the semi-curvature, the stiffness, and the distance terms, and ensure to penalize errors of both the distance and the semi-curvature terms in a guaranteed stable region. For a robust and efficient optimization process, we linearize the semi-curvature term, where the region of attraction is defined and the stability of the approach is proven. Experimental results show that features of the local areas on the original surface with higher curvature values are better preserved in comparison with the conventional methods. In comparison with the other methods, this leads to, on average, 75\%, 8\% and 82\% improvement in terms of quality of correspondences selection, quality of surface after registration, and time spent of the convergence process respectively, mainly due to that the semi-curvature term logically increases the constraints and dependency of each point on the neighboring vertices based on the point's degree of curvature.@en

Aiming at a more efficient and accurate performance of parallel manipulators in the existence of complex kinematics and dynamics, a robust generalizable methodology is proposed here for an integrated 6-DOF Stewart platform with rotary time-delayed actuators torque control. The suggested method employs a time-delay linear–quadratic integral regulator with online artificial neural network gain adjustment. The unknown time-delay is estimated through a novel robust adaptive estimator. The global asymptotic stability of the estimator is proved via a Lyapunov function. The controller is developed in MATLAB software and implemented on the robot designed in ADAMS software to ensure that the real-time tracking error of a nonlinear system with an unknown time-delay is kept to a minimum. The sensitivity of the controller to the parameter choices is studied via implementing the controller in ADAMS software and is validated by investigating the performance on a naturalistic fabricated robot. The approach is assessed using simulation and experimental tests to show the feasibility, optimum, and zero-error convergence of the technique developed.@en

Fitting apparel and apparel in performing different activities is essential for the functional yet comfortable experience of the user. 4D scans, i.e. 3D scans in continuous timestamps, of the body (part) in performing those activities are the basis for the design of garments/apparel in 4D. In this paper, we proposed a semi-automatic workflow for constructing 4D scans of the body parts with the emphasis on registering noisy scans at a given timestamp. Continuous 3D scans regarding the moving body parts are captured first from different depth cameras from different view angles. In a given timestamp, the collected 3D scans are roughly aligned to a template using the rigid Iterative Closest Points (ICP) algorithm. Then these scans are further registered using a newly proposed non-rigid Iterative Closest-Farthest Points (ICFP) algorithm, in which correspondences between the source and the target are established by either closest or farthest points based on the newly defined logical distance concept and the probability theory. Experimental results indicated that the ICFP method is robust against noise and the scanning accuracy can be as high as 3.4 %. It also reveals that, for the human foot, the differences of ball width and ball angles between the loaded and the unloaded situation can be as large as 8 mm and 2 degrees, respectively. This highlights the importance of using 4D scan in designing garments and apparel.

@en

Predicting pediatric spinal deformity (PSD) from X-ray images collected on the patient’s initial visit is a challenging task. This work builds on our previous method and provides a novel bio-informed framework based on a mechanistic machine learning technique with dynamic patient-specific parameters to predict PSD. We provide a geometry-based bone growth model that can be utilized in a range of applications to enhance the bio-informed mechanistic machine learning framework. The proposed technique is utilized to examine and predict spine curvature in PSD cases such as adolescent idiopathic scoliosis. The best fit of a segmented 3D volumetric geometry of the human spine acquired from 2D X-ray images is employed. Using an active contour model based on gradient vector flow snakes, the anteroposterior and lateral views of the X-ray images are segmented to derive the 2D contours surrounding each vertebra. Using minimal user input, the snake parameters are calibrated and automatically computed over the dataset, resulting in fast image segmentation and data collection. The 2D segmented outlines of each vertebra are transformed into a 3D image segmentation result. The Iterative Closest Point mesh registration technique is then used to establish a mesh morphing approach and creates a 3D atlas spine model. Using the comprehensive 3D volumetric model, one can automatically extract spinal geometry data as inputs to the mechanistic machine learning network. Moreover, the proposed bio-informed deep learning network with the modified bone growth model achieves competitive or even superior performance against other state-of-the-art learning-based methods.Please check and confirm if the author names and initials are correct for “Yongjie Jessica Zhang” and “Wing Kam Liu”.We confirm they are correct.

@en

Personalized designs bring added value to the products and the users. Meanwhile, they also pose challenges to the product design process as each product differs. In this paper, with the focus on personalized fit, we present an overview as well as details of the personalized design process based on design practice. The general workflow of personalized product design is introduced first. Then different steps in the workflow such as human data/parameters acquisition, computational design, design for digital fabrication, and product evaluation are presented. Tools and methods that are often used in different steps in the process are also outlined where in human data acquisition, 3D scanning, and digital human models are addressed. For computational design, the use of computational thinking tools such as abstraction, decomposition, pattern recognition and algorithms are discussed. In design for digital fabrication, additive manufacturing methods (e.g. FDM), and their requirements on the design are highlighted. For product evaluation, both functional evaluation and usability evaluation are considered and the evaluation results can be the starting point of the next design iteration. Finally, several case studies are presented for a better understanding of the workflow, the importance of different steps in the workflow and the deviations in the approach regarding different contexts. In conclusion, we intend to provide designers a holistic view of the design process in designing personalized products as well as help practitioners trigger innovations regarding each step of the process.@en

Optical motion capturing explains the three-Dimensional (3D) position estimation of points through triangulation employing several depth cameras. Prosperous performance relies on level of visibility of points from different cameras and the overlap of captured meshes in-between. Generally, the accuracy of the estimation is practically based on the camera parameters e.g., location and orientations. Accordingly, the camera network configurations play a key role in the quality of the estimated mesh. This paper proposes an optimal approach for camera placement based on characteristics of a depth camera D435i - Intel RealSense. The optimal problem includes a cost function that contains several minimisation and maximisation terms. The minimisation terms are distance of the cameras to the center of the scanning object, resolution error, and sparsity. And the maximisation terms are distance between each two pair of cameras, percent of captured point from an object, and the level of overlap between cameras. The object is designed based on practical experiments of human walking and is a bounding box around one step of dynamic foot work-space from heel strike posture to toe-off posture. The accuracy and robustness of the algorithms are assessed via experiment measurement, and sensitivity to the number of cameras is investigated. Accordingly, the experiment results determined that the scanning accuracy can be as high as 2.5 % based on a reference scan with a high-end scanner (Artec Eva).

@en

In the presence of complicated kinematic and dynamic, we present a generalizable robust control technique for the 6-Degree of Freedom (6DoF) Stewart integrated platform with revolving, time-delayed torque control actuators to achieve faster, and reliable efficiency for parallel control manipulators. The suggested optimal solution involves the construction of a time-delay Linear Quadratic Integral (LQI) controller integrated with an on-line Artificial Neural Network (ANN) as the cost function gain tuner. The controller is formulated to robustly mitigate the nonlinear system’s real-time tracking error with large time-delay, which is implemented via ADAMS software. The method is validated through simulation experiments to demonstrate that the developed methodology is practical, optimum, and zero-error convergence.@en

Scoliosis, an abnormal curvature of the human spinal column, is characterized by a lateral deviation of the spine, accompanied by axial rotation of the vertebrae. Adolescent Idiopathic Scoliosis (AIS) is the most common type, affecting children between ages 8 to 18 when bone growth is at its maximum rate. We propose a mechanistic machine learning algorithm in order to study patient-specific AIS curve progression, which is associated with the bone growth and other genetic and environmental factors. Two different frameworks are used to analyse and predict curve progression, one with implementing clinical data extracted from 2D X-ray images and the other one with incorporating both clinical data and physical equations governing the non-uniform bone growth. The physical equations governing bone growth are affiliated with calculating all stress components at each region. The stress values are evaluated through a surrogate finite element simulation and a bone growth model on a detailed patient-specific geometry of the human spine. We also propose a patient-specific framework to generate the volumetric model of human spine which is partitioned into different tissues for both vertebra and intervertebral disc. It is shown that implementing physical equations governing bone growth into the prediction framework will notably improve the prediction results as compared to only using clinical data for prediction. In addition, we can predict curve progression at ages outside the range of training samples.@en

Aiming at operating optimally minimizing error of tracking and designing control effort, this study presents a novel generalizable methodology of an optimal torque control for a 6-degree-of-freedom Stewart platform with rotary actuators. In the proposed approach, a linear quadratic integral regulator with the least sensitivity to controller parameter choices is designed, associated with an online artificial neural network gain tuning. The nonlinear system is implemented in ADAMS, and the controller is formulated in MATLAB to minimize the real-time tracking error robustly. To validate the controller performance, MATLAB and ADAMS are linked together and the performance of the controller on the simulated system is validated as real time. Practically, the Stewart robot is fabricated and the proposed controller is implemented. The method is assessed by simulation experiments, exhibiting the viability of the developed methodology and highlighting an improvement of 45% averagely, from the optimum and zero-error convergence points of view. Consequently, the experiment results allow demonstrating the robustness of the controller method, in the presence of the motor torque saturation, the uncertainties, and unknown disturbances such as intrinsic properties of the real test bed.@en

Sophisticated surgeons are widely indicating the use of surgical robots in order to reject human error, increase precision, and speed. Among well-known robotic mechanisms, parallel robots are broadly more investigated regarding their special characters as higher acceleration, speed, and accuracy, and less weight. Specific surgical procedures confine, and restrict their workspace, while controlling and validating the robots are complicated regarding to their complex dynamic. To this end, in this paper, a 6-DOF robot, with rotary manipulators, is designed and controlled. Addressing nonlinearity of parallel robots, an innovative methodology is formulated to robustly penalize the error of tracking at end effector through a Linear Quadratic Integral (LQI) regulator with online Artificial Neural Network (ANN) gain tuning, based on non-linear model in format of a Linear Time Invariant (LTI) model. As validation, the controller is implemented using MATLAB on the non-linear model designed in Adams software online. Simulation results demonstrate the optimal controller penalizing the error while minimizing torque on each of the rotary manipulator. In addition, the method defines the workspace of both the states and torques, which is an introduction to comprehensive design of such robots.

@en

A high-fidelity digital representation of (part of) the human body is a key enabler for integrating humans in a digital twin. Among different parts of human body, building the model of the hand can be a challenging task due to the posture deviations among collected scans. In this article, we proposed a posture invariant statistical shape model (SSM) of the human hand based on 59 3D scans of human hands. First, the 3D scans were spatially aligned using a Möbius sphere-based algorithm. An articulated skeleton, which contains 20 bone segments and 16 joints, was embedded for each 3D scan. Then, all scans were aligned to the same posture using the skeleton and the linear blend skinning (LBS) algorithm. Three methods, i.e., principal component analysis (PCA), kernel-PCA (KPCA) with different kernel functions, and independent component analysis (ICA), were evaluated in the construction of the SSMs regarding the compactness, the generalization ability, and the specificity. The PCA-based SSM was selected, where 20 principal components were used as parameters for the model. Results of the leave-one-out validation indicate that the proposed model was able to fit a given 3D scan of the human hand at an accuracy of 1.21 ± 0.14 mm. Experiment results also indicated that the proposed SSM outperforms the SSM that was built on the scans without posture correction. It is concluded that the proposed posture correction approach can effectively improve the accuracy of the hand SSM and therefore enables its wide usage in human-integrated digital twin applications.

@en

Sophisticated surgeons are widely indicating the use of surgical robots in order to reject human error, increase precision, and speed. Among well-known robotic mechanisms, parallel robots are broadly more investigated regarding their special characters as higher acceleration, speed, and accuracy, and less weight. Specific surgical procedures confine and restrict their workspace, while controlling and validating the robots are complicated based on their complex dynamic. To this end, in this paper, a 6-DOF robot, with linear manipulators, is designed and controlled. Addressing the inherent nonlinearity of the robot mechanism, an adaptive PID manipulator is employed and validated with nonlinear model. Thus, this paper aims to implement the controller using MATLAB on the nonlinear model designed in Adams software as online. Furthermore, as feasibility study, a 3-axis gyro sensor is calibrated and used to produce complex real movements and desires, which is sending real time signals to MATLAB with microcontroller ATmega 2560. Simulation results show that the adaptive controller identifies the system's dynamic and proceed to reducing error of the position by the time. In addition, the method defines the workspace of both the states and forces which can be an introduction to comprehensive design of such robots.

@en