Hierarchical Reinforcement Learning (HRL) provides an option to solve complex guidance and navigation problems with high-dimensional spaces, multiple objectives, and a large number of states and actions. The current HRL methods often use the same or similar reinforcement learning methods within one application so that multiple objectives can be easily combined. Since there is not a single learning method that can benefit all targets, hybrid Hierarchical Reinforcement Learning (hHRL) was proposed to use various methods to optimize the learning with different types of information and objectives in one application. The previous hHRL method, however, requires manual task-specific designs, which involves engineers’ preferences and may impede its transfer learning ability. This paper, therefore, proposes a systematic online guidance and navigation method under the framework of hHRL, which generalizes training samples with a function approximator, decomposes the state space automatically, and thus does not require task-specific designs. The simulation results indicate that the proposed method is superior to the previous hHRL method, which requires manual decomposition, in terms of the convergence rate and the learnt policy. It is also shown that this method is generally applicable to non-stationary environments changing over episodes and over time without the loss of efficiency even with noisy state information.

@en

Urban air mobility is a relatively new concept that has been proposed in recent years as a means of transporting passengers and goods in urban areas. It encompasses a diverse range of Vertical TakeOff and Landing (VTOL) vehicles that function more like passenger-carrying drones for on-demand transportation. Among them, the car-like VTOL is advantageous due to its compact configuration, safe rotors, high user affinity, and technological fashion. These characteristics are frequently derived from the flying car’s Ducted Fan Lift System (DFLS). This study aims to develop a method for the rapid design and the evaluation of the aerodynamic performance of the DFLS, to support the preliminary scheme demonstration of the ducted fan flying car. The proposed method uses blade element theory to design the unducted fan and applies momentum theory to calculate the aerodynamic thrust of the DFLS. The DFLS of a 1:3 scale verifier for a flying car scheme was designed and evaluated using the proposed method and a numerical method, respectively. To validate the proposed method, a prototype of the scale DFLS was manufactured and tested, and the result was compared with those of the proposed theoretical method and the numerical method. This study demonstrates that while both the theoretical and numerical methods are capable of designing an unducted fan accurately, the theoretical method is simpler and faster. Compared to the DFLS test results, the theoretical method’s average difference is approximately 1.9%. When evaluating the DFLS, the accuracy of the numerical calculation is reduced, and the difference is greater than 30% at low power. The theoretical method presented in this paper can be used to improve the aerodynamic design and evaluation efficiency of the DFLS and to aid in the configuration evaluation of VTOLs equipped with ducted fans.

@en

Optical flow-based control strategies have always inspired robotic scientists, especially those in the field of Micro Air Vehicles (MAVs), thanks to their computational efficiency and relative simplicity. A major problem is that the success of optical flow control is governed by the availability of distance estimates, while optical flow provides only the ratio of velocity to distance. Therefore, with only monocular visual information, the inherent nonlinearity of optical flow observables has imposed several challenges in the controller design. In this paper, we propose a newly formulated controller, Extended Incremental Nonlinear Dynamic Inversion (EINDI), to deal with nonlinearities in the system output, such as optical flow control problems. The proposed method unlocks the potential of its predecessor (INDI) in output feedback control by removing the common assumption of time-scale separation, allowing internal dynamics to exist, and requiring only the input and output measurements. Furthermore, the EINDI method has been implemented on an MAV and tested successfully for optical flow landing in a simulation and a real-world outdoor environment. Both simulation and flight test results show 1) good tracking performance of the EINDI control compared to the conventional feedback control, 2) smooth landing trajectories without any oscillation, and 3) fast adaptation of the EINDI control even for landings at different heights and desired setpoints.

@en

In recent years, Unmanned Aerial Vehicles (UAVs) are widely utilized in precision agriculture, such as tree plantations. Due to limited intelligence, these UAVs can only operate at high altitudes, leading to the use of expensive and heavy sensors for obtaining important health information of the plants. To fly at low altitudes, these UAVs must possess the capability of obstacle avoidance to prevent crashes. However, most current obstacle avoidance systems with active sensors are not applicable to small aerial vehicles due to the cost, weight, and power consumption constraints. To this end, this paper presents a novel approach to the autonomous navigation of a small UAV in tree plantations only using a single camera. As the monocular vision does not provide depth information, a machine learning model, Faster Region-based Convolutional Neural Network (Faster R-CNN), was trained for the tree trunk detection. A control strategy was implemented to avoid the collision with trees. The detection model uses image heights of detected trees to indicate their distances from the UAV and image widths between trees to find the widest obstacle-free space. The control strategy allows the UAV to navigate until any approaching obstacle is detected and to turn to the safest area before continuing its flight. This paper demonstrates the feasibility and performance of the proposed algorithms by carrying out 11 flight tests in real tree plantation environments at two different locations, one of which is a new place. All the successful results indicate that the proposed method is accurate and robust for autonomous navigation in tree plantations.

@en

The interest in building hybrid Unmanned Aerial Vehicles (UAVs) is increasing intensively due to its capability to perform Vertical Take-Off and Landing (VTOL), in addition to forward flight. With this capability, the hybrid UAVs are highly on demand in various industries. In this paper, a fixed-wing VTOL UAV with a novel configuration of a dual rotor-embedded wing was designed and developed. The methodology used in the design process adopted the traditional sizing and aerodynamic estimation method with advanced computational simulations and estimation approaches. The design was determined based on a thorough analysis of weight contribution, aerodynamics, propulsion, and stability and control. The results show that the UAV's preliminary design has successfully reached a total weight of 1.318 kg, achieved a high lift-to-drag ratio of approximately 4, and ensured stable flights with Level 1 flying qualities. A fixed-wing VTOL prototype was developed and fabricated based on the final design parameters using a low-cost hand lay-up process.

@en

Monocular optical flow has been widely used to detect obstacles in Micro Air Vehicles (MAVs) during visual navigation. However, this approach requires significant movement, which reduces the efficiency of navigation and may even introduce risks in narrow spaces. In this paper, we introduce a novel setup of self-supervised learning (SSL), in which optical flow cues serve as a scaffold to learn the visual appearance of obstacles in the environment. We apply it to a landing task, in which initially ‘surface roughness’ is estimated from the optical flow field in order to detect obstacles. Subsequently, a linear regression function is learned that maps appearance features represented by texton distributions to the roughness estimate. After learning, the MAV can detect obstacles by just analyzing a still image. This allows the MAV to search for a landing spot without moving. We first demonstrate this principle to work with offline tests involving images captured from an on-board camera and then demonstrate the principle in flight. Although surface roughness is a property of the entire flow field in the global image, the appearance learning even allows for the pixel-wise segmentation of obstacles.@en

A control strategy is proposed to deal with the fundamental gain selection problem of optical flow landings. It involves detecting the height by means of an oscillating movement and setting the control gains accordingly at the start of a landing. Then, during descent, the gains are reduced exponentially, with mechanisms in place to ensure high-performance landings. Real-world experiments with a quadrotor demonstrate successful landings in both indoor and outdoor environments.

@en

Monocular vision is increasingly used in micro air vehicles for navigation. In particular, optical flow, inspired by flying insects, is used to perceive vehicle movement with respect to the surroundings or sense changes in the environment. However, optical flow does not directly provide us the distance to an object or velocity, but the ratio of them. Thus, using optical flow in control involves nonlinearity problems which add complexity to the controller. To deal with that, we propose an algorithm that estimates distance and velocity of the vehicle based on optical flow measured from a monocular camera and the knowledge of control inputs. This algorithm applies an extended Kalman filter to state estimation and uses the estimates for landing control. We implement and test our algorithm in computer simulation and on board a Parrot AR.Drone 2.0 to demonstrate its feasibility for micro air vehicles landings. Results of the simulation and multiple flight tests show that the algorithm is able to estimate height and velocity of the micro air vehicles accurately, and achieves smooth landings with these estimates, even in windy outdoor environments.

@en

Monocular vision is increasingly used in Micro Air Vehicles for navigation. In particular, optical flow, inspired by flying insects, is used to perceive vehicles’ movement with respect to the surroundings or sense changes in the environment. However, optical flow does not directly provide us the distance to an object or velocity, but the ratio of them. Thus, using optical flow in control involves nonlinearity problems which add complexity to the controller. To deal with that, we propose an algorithm to use an extended Kalman filter to estimate the distance and velocity of the vehicles from optical flow while approaching a surface, and then use these estimates for control. We implement and test our algorithm in computer simulation and on-board a Parrot AR.Drone 2.0 to demonstrate its feasibility for MAVs landings. Both results show that the algorithm is able to estimate height and velocity of the MAV accurately. @en