Autonomous vehicles have been used for a variety of driving tasks, in which path planning and motion control are important research parts to realize the autonomous driving. A hierarchical framework consisting of path planning and motion control of the vehicle for non-specific scenarios is proposed in this paper. Firstly, the description and the formulations of the problem are given, and the corresponding models are constructed. Then, the logical construction of proposed framework is expounded with several logical associations and algorithmic improvements. The bidirectional heuristic planning with adaptive scale search is designed and incorporated with robust weighted regression algorithm to plan the optimal global path, while the multi-step predictive control method based on heuristic reinforcement learning algorithm is proposed to improve the effect of the motion control. The results show that the proposed framework for autonomous driving achieves better performance in both path planning and motion control than several existing algorithms and methods. The adaptability of hierarchical framework is demonstrated. Furthermore, the effectiveness of the hierarchical framework in real world scenario application is also validated.

@en

With the development of unmanned vehicle technology, unmanned vehicles have played a huge role in logistics transportation, emergency rescue and disaster relief, etc., so the research on unmanned vehicles is becoming more and more important. Road detection is an important part of environmental perception and an important factor in the realization of assisted driving and unmanned driving technology. High-precision road detection technology can provide important environmental information for efficient planning and reasonable decision-making of unmanned vehicles. Firstly, the technical framework of road detection is given, and the road detection process is introduced in detail. Then, the vision-based road detection algorithm is introduced. Finally, some related data sets in the field of road detection are collected, which provides new ideas and methods for road detection researchers.

@en

In the autonomous driving process, the decision-making system is mainly used to provide macro-control instructions based on the information captured by the sensing system. Learning-based algorithms have apparent advantages in information processing and understanding for an increasingly complex driving environment. To incorporate the interactive information between agents in the environment into the decision-making process, this paper proposes a generalized single-vehicle-based graph neural network reinforcement learning algorithm (SGRL algorithm). The SGRL algorithm introduces graph convolution into the traditional deep neural network (DQN) algorithm, adopts the training method for a single agent, designs a more explicit incentive reward function, and significantly improves the dimension of the action space. The SGRL algorithm is compared with the traditional DQN algorithm (NGRL) and the multi-agent training algorithm (MGRL) in the highway ramp scenario. Results show that the SGRL algorithm has outstanding advantages in network convergence, decision-making effect, and training efficiency.

@en

As one of the main elements of reinforcement learning, the design of the reward function is often not given enough attention when reinforcement learning is used in concrete applications, which leads to unsatisfactory performances. In this study, a reward function matrix is proposed for training various decision-making modes with emphasis on decision-making styles and further emphasis on incentives and punishments. Additionally, we model a traffic scene via graph model to better represent the interaction between vehicles, and adopt the graph convolutional network (GCN) to extract the features of the graph structure to help the connected autonomous vehicles perform decision-making directly. Furthermore, we combine GCN with deep Q-learning and multi-step double deep Q-learning to train four decision-making modes, which are named the graph convolutional deep Q-network (GQN) and the multi-step double graph convolutional deep Q-network (MDGQN). In the simulation, the superiority of the reward function matrix is proved by comparing it with the baseline, and evaluation metrics are proposed to verify the performance differences among decision-making modes. Results show that the trained decision-making modes can satisfy various driving requirements, including task completion rate, safety requirements, comfort level, and completion efficiency, by adjusting the weight values in the reward function matrix. Finally, the decision-making modes trained by MDGQN had better performance in an uncertain highway exit scene than those trained by GQN.

@en

Precisely modeling interactions and accurately predicting trajectories of surrounding vehicles are essential to the decision-making and path-planning of intelligent vehicles. This paper proposes a novel framework based on ensemble learning to improve the performance of trajectory predictions in interactive scenarios. The framework is termed Interactive Ensemble Trajectory Predictor (IETP). IETP assembles interaction-aware trajectory predictors as base learners to build an ensemble learner. Firstly, each base learner in IETP observes historical trajectories of vehicles in the scene. Then each base learner handles interactions between vehicles to predict trajectories. Finally, an ensemble learner is built to predict trajectories by applying two ensemble strategies on the predictions from all base learners. Predictions generated by the ensemble learner are final outputs of IETP. In this study, three experiments using different data are conducted based on the NGSIM dataset. Experimental results show that IETP improves the predicting accuracy and decreases the variance of errors compared to base learners. In addition, IETP exceeds baseline models with 50% of the training data, indicating that IETP is data-efficient. Moreover, the implementation of IETP is publicly available at https://github.com/BIT-Jack/IETP.

@en

The deep reinforcement learning-based energy management strategies (EMS) have become a promising solution for hybrid electric vehicles (HEVs). When driving cycles are changed, the neural network will be retrained, which is a time-consuming and laborious task. A more efficient way of choosing EMS is to combine deep reinforcement learning (DRL) with transfer learning, which can transfer knowledge of one domain to the other new domain, making the network of the new domain reach convergence values quickly. Different exploration methods of DRL, including adding action space noise and parameter space noise, are compared against each other in the transfer learning process in this work. Results indicate that the network added parameter space noise is more stable and faster convergent than the others. In conclusion, the best exploration method for transferable EMS is to add noise in the parameter space, while the combination of action space noise and parameter space noise generally performs poorly. Our code is available at https://github.com/BIT-XJY/RL-based-Transferable-EMS.git.@en

A reliable multi-agent decision-making system is highly demanded for safe and efficient operations of connected and autonomous vehicles (CAVs). In order to represent the mutual effects between vehicles and model the dynamic traffic environments, this research proposes an integrated and open-source framework to realize different Graph Reinforcement Learning (GRL) methods for better decision-making in interactive driving scenarios. Firstly, an interactive driving scenario on the highway with two ramps is constructed. The vehicles in this scenario are modeled by graph representation, and features are extracted via Graph Neural Network (GNN). Secondly, several GRL approaches are implemented and compared in detail. Finally, The simulation in the SUMO platform is carried out to evaluate the performance of different G RL approaches. Results are analyzed from multiple perspectives to compare the performance of different G RL methods in intelligent transportation scenarios. Experiments show that the implementation of GNN can well model the interactions between vehicles, and the proposed framework can improve the overall performance of multi-agent decision-making. The source code of our work can be found at https://github.com/Jacklinkk/TorchGRL.@en

This paper proposes a linear quadratic controller based on particle swarm algorithm for the rear wheel control of four-wheel steering vehicle. Particle swarm optimization with fitness functions is used to optimize the coefficients of the weight matrix offline. The fuzzy rules following the controller is used if the road condition is terrible. The simulation results show that the LQR control model based on particle swarm optimization makes the trajectory tracking of the vehicle better and the side slip angle of the vehicle lower. It can be proved that the controller has positive effect on handling stability of the vehicle and safety of drivers.

@en

Autonomous vehicles have a great potential in the application of both civil and military fields, and have become the focus of research with the rapid development of science and economy. This article proposes a brief review on learning-based decision-making technology for autonomous vehicles since it is significant for safer and efficient performance of autonomous vehicles. Firstly, the basic outline of decision-making technology is provided. Secondly, related works about learning-based decision-making methods for autonomous vehicles are mainly reviewed with the comparison to classical decision-making methods. In addition, applications of decision-making methods in existing autonomous vehicles are summarized. Finally, promising research topics in the future study of decision-making technology for autonomous vehicles are prospected.

@en