In recent years, Unmanned Aerial Vehicles (UAVs) are widely utilized in precision agriculture, such as tree plantations. Due to limited intelligence, these UAVs can only operate at high altitudes, leading to the use of expensive and heavy sensors for obtaining important health information of the plants. To fly at low altitudes, these UAVs must possess the capability of obstacle avoidance to prevent crashes. However, most current obstacle avoidance systems with active sensors are not applicable to small aerial vehicles due to the cost, weight, and power consumption constraints. To this end, this paper presents a novel approach to the autonomous navigation of a small UAV in tree plantations only using a single camera. As the monocular vision does not provide depth information, a machine learning model, Faster Region-based Convolutional Neural Network (Faster R-CNN), was trained for the tree trunk detection. A control strategy was implemented to avoid the collision with trees. The detection model uses image heights of detected trees to indicate their distances from the UAV and image widths between trees to find the widest obstacle-free space. The control strategy allows the UAV to navigate until any approaching obstacle is detected and to turn to the safest area before continuing its flight. This paper demonstrates the feasibility and performance of the proposed algorithms by carrying out 11 flight tests in real tree plantation environments at two different locations, one of which is a new place. All the successful results indicate that the proposed method is accurate and robust for autonomous navigation in tree plantations.