Reinforcement Learning (RL) shows great potential for robotic manipulation tasks, yet it suffers from low sample efficiency and needs extensive exploration of state-action spaces. Some recent methods leverage the commonsense knowledge and reasoning abilities of Large Language Models (LLMs) to guide RL exploration toward more meaningful states. However, LLMs may generate semantically correct but physically infeasible plans, leading to unreliable solutions. In this paper, we propose \textit{Language-Guided exploration for Reinforcement Learning} (LGRL), a novel framework that utilizes LLMs' planning capability to directly guide RL exploration. This approach utilizes LLM planning at both the task and affordance levels, enhancing learning efficiency by directing RL agents toward semantically meaningful actions. Unlike previous methods that rely on the optimality of LLM-generated plans or rewards, LGRL corrects sub-optimality and explores multimodal affordance-level plans without human intervention.
We evaluated LGRL on pick-and-place tasks within standard RL benchmark environments, demonstrating significant improvements in both sample efficiency and success rates.

Tactile sensing provides crucial information about the stability of a grasped object by a robotic gripper. Tactile feedback can be used to predict slip, allowing for timely response to perturbations and to avoid dropping objects. Tactile sensors, included in robotic grippers, measure vibrations, strain or shearing forces which are produced by the movement of the grasped object. With sufficient spatial resolution, tactile sensors can even classify slip or estimate the 3d force displacement field. However, current tactile sensors fail to preemptively detect slippage, requiring fast reaction times during applications in real-time control. Here we show a perception framework that can predict slippage before it occurs by estimating the frictional safety margin. The safety margin indicates the margin to the frictional strength of a grasp, which decreases for reduced friction or increased load force. An accurate safety margin estimate allows for more efficient robot grip force control while providing robustness against object uncertainty and frictional conditions. We developed a high resolution tactile sensor, on which we trained a convolutional neural network to learn the relationship between tactile images and the safety margin. The network’s performance is evaluated on unseen test data, showing robustness to variations in environmental conditions. The results demonstrate that the tactile images contain the information needed to produce accurate safety margin estimates. These estimates can be used for control up to 20% of the minimum required grip force, mimicking human grasping behavior. This approach can drive new grasp control methods and enable robotic grasping of fragile objects in highly dynamic environments. Applications can be found in harvesting, parcel sorting, or improving human-robot interaction.