Continual learning (CL) enables intelligent systems to continually acquire, adapt, and apply knowledge, representing a dynamic paradigm in AI. For embodied agents—interacting with their environment physically and cognitively—CL enhances adaptability and reduces training costs significantly. In this literature review, we contribute by focusing on the application of CL in such agents, showcasing the approaches, means of evaluation and practical uses of this cognitive framework in real-world scenarios. We conclude that while CL holds promise for embodied agents, there exists a notable gap between the theoretical evaluation of CL and the complex real-world scenarios these agents operate in.