The introduction of artificial intelligence (AI) technologies in healthcare is expected to set a paradigm shift to medical practice because these systems will have a significant role in applications such as diagnosis-support and image analysis. However, this implementation does not come without risks. There are important ethical concerns that should be addressed beforehand to ensure public trust and acceptability. Privacy, safety, transparency, reliability and potential biases are some of the issues to consider. Responsible Research and Innovation (RRI) frameworks have been designed by academics to tackle this sort of problems but there is no application of these frameworks in the field of AI in healthcare. This problem is even more salient in the private sector, due to the unawareness of the RRI concept in industry. Consequently, the research objective of this project was to offer recommendations on how to implement RRI practices to avoid potential risks and improve the social acceptability of AI. For this, we studied the case of Philips and carried out interviews with the company’s experts in AI and corporate social responsibility (CSR). This information was complemented with a comprehensive study of the literature on topics related to AI in healthcare and RRI. The results from these activities were used to create a roadmap to introduce RRI practices in the AI innovation activities within Philips. The results showed that large companies should build upon their existing CSR practices to develop RRI. This will increase the acceptability of RRI within the research and development (R&D) teams. Based on that, we came up with 14 recommendations for the case of Philips. These actions range from current practices, such as continuing with the rigorous process of patient data selection and curation, to novel solutions such as including better interactive features in the design of telehealth platforms (i.e. virtual reality, video calls or social networks). Further research can be carried out in different companies to come up with common principles that contribute to the creation of a more comprehensive RRI framework for AI in healthcare.