The accurate simulation of Room Impulse Responses (RIRs) is important in a variety of applications in acoustics such as automatic speech recognition, speech enhancement, and architectural acoustic design. While objective metrics for evaluating RIRs have been researched extensively, the subjective perceptual accuracy of the simulations is largely overlooked. This paper seeks to address this gap, designing a subjective testing methodology for evaluating the perceptual accuracy of simulated RIRs. A framework is proposed that combines the ABX testing methodology with a modified Multiple Stimuli with Hidden Reference and Anchor (MUSHRA) approach, measuring attributes such as clarity, warmth, environment, and reverbera- tion. The study involved 50 participants evaluating audio samples convolved with both real and simulated RIRs. Results seem to indicate that participants could reliably distinguish between real and simulated RIRs, with perceptual differences observed in the “clarity” and “reverberation” attributes. The findings suggest that current simulation methods for RIRs do not fully capture the perceptual aspects of acoustic environments.