Antiviral signalling, which can be activated in host cells upon virus infection, restricts virus replication and communicates infection status to neighbouring cells. The antiviral response is heterogeneous, both quantitatively (efficiency of response activation) and qualitatively (transcribed antiviral gene set). To investigate the basis of this heterogeneity, we combined Virus Infection Real-time IMaging (VIRIM), a live-cell single-molecule imaging method, with real-time readouts of the dsRNA sensing pathway to analyse the response of human cells to encephalomyocarditis virus (EMCV) infection. We find that cell-to-cell heterogeneity in viral replication rates early in infection affect the efficiency of antiviral response activation, with lower replication rates leading to more antiviral response activation. Furthermore, we show that qualitatively distinct antiviral responses can be linked to the strength of the antiviral signalling pathway. Our analyses identify variation in early viral replication rates as an important parameter contributing to heterogeneity in antiviral response activation.