This article postulates that a better grasp of the interaction between the individual and the work of art is the necessary foundation for a genuine understanding of how the arts can affect people. Through a critique of philosophical and empirical attempts to capture the main features of the aesthetic encounter, the article draws attention to the gaps in our current understanding of the responses to art. It proposes a classification and exploration of the factors—social, cultural and psychological—that contribute to shaping the aesthetic experience, thus determining the possibility of impact. The ‘determinants of impact’ identified are distinguished into three groups: those that are inherent to the individual who interacts with the artwork; those that are inherent to the artwork; and ‘environmental factors’, which are extrinsic to both the individual and the artwork. The article concludes that any meaningful attempt to assess the impact of the arts would need to take these ‘determinants of impact’ into account, in order to capture the multidimensional and subjective nature of the aesthetic experience.