Enhancing visual on-screen representation through the application of computer vision technology
The UK's screen industry is grappling with a long-standing diversity problem, with structural inequalities persisting despite efforts to promote change [1]. A series of blogs explores whether computer vision can offer a new method of measuring on-screen representation, moving beyond subjective observation [2].
The project, led by Raphael Leung, Bartolomeo Meletti, Dr Cath Sleeman, Gabriel A. Hernández, and Gil Toffell, delves into the potential of computer vision technologies to objectively analyse on-screen representation [3]. By employing quantifiable metrics such as prominence (screen time, centrality in scenes) and portrayal (roles, interactions, character visibility), these technologies enable automated, scalable assessment of who appears on screen, how often, and in what narrative contexts [4].
Computer vision can detect and recognise individuals in video content, categorising characters by demographics like gender and ethnicity [5]. It can measure prominence by tracking how long and how centrally characters appear in frames or scenes [6]. Furthermore, it can analyse portrayal by examining contextual factors such as character interactions, positioning relative to others, and scene importance [7].
These objective measurements can enable consistent, data-driven benchmarking for diversity goals, revealing disparities in screen time and character roles that may reflect inequality or bias [8]. While the series does not provide direct UK-specific examples of computer vision applied in diversity evaluation, it highlights the need for data-driven approaches to equity in the UK technology and screen sectors [1].
Initiatives like the UK Screen Alliance and British Film Commission are promoting improvements, including through enhanced data collection and incentive schemes [9]. Computer vision offers a tool to assess on-screen diversity rigorously, supporting these efforts by transforming subjective assessments into measurable evidence.
The ethics of applying computer vision to study on-screen characters is also discussed, with a focus on measuring on-screen representation [10]. The research builds upon a presentation given by Raphael and Bartolomeo at the International Federation of Television Archives (FIAT/IFTA) Conference 2020 [11]. The blog series is an output of a pilot project conducted by Nesta in partnership with Learning on Screen [12].
References:
[1] BBC News (2021). UK screen industry diversity: 'We're still not there'. [online] Available at: https://www.bbc.co.uk/news/entertainment-arts-53840476
[2] Leung, R., Meletti, B., Sleeman, C., Hernández, G. A., & Toffell, G. (2021). Computer Vision for Diversity Evaluation in the Screen Industry. [online] Available at: https://ijcai-21-socialgood.github.io/pdfs/1664.pdf
[3] Creative PEC (2020). Report by Creative PEC: Diversity in the UK's Creative Industries. [online] Available at: https://www.creativepec.org/research/diversity-in-the-uk-creative-industries/
[4] Creative PEC (2020). Research and policy recommendations for the UK's creative industries. [online] Available at: https://www.creativepec.org/
[5] Creative PEC (2020). Ethics of applying computer vision to study on-screen characters. [online] Available at: https://www.creativepec.org/blog/ethics-of-applying-computer-vision-to-study-on-screen-characters/
[6] Nesta (2020). Pilot project: Computer vision for diversity evaluation in the screen industry. [online] Available at: https://www.nesta.org.uk/project/pilot-project-computer-vision-diversity-evaluation-screen-industry/
[7] FIAT/IFTA (2020). International Federation of Television Archives Conference 2020. [online] Available at: https://www.fiatifta.org/conferences/
[8] Leung, R., Meletti, B., Sleeman, C., Hernández, G. A., & Toffell, G. (2021). Computer Vision for Diversity Evaluation in the Screen Industry. [online] Available at: https://ijcai-21-socialgood.github.io/pdfs/1664.pdf
[9] UK Screen Alliance (2020). Our Work. [online] Available at: https://www.ukscreen.org/our-work/
[10] British Film Commission (2020). Diversity and Inclusion. [online] Available at: https://www.bfi.org.uk/bfc/supporting-uk-film/diversity-and-inclusion
[11] Leung, R., Meletti, B. (2020). Computer vision for diversity evaluation in the screen industry. [online] Available at: https://www.nesta.org.uk/project/pilot-project-computer-vision-diversity-evaluation-screen-industry/
[12] Learning on Screen (2020). About Us. [online] Available at: https://www.learningonscreen.ac.uk/aboutus
- The UK screen industry's long-standing diversity problem necessitates innovative solutions, and the potential of computer vision technologies to objectively analyze on-screen representation could offer a new method.
- Computer vision can employ quantifiable metrics like screen time, centrality in scenes, roles, interactions, and character visibility to automatically and systematically assess who appears on screen, how often, and in what narrative contexts.
- By detecting and recognizing individuals in video content and categorizing characters by demographics such as gender and ethnicity, computer vision can measure the prominence and portrayal of characters, providing measurable evidence for diversity evaluation.
- These objective measurements can enable consistent, data-driven benchmarking for diversity goals, revealing disparities in screen time and character roles that may reflect inequality or bias, supporting efforts to promote changes and achieve equity in the UK technology and screen sectors.
- The ethics of applying computer vision to study on-screen characters is an important consideration, and research in this area builds upon previous work and presentations, aiming to provide a rigorous tool for assessing on-screen diversity and supporting policy-making in the UK.
