Painting Music: Using artificial intelligence to create music from live painted drawings

Andrew Starkey, Kate Steenhauer, Jack Caven

Research output: Contribution to journalArticle

Abstract

This article describes the development of artificial intelligence (AI) techniques that monitor a painting or drawing evolving in real time and produce musical notes that relate to the individual elements of art as the artwork develops on the canvas. The article describes the practical approach required to capture the artwork unfolding in real time and then describes the framework used to develop the correlations between visual art and music. The AI technique exploits these areas of similarity within the two distinct artforms in order to respond to the live-painted elements and produce musical notes that reflect the development of
the evolving artwork. A prototype of this system was implemented in a live stage performance at Aberdeen May Festival 2019 whose narrative centred on the question Is AI good or bad? Other outputs of this project are a 20-minute film and a body of (tangible) visual artwork for digital platforms and gallery environments informed and inspired by AI. The integration of these disciplines through AI transforms a static artform into one that is dynamic, interactive, transformational, transient and temporal
Original languageEnglish
Pages (from-to)209-224
JournalDrawing: Research, Theory, Practice
Volume5
Issue number2
DOIs
Publication statusAccepted/In press - 20 Nov 2020

Keywords

  • artificial intelligence
  • live drawing
  • music improvisation
  • drawing performance
  • collaboration
  • figurative drawing

Fingerprint Dive into the research topics of 'Painting Music: Using artificial intelligence to create music from live painted drawings'. Together they form a unique fingerprint.

Cite this