Composer, musicologist, and improviser George Lewis discusses Rainbow Family (1984), a groundbreaking work that employs proto-machine-listening software to analyze an improviser’s performance in real time, while simultaneously generating both complex responses to the musician’s playing and independent behavior arising from the program’s internal processes. The essay explores issues of subjectivity, agency, intelligibility, and social responsibility that arise in encounters between machine listeners and their biological counterparts.
I’ve been making interactive computer music since around 1980, and like a number of other composer-technologists, I have drawn upon AI and practices of free improvisation in creating a kind of music making that includes machine subjectivities as central actors. These “creative machines” have been designed to stake out musical territory, assess and respond to conditions, and assert identities and positions—all aspects of improvisative interaction, both within and beyond the domain of music.
In an article from 2000, I wrote that this kind of musical work “deals with the nature of music and, in particular, the processes by which improvising musicians produce it. These questions can encompass not only technological or music-theoretical interests but philosophical, political, cultural and social concerns as well. In fact, I can say that much of my work in critical improvisation studies emerges from my practice with interactive computing. Thus, I’d like to offer this essay—published here in tandem with video and audio examples from Rainbow Family (1984), the first of my interactive virtual orchestra —as a small but possibly potent provocation concerning the act of listening.
The ontologies of “creative machines,” and the hybrid ontologies of encounters between them and human musical improvisors (or even with other machines like themselves), depend upon a fundamental construction of something that we are pleased to call “listening,” in which the exchange of sonic features leads to a form of sociality. At some level, framing machines as listeners becomes phenomenological and social as well as technological; in fact, all three aspects of the listening experience need to be considered together as we construct technologies of both human and machine listening, the modalities of which are similar to each other, yet different as well.
I begin my gentle provocation by presenting my understanding that the experience of listening is an improvisative act, engaged in by everyone, that amounts to an expression of agency, judgment, and choice, conducted in a condition of indeterminacy. Immersing ourselves conceptually in this improvisative assemblage allows us to recognize our vulnerability as listeners, even as we practice active engagement with the world. If the subaltern cannot speak, he or she is obliged to listen, and acts of listening and responding inevitably place us in a condition of momentary subalterity, whatever our designated social, racial, gender, or class position. Indeterminacy, often posed musically since John Cage as separable from improvisation, becomes instead an aspect of everyday life that is addressed improvisatively.
Retheorizations of both subjectivity and agency are on offer in the encounter between machine listeners and their biological counterparts. In an article from 2000, I characterize performances of Voyager, a later generation of my creative machine orchestras, first performed in 1987, as “parallel streams of music generation, emanating from both the computers and the humans—a nonhierarchical, improvisational, subject-subject model of discourse, which is congruent with the 2011 argument of anthropologist of technology Lucy Suchman: “In attempting to understand the constitution of the human/machine interface I have argued that subject/subject intra-actions involve forms of mutual intelligibility intimately connected with, but also importantly different from, the intelligibilities involved in relations of subject and objects.
The ways in which creative machines listen also have implications regarding intelligibility. For the philosopher Arnold I. Davidson, thinking about improvisation with creative machines inevitably connects intelligibility with social responsibility: “Collective intelligibility unfolds in real time when the participants in social interaction are committed to making sense of, and giving sense to, themselves and others. Theorizing relations among people and interactive systems as microcosms of the social leads to a notion of a hybrid, cyborg sociality that casts listening as both a social act and social fact, even when an individual in an ostensibly solitary state is pleased to believe in his or her individuation.
In a 2010 panel discussion at the University of Chicago between me, Davidson, and the pianist Alexander von Schlippenbach, Davidson theorized the sonic encounter between Schlippenbach and the interactive computer pianist version of Voyager, from 2004 as an example of social responsibility as expressed through listening:
When you try to respond to someone, being responsible to them, to that particular person, one of the ways you tell is how they then respond to you. So when you hear the way they’re playing, I think you see a very clear aspect of interactive responsibility, the social dimension of responsibility. … It seemed to me that in the piece here, [we have] all of those aspects of social responsibility—trying to respond to what it [the computer] was doing, letting it have its say, sometimes trying to influence what it would say—which is part of responsibility as well
“So all of those aspects of social responsibility,” Davidson concluded, “seem to me expressed in this context of improvisation. It’s actually a deeper sense of social responsibility than if one has some very abstract principles that one knows in advance that are just fixed.
My concluding provocation lies in my claim that this assemblage of conditions—indeterminacy, agency, judgment, choice (each modality affecting the others)—is fundamental to the ontology of improvisation, and will be found in any improvised act whatsoever, by people, machines, or other entities. I further propose that this assemblage of conditions acts as a marker by which we can recognize the presence of improvisation itself—including the improvisation of listening, attending, and related acts. Even as I propose this assemblage as a means of freeing both improvisation and listening from musical and artistic models, I also venture that our improvising computers, and improvised music itself, can show us what listening can be, and teach us to celebrate our place as listeners as part of a continuous, interactive transformation of both Other and Self.