I recently started listening to the Art of the Score podcast, which does an amazing job of unpacking movie soundtracks in depth, beginning with Raiders of the Lost Ark in the first episode. Like most such explorations of music, however, it ultimately leaves me wanting something that I’ve long thought there should be much more of—namely, more tightly interwoven cultural commentary, especially for music. As I will describe below, I’m broadly interested in different ways of combining context, commentary, and criticism with the thing being commented on or critiqued.
As you might expect, episodes of Art of the Score include parts of the soundtrack from the movie under discussion (as well as music from other films). This, of course, is an obvious advantage of listening to media about music, rather than just reading a book: you can integrate the music directly into the audio. In the case of this podcast, they also have someone with a piano in front of them while they are recording, so that that person can perform the melodies, for example, or demonstrate differences between slight variations. (As always, I am unbelievably impressed by those people that have the ability to just play such things so casually by ear).
As good as this show is, I find it still tends to be too much talking and not enough music, relative to my ideal. These are in-depth, hour-long episodes (or longer), and the hosts spend a lot of time talking about a particular film or composer or piece of music before finally playing part of that music. The live demonstrations on the piano definitely help, but overall the effect tends to be comparable to including a few occasional illustrations in a book: the overall experience is that you are primarily reading the book (or equivalently, listening to a conversation), and this will only occasionally be punctuated by an illustration (or equivalently, a bit of music) to ground your understanding in the actual thing being discussed.
What I’d really like instead, is to not only incorporate music directly with the commentary, but to flip the ratio to be something more like 70:30, music to commentary, in terms of the total amount of time devoted to each. I would absolutely love a history of music (for pretty much any genre), that took a particular angle (e.g., tracing the evolution of a particular style), and provided history and context to help appreciate the music to be played, interwoven with actual examples of that music. Ideally it would alternate between a few minutes of history, description, or commentary, followed by a couple of songs to demonstrate those points (played in full), and then iterate. Obviously this would massively inflate the runtime of the corresponding text, but I think it would dramatically enhance the experience. It’s also not hard to imagine some sort of interface which would make navigation easy (e.g., skipping tracks or finding particular songs). I have not yet encountered anything quite like this in the wild, but if any one knows of good examples of such a format, please let me know in the comments!
From my experience, it seems like most shows of this type tend to follow the Art of the Score model, with only a fairly small amount of music surrounded by a relatively large amount of commentary. For example, Song Exploder is another well known example, which takes the very compelling approach of breaking a piece of music into its constituent parts, and exploring the song’s genesis with the artists. But again, the emphasis is mostly on discussion, with typically only one song analyzed per episode.
By far the most disappointing examples of this sort of thing that I have encountered are audiobooks about the history of music. I’ve tried a few of these, and pretty much all of them have been just the text of the printed books, with no actual music incorporated at all! I’m assuming this could in part be due to licensing issues, but it seems like a huge missed opportunity to provide a much more meaningful listening experience, especially since the “reader” has already chosen to consume the text in an audio format. If done properly, this could be orders of magnitude better than a printed text.
Of course, these books will still likely refer to particular works of music, which you could in principle track down and listen to, but this relatively small difference in affordances creates a very different experience. This is especially true for classical music, where there are any number of different performances of many pieces of music available, not all of which would necessarily reflect what is being described. This is almost like reading a book about the history of painting that contains no images, where you have to go and find each work online each time you want to see what the author is talking about.
Indeed, the most basic case of this kind of interwoven cultural commentary, as alluded to above, is including images in books (which I think there should be far more of!), but this too can be done well or poorly. The bad version is to put all the images on a few adjacent pages in the middle of the book, rather than being integrated directly with the text. It’s not terrible, but that little bit of flipping back and forth makes it harder to appreciate the images in the right context. An interesting variation we could imagine would be something like an ebook or online format, where part of the screen would always have one or more images displayed (which could be expanded for more detail), using whatever was most relevant to the textual content. Indeed, this is basically the format (created in a much more deliberate fashion) used by many scrolling explainers that you see in places like Close Read in the New York Times, where a scrolling pane of text is augmented with a parallel pane used for visualizations or animations. Packages like Idyll make this quite easy to do on a technical level, but taking full advantage of the form is still quite a challenge. (Again, I’d love to be pointed to more compelling examples of this sort of thing).
Expanding beyond texts and music, another classic example of this form is the audio guides that museums sometimes provide for visitors. This is in some sense an ideal pairing of media formats, as most works of art in museums tend to be purely visual (or at least, don’t provide much in the way of audio affordances, except perhaps in contemporary art). As such, the audio channel is still available for people to make use of as they wander about. Some might think of audio guides as impoverished versions of live tours, but in reality they are quite different experiences. Whereas a guided tour is an interactive experience in which someone is guiding you through things in a specific order (and can respond to questions), a good audio guide facilitates more self-directed exploration, where you can pick and choose what you want to learn more about.
The amount of content in most audio guides tends to be quite limited (relative to the number of works in a museum), so they often just help you navigate the highlights. I suspect people are already working on this, but one could imagine trying to create a generative AI based system that would be able to provide more detail about any work of art in a museum, as well as general history for the corresponding time period, etc.1 You could even imagine it being tied intelligently to your location in the museum, providing an initial bit of context about whatever you are looking at, but capable of providing more information when desired. The input format would be tricky, as allowing for voice input from everyone might disrupt the quiet, meditative, experience provided by many museums. Nevertheless, a handful of options such as “tell me more” or “provide additional context”, might be sufficient.
A final case of tightly woven cultural commentary that often works extremely well, but sadly seems to be somewhat of a dying form, is the director’s commentary on film, which used to be quite a common bonus feature on DVDs. You wouldn’t think these would work as well as they do, given that film depends on both audio and visual elements. However, if you’ve already watched the film (which is virtually a necessity for most of these), having someone knowledgeable speaking over the film, (with the original audio playing at a low volume), is an incredible way to provide information and insight about what is on screen. The least interesting of these tend to be just separate interviews stitched together, or things like that. Much better, typically, is where the commentators are actually watching the film as they go, commenting directly on what is on screen. There is of course a huge range of quality, and some directors (or other commentators) don’t actually have much to say, but some are absolutely fascinating.
As with the other examples above, there would seem to be an easy opportunity (in terms of the technology), to provide a much broader range of these sorts of things in a streaming context. Even for films where the commentary already exists (available on a DVD, for example), most streaming services don’t normally provide this as an option for the viewer, though it seems like it would be trivial (technically) to do so.2 Beyond what has already been recorded, however, one could easily imagine a whole marketplace of amateur commentaries, recorded by fans or researchers or other filmmakers who have something to say. Once again, I expect the legal issues are the biggest obstacle to this, but it seems like both an obvious opportunity and a relatively straightforward application of existing systems.
I’m sure there are both many other great examples of this sort of interweaving of commentary with art, which I hope people will point me to, but it’s also fun to imagine new combinations and forms that haven’t been invented yet. Although this is probably most relevant to art and culture, some of the same principles apply to other domains, such as scientific papers and science more broadly. Ultimately, what I want is a way for knowledgeable people to share their ideas on things in a way that takes advantage of technological possibilities to directly interweave commentary and content, and encourages more of this sort of thing in the future.
Indeed, as reported in the New York Times, it turns out the Musée d’Orsay has already used a similar idea to try to create an interactive experience for Vincent Van Gogh, based on his letters and biographies about him. ↩︎
Apparently some new shows have started bringing back commentary tracks, such as Jury Duty on Amazon Prime. Some directors’ commentaries for older films are also available through a limited number of streaming services, such as the Criterion Channel. ↩︎