An interactive multimedia companion to Wagner's Lohengrin:  encoding and visualising a motivic study

1. Abstract

We introduce a tablet based interactive application which presents a comprehensive digital exploration as a companion to a musicological article. Taking the opera Lohengrin, the article and digital companion show how one motive is altered each time it recurs, reflecting its role in the drama, and exploring Wagner’s sophisticated treatment of recurring themes. Musicological analysis is encoded using Linked Data as an independent, repurposable, and open Research Object. Interactive user views are generated directly and dynamically in the browser from this knowledge graph using novel visualisations, which in turn enable the user to navigate all possible paths through the evidential multimodal materials. Our companion explores the different compositional devices Wagner uses to vary his motives, browsing the whole opera for motive occurrences and their musical and textual contexts. Visualisations and recordings support the analysis, making it accessible to an audience that may struggle with a Wagnerian orchestral score.

Musicological argument has traditionally been communicated in writings that are textual, linear and illustrated by occasional figures, despite almost always being concerned with diverse subjects and evidential materials, each potentially exemplified by different media, and each potentially a springing point for digression and exploration of the author's argument. While exploring the referenced materials may be non-linear, this is neither embodied in nor enabled by traditional communication mediums.  

Previously we have digitally enhanced an extract of a musicological article[1] with dynamic and interactive elements. Here, we introduce a tablet based interactive application which presents a comprehensive digital exploration as a companion to a complete musicological article. Taking the opera Lohengrin, the article and digital companion show how one motive is altered each time it recurs, reflecting its role in the drama, and exploring Wagner’s sophisticated treatment of recurring themes.

Musicological analysis is encoded – along with relationships to multimedia materials – using Linked Data as an independent, repurposable, and open Research Object. Interactive user views are generated directly and dynamically in the browser from this knowledge graph using novel visualisations, which in turn enable the user to navigate all possible paths through the evidential multimodal materials.

Our companion explores the different compositional devices Wagner uses to vary his motives, browsing the whole opera for motive occurrences and their musical and textual contexts. Visualisations and recordings support the analysis, making it accessible to an audience that may struggle with a Wagnerian orchestral score. Exploration of this material can follow or be triggered by the article (Fig.1), but can also be reader-driven, with free browsing of the curated musical landscape. A video essay also provides a source of narrative paths through the companion, as a guide itself and as a source of starting points.

Figure 1: The musicological essay view in the digital companion.

The application is built with a new version of the MELD (Music Encoding and Linked Data) framework[2]. MELD traverses Linked Data graphs to select and filter relevant information, with reusable components for creating and retrieving annotations, and for displaying and interacting with musical, textual, graphical and audio-visual materials. MELD is written in Javascript and Python, with resources using standards including the MEI[3] music encoding, TEI, the Music Ontology and Web Annotations[4].

Figure 2: A view with labelled vocal score, text and translation.

We provide two views for music notational content. Vocal score reductions are rendered from MEI with structural analysis dynamically overlaid (Fig.2); annotations trigger audio playback from that point. A second notational visualisation of MEI simplifies the complexity of a Wagnerian orchestral score: each instrument playing at a particular time is shown as a coloured ribbon, with the instrument's section of the orchestra providing the colour (Fig.3). This highlights differences across orchestration iterations that may seem identical as vocal score.

Figure 3. The same iteration showing the orchestral summary view.

For an opera thousands of bars long, overviews are crucial. An ever-present timeline shows all occurrences of a motive, providing a visual summary and a base for navigation. In the Time Machine view (Fig.4), users can also flick through motive occurrences – visualised as libretto, vocal score or orchestration – summarising the sequence within the opera, supporting quick comparisons, and as an index to detail views.

Figure 4. The Time Machine view in the Digital Companion.

Links to software and sourcecode:

• The Lohengrin TimeMachine Digital Companion app and introductory video: https://um.web.ox.ac.uk/lohengrin

• Lohhengrin app source code: https://github.com/oerc-music/ForbiddenQuestion

• The MELD framework the app is build upon: https://github.com/oerc-music/meld

• MELD components in turn use open source libraries including:

  • Verovio: https://github.com/rism-ch/verovio

  • CETEIcean: https://github.com/TEIC/CETEIcean

[1] D. Lewis, D.M. Weigl, J. Bullivant, and Kevin R. Page. Publishing musicology using multimedia digital libraries: Creating interactive articles through a framework for linked data and mei. In Proc. DLfM, DLfM ’18, pages 21–25, New York, NY, USA, 2018. ACM.

[2] D.M. Weigl and K.R. Page. A Framework for Distributed Semantic Annotation of Musical Score: “Take It To The Bridge!". In Proc. ISMIR 2017, Suzhou, China, October 23-27, 2017, pages 221–228, 2017.

[3] T. Crawford and R. Lewis. Review: Music encoding initiative. Journal of the American Musicological Society, 69(1):273–285, Spring 2016.

[4] M.E. Barnes, N. Ledchumykanthan, K. Pham, and K. Stapelfeldt. Annotation-based enrichment of digital objects using open-source frameworks. Code4Lib Journal, (37), 2017.