T EXPRESSIVE NOTATION PACKAGE - Semantic …€¦ · and an interactive Tutorial both provided as a part of the ENP program. ... It uses an implementation of ANSI Common Lisp by LispWorks

Mika Kuuskankare

THE EXPRESSIVE NOTATION PACKAGE

A final report in the Development Study Pro-

gramme submitted in partial fulfillment of the re-

quirements for the degree of Doctor of Music

DocMus –

Department of Doctoral Studies in Musical Performance and Research

Sibelius Academy

Helsinki, Finland

2006

2

The Development Study Programme

Portfolio

This report is part of the portfolio required for the Development Study Programme for

the degree of Doctor of Music. It consists of an introductory essay and seven peer-

reviewed articles.

The portfolio comprises this report, a cross-platform music notation program—the

Expressive Notation Package, or ENP—conceived by the author, a Reference Manual,

and an interactive Tutorial both provided as a part of the ENP program.

ENP is primarily intended for composers and music analysts. It is meant to be

used as a notational front-end in a visual programming language called PWGL. PWGL,

in turn, is a cross-platform general-purpose visual programming language that the

author has co-developed in collaboration with Mikael Laurson (the lead developer)

and Vesa Norilo. ENP is distributed as a part of PWGL and can be downloaded at the

following address: http://www.siba.fi/PWGL/downloads.html.

3

4

Preface

I would like to express my gratitude to the following people.

The members of my project advisory group: Professor Marcus Castren, Dr. MikaelLaurson, and Professor Matti Karjalainen.

My supervisor, Professor Marcus Castren, for convincing me to become a doctoral stu-dent.

Dr. Mikael Laurson for essentially putting all this in motion about 15 years ago byintroducing me to Lisp. I would also like to thank him for his support and friendshipover the years.

The pre-examiners of my portfolio, Professor Roger B. Dannenberg and Dr. Jean-Baptiste Barriere, for their invaluable insights and comments.

Dr. Annikka Konttori-Gustafsson, the wise and lovely leader of the DocMus Depart-ment, for her gracious attitude towards my work.

My dear friend, the saxophone artist Olli-Pekka Tuomisalo, for his extraordinarily un-complicated friendship, and for the shared enthusiasm for high frequencies.

The composer Kimmo Kuitunen for sharing his insightful knowledge and understand-ing of music, and also for his persistence in learning about ENP.

The Boston Promenade Big Band for providing a good home for the out-of-tune-too-high-too-loud notes that I can generate with my trumpet. The wave-guide model is onthe way...

The Sibelius Academy Innovation Centre for the four-year postgraduate funding thatenabled me to carry out my research.

Joan Nordlund for checking the language.

Last, but definitely not least, I wish to thank my mother, who has supported me inmany different ways over the years.

5

6

Contents

List of Publications 9

I Introductory Essay 11

1 Introduction 131.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131.2 Overview of ENP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151.3 ENP Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171.4 The ENP Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221.5 The Scope of this Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271.6 The Contents of this Report . . . . . . . . . . . . . . . . . . . . . . . . . . 281.7 The Structure of this Report . . . . . . . . . . . . . . . . . . . . . . . . . . 291.8 A Note on Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

2 Supplementary Examples 312.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.1.1 Modern Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312.1.2 Orchestral Notation . . . . . . . . . . . . . . . . . . . . . . . . . . 342.1.3 Tape Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352.1.4 Music Analytical Notation . . . . . . . . . . . . . . . . . . . . . . . 36

2.2 ENP-Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372.2.1 Lyrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382.2.2 Dynamic Expressions . . . . . . . . . . . . . . . . . . . . . . . . . 392.2.3 Expressions Containing Break-point Functions . . . . . . . . . . . 412.2.4 Instrument-sensitive Expressions . . . . . . . . . . . . . . . . . . . 422.2.5 Auto-canceled Expressions . . . . . . . . . . . . . . . . . . . . . . 42

2.3 ENP-Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432.3.1 Input Shortcut . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442.3.2 Inputting Rhythm . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

7

8 Contents

2.3.3 Score Manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 482.3.4 Analysis and Visualization . . . . . . . . . . . . . . . . . . . . . . 51

3 Conclusions 553.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553.2 Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

3.2.1 Short-term Enhancements . . . . . . . . . . . . . . . . . . . . . . . 563.2.2 New Avenues of Research . . . . . . . . . . . . . . . . . . . . . . . 58

4 A Summary of the Publications 61

References 63

II Publications 71

Publication [P1] 77

Publication [P2] 91

Publication [P3] 97

Publication [P4] 102




List of Publications

The report consists of an introductory essay and the following seven articles referredto as [P1]–[P7]. [P2]–[P7] are verbatim copies of the published articles, [P1] is attachedas a revised manuscript.

[P1] M. Kuuskankare and M. Laurson. Expressive Notation Package. Computer MusicJournal, 30(4):67–79, Winter 2006. (in press; attached as a manuscript).

[P2] M. Kuuskankare and M. Laurson. Annotating Musical Scores in ENP. In Inter-national Symposium on Music Information Retrieval, London, UK, 2005.

[P3] M. Kuuskankare and M. Laurson. Intelligent Scripting in ENP using PWCon-straints. In Proceedings of International Computer Music Conference, pages 684–687,Miami, USA, 2004.

[P4] M. Laurson and M. Kuuskankare. From RTM-notation to ENP-score-notation.In Journees d’Informatique Musicale, Montbeliard, France, 2003.

[P5] M. Kuuskankare and M. Laurson. Recent Developments in ENP-score-notation.In Sound and Music Computing ’04, Paris, France, 2004.

[P6] M. Kuuskankare and M. Laurson. ENP Expression Designer - a Visual Tool forCreating User Definable Expressions. In International Computer Music Conference,pages 307–310, Barcelona, Spain, 2005.

[P7] M. Kuuskankare and M. Laurson. ENP Note Head Designer. In Journeesd’Informatique Musicale, Paris, France, 2005.

9

10

Part I

Introductory Essay

11

Chapter 1

Introduction

The focus of attention in this report is a computer-based music notation application,

the Expressive Notation Package or ENP, conceived by the author.

ENP is among the various music-related software packages developed at the

Sibelius Academy during the past 20 years. Since its inception it has been used ex-

tensively in several research projects.

1.1 Background

Classical music is primarily a written tradition. Throughout the centuries musical

compositions have been preserved in music notation, as opposed to being transmitted

orally. At the present time, common music notation is the lingua franca of both music

education and professional musicians. Regardless of its apparent success, it has also

been subject to criticism, and many proposals have been made to modernize it. Yet,

it is still the most complete and widespread method that we know of for notating the

complex and interrelated properties of a musical sound: pitch, intensity, time, timbre,

and pace.

Common music notation is also an invaluable tool in the field of computer mu-

sic. The history of computer-aided music notation begun as early as in the 1960s. To-

13

14 1. Introduction

wards the end of the decade a few computer-based music notation programs started

to emerge, and arguably one of the earliest systems was developed at the National

Research Council in Ottawa, Canada [52]. In the 1970s several text-based programs

were introduced, among others, Leland Smith’s early work MS [52] and Donald Byrd’s

SMUT [11]. Among the first interactive editors were Scriva [10] and Mockingbird [31].

The first commercial notation programs (including, among others, Score [54] program

by Leland Smith) were also introduced in the 1970s but it was not until the 1980s that

these programs were made available to the general public. [P1] gives a capsule list of

some of the current applications. Furthermore, a detailed history of computer-based

music notation can be found in [52].

According to Byrd:

[...] fully automatic ’high-quality music notation’ is not merely nontrivial but in

general impossible without human-level intelligence.[11]

The complexity of music notation makes it a difficult programming task. Creating a

comprehensive computer-based music notation application requires addressing sev-

eral challenging issues, including, but not limited to:

1. computer representation of music notation [11, 16, 15, 19, 57, 28, 27, 7, 22, 37];

2. visualization of musical information [32, 58];

3. automatic spacing algorithms [25, 8, 51];

4. data formats, structures and languages [11, 24, 13, 46, 29];

5. scripting languages (e.g., ManuScript of Sibelius [53]); and

6. user-interface [14, 12, 6, 48].

Advanced computer-based music notation has several interesting applications. It

can be used to produce exceptionally sophisticated results, for example, in computer

1.2 Overview of ENP 15

based performance of music. Although musical performance in this regard is becom-

ing increasingly life-like there is still room for improvement in terms of controllability

and naturalness. The interaction between computer-based music notation and sound

source control—which allows for the production of convincing renditions of musi-

cal compositions—remains among the most interesting applications. Computer-aided

composition and music analysis present challenges in terms of visualization, represen-

tation, and accessibility. Algorithmic control requires a versatile representation of the

complex structural and notational information behind the visual front-end of a musical

score. Computer-assisted instruction, in turn, requires a high level of interactivity and

built-in musical knowledge. This list of applications, although not exhaustive, already

introduces a number of challenges related to their potential implementation.

1.2 Overview of ENP

ENP [35] is a music notation program that is written in Lisp, CLOS [55], and

OpenGL [59]. It uses an implementation of ANSI Common Lisp by LispWorks (LW,

http://www.lispworks.com). LW is source code compatible across Windows, Linux, Mac

OS X, and some UNIX platforms.

ENP has been developed in order to meet the requirements of computer-aided com-

position, music analysis and synthesis control. It is intended to represent Western mu-

sical notation, currently excluding that written before the 17th century. There is strong

emphasis on the applications of new music and contemporary notation (for examples

see [9] and [33]). ENP is not a full-featured music typesetting program but it is de-

signed to produce automatic, reasonable musical typesetting according to common

practices (See [49] and [56]).

The graphics part of ENP is programmed using OpenGL—a cross-platform API for

developing portable, interactive 2D and 3D graphics applications. It features double-

buffering, anti-aliasing and hardware acceleration, for example, and is an inexpensive

16 1. Introduction

way of creating state-of-the-art graphical output and user interaction. One of its most

important features is its ability to interact with arbitrary complex graphical objects.

This mechanism—called ’picking’—is a powerful tool when programming graphical

user interfaces.

ENP has an object-oriented graphical user interface (GUI) that is based on direct

manipulation. According to Cooper [14], direct manipulation can be broken down into

three basic principles: (1) visual representation of the objects, (2) the use of dragging

instead of text entry or menu selection, and (3) feedback on the impact of the manipu-

lation. Object-oriented UI, in turn, is an interface paradigm where the user first selects

an object and then chooses an action from a context menu or via direct manipulation.

The underlying idea behind ENP GUI is to allow the user to manipulate the informa-

tion contained in a digital musical score as straightforwardly as possible. As a general

guideline, any musical object of any complexity can be edited directly in the score. All

editing operations also provide synchronized visual feedback for the user.

ENP is primarily intended to be used as a notational front-end in a cross-platform

general-purpose visual programming language called PWGL [41], which has a strong

emphasis on music- and sound-related applications. It is a combination of several soft-

ware packages built on top of Common Lisp and CLOS [55] its components including a

rule-based programming language called PWGLConstraints [38], and the PWGLSynth

[40] sound synthesis program.

ENP is an important bridge between the different software components inside

PWGL in that it allows the packages to inter-operate by using its representation of mu-

sical objects. It is closely integrated with PWGLSynth and enables PWGLConstraints

to be used to generate musical material. It can also be used in PWGL to view and edit

musical material in many different ways, and to algorithmically generate and process

musical objects.

1.3 ENP Applications 17

1.3 ENP Applications

According to Glendon Diener [21], music notation programs have three basic types of

use: compositional, archival, and analytic. In its current state ENP can be used as a

compositional and an analytical tool, and also as an ordinary music notation program

(archival use). Furthermore, a fourth category, algorithmic, is being added. The current

applications of ENP could thus be loosely categorized as follows:

1. Archival

(a) to represent complex music notation including special facilities for 20th cen-

tury music

(b) to manipulate a score through a mouse-driven GUI using direct editing

(c) to produce high quality paper output using PostScript

2. Compositional

(a) to generate musical material with the help of a rule-based language

(b) to add, remove, and modify information (pitch, rhythm, markings) using an

intelligent scripting language

3. Analytic

(a) to annotate a score, either manually or automatically, to display music theo-

retical analysis information, e.g., motives, harmonic progressions, Schenke-

rian graphs, or pitch-class set theoretical information

(b) to add visible or invisible notes to musical objects

(c) to dynamically inspect and visualize data contained by the score

4. Algorithmic

(a) to algorithmically generate and manipulate musical structures, scores, parts,

chords, and notes, for example

(b) to use a score as a user interface component

18 1. Introduction

(c) to visually create custom notational attributes that are immediately usable

without recompiling and

(d) to textually describe a score and the individual notational objects.

ENP has already been used as a tool in several different projects to produce control

information for various virtual instrument models [39]. One of its unique aspects is

that control information can be drawn directly on the score. Figure 1.1 shows visual

control information specialized for a virtual guitar instrument model. An elaborate

interpretation can be generated with the help of both standard and non-standard ex-

pressions. Here, the standard expressions include some idiomatic guitar markings,

such as string numbers and left-hand slurs. The non-standard expressions, in turn,

include a tempo function (marked as ’/time’), different kinds of sustains (’s’ and ’gs’),

and a macro-note expression (’vlarp’) [44]. The playback information generated by the

macro-note expressions can be seen in the lower staff written as stemless notes.

Figure 1.1: A score written for a virtual guitar containing both standard and non-standardexpressions. There are also some macro-note expressions that create complex arpeggio gestures(see the stemless notes in the lower staff). The score is a free adaptation of a guitar piece byHeitor Villa-Lobos (12 Etudes: No.1 Allegro Non Troppo).

The composer Kimmo Kuitunen used ENP to visualize the ideas behind large-scale

durations in his composition Hodoi to Ergo [34]. He used it in another piece, Trio for


Clarinet, Cello and Piano, to visualize its structural organization (see Figure 1.2). Here,

the textual annotations above the notes denote different kinds of texture classes (SOL

for solo, HOQ for Hoquetus, etc.). Furthermore, colors and colored shapes are used

to indicate similar texture classes.1 The durations of the notes, in turn, indicate the

lifespan of the given texture class in 16th notes (the number of required 16th notes is

also marked above the texture-class annotation).

Figure 1.2: One potential view of the structural organization of a piece, demonstrating thecombination of rhythmic information, textual annotations, colors, and shapes as a visualizationaid (Kimmo Kuitunen: a visualization of the structure of his Trio for Clarinet, Cello and Piano.Here, the structure of roughly the first half of the first movement is shown).

One interesting application of ENP is in computer-assisted music analysis and the

visualization of the analysis information. Currently, there are several sophisticated

music analysis applications available (e.g., HumDrum, [30]; athenaCL, [4]) that use

text-based representation as a starting point. This kind of approach is usually suitable

1The colors are visible only in the electronic version of this report.

20 1. Introduction

for analytical questions that are statistical in nature. However, it makes it more difficult

to verify the result because the analysis is given as a separate textual document without

the corresponding notational counterpart. ENP, in contrast, makes it possible to use a

set of specialized graphical devices to visualize the analysis information directly in the

score, and to automatize a large part of the analysis process.

Figure 1.3 gives the realization of the first five texture classes that are shown in

Figure 1.2. The correlation between the abstract structural outline and the final com-

position can now be seen more clearly. The texture classes are brought out by a line

enclosing the corresponding musical material. To emphasize the association with the

texture classes found in Figure 1.2, the original colors and names are used.

Publications [P1] and [P2] give numerous examples of music analytical applications

of ENP.


Figure 1.3: The structural plan presented in Figure 1.2 is displayed as part of the musicaltexture (Kimmo Kuitunen: Trio for Clarinet, Cello and Piano, 2004–2005).

Figure 1.4 gives an example of ENP as an algorithmically generated user interface

component. The two score views, shown inside a specialized PWGL application-box

[42], are generated automatically based on information retrieved from an instrumental

analysis database. Here, the most audible partials of the selected sound are displayed

22 1. Introduction

using a detailed arpeggio view (left) and a more concise chord view (right). This box

also contains other components that display amplitude envelopes and sound samples,

for example.

Publications [P6] and [P7] give further examples of how ENP can be used as an

interactive user interface component.

Figure 1.4: This example demonstrates the use of ENP as a GUI component.

1.4 The ENP Project

The end result of this project is a portfolio that includes ENP for Macintosh OS X and

Windows XP operating systems, this written report incorporating seven peer-reviewed

articles, and an ENP Reference Manual and interactive tutorial both distributed as a

part of ENP.

Although it would have been possible to provide a stand-alone version of ENP for

the purpose of this doctoral project, it is provided as a part of PWGL. There are several

compelling reasons for this. First, ENP will never exist outside of the PWGL environ-

ment, and it would have been unreasonable to try to separate the two. Secondly, the

1.4 The ENP Project 23

unique potential of ENP is more clearly realized when it is used in tandem with the

other software packages in PWGL. Furthermore, there is no ambiguity when it comes

to distinguishing the author’s work from that of the other developers of PWGL: every-

thing relating to symbolic music notation is essentially the work of the author. PWGL,

along with ENP, can be downloaded at http://www.siba.fi/PWGL/downloads.html.

The main part of this project involved software development. ENP itself is a com-

bination of, among others, the following software components:

1. scripting language

2. visual editors (note head designer, expression designer)

3. rule based Find

4. plug-in architecture

5. note property editors

6. textual score description language

7. exporter (EPS, PS, MIDI, ETF, LilyPond, MusicXML).

During this project, a considerable amount of effort was put into publishing activ-

ities. 28 papers (including two journal articles) have been published throughout this

doctoral project. Most of these were written in collaboration with Mikael Laurson,

although some articles were written jointly with researchers from the Acoustics Labo-

ratory of Helsinki University of Technology. Mika Kuuskankare was the first author of

14 of the articles. These publications are listed below:

International Scientific Journals

1. M. Kuuskankare and M. Laurson. Expressive Notation Package. Computer Music

Journal, 30(4):67–79, Winter 2006. (in press).

2. M. Laurson, V. Norilo, and M. Kuuskankare. PWGLSynth: A Visual Synthesis

Language for Virtual Instrument Design and Control. Computer Music Journal,

29(3):29–41, Fall 2005.

24 1. Introduction

International Conference Proceedings

1. M. Castren, M. Laurson, and M. Kuuskankare. MacSet: A Pcset-Theoretical

Computer Application. In 26th Annual Meeting of the Society for Music Theory,

Madison, 2003.

2. K. Kuitunen, M. Kuuskankare, and M. Laurson. Large Scale Duration Organiza-

tion in ‘Hodoi to Ergo’. In Proceedings of International Computer Music Conference,

pages 713–716, 2005.

3. M. Kuuskankare and M. Laurson. ENP2.0 A Music Notation Program Imple-

mented in Common Lisp and OpenGL. In Proceedings of International Computer

Music Conference, pages 463–466, Gothenburg, Sweden, September 2002.

4. M. Kuuskankare and M. Laurson. SpeCS, a Spectrogram-based Compositional

System. In Symposium on Systems Research in the Arts 2002 -IIAS, Baden-Baden,

2002.

5. M. Kuuskankare and M. Laurson. ENP-Expressions, Score-BPF as a Case Study.

In Proceedings of International Computer Music Conference, pages 103–106, Singa-

pore, 2003.

6. M. Kuuskankare and M. Laurson. ENP User-interface: Some Issues with Expert

Systems in Music Notation. In Symposium on Systems Research in the Arts -IIAS,

Baden-Baden, 2003.

7. M. Kuuskankare and M. Laurson. Expressive Notation Package - an Overview.

In International Symposium on Music Information Retrieval, 2004.

8. M. Kuuskankare and M. Laurson. Intelligent Scripting in ENP using PWCon-

straints. In Proceedings of International Computer Music Conference, pages 684–687,

2004.

9. M. Kuuskankare and M. Laurson. Recent Developments in ENP-score-notation.

In Sound and Music Computing ’04, October 2004.

10. M. Kuuskankare and M. Laurson. Some Aspects on Music Scripting Languages.

In Symposium on Systems Research in the Arts -IIAS, 2004.

11. M. Kuuskankare and M. Laurson. Annotating Musical Scores in ENP. In Interna-

tional Symposium on Music Information Retrieval, 2005.

1.4 The ENP Project 25

12. M. Kuuskankare and M. Laurson. ENP Expression Designer - a Visual Tool for

Creating User Definable Expressions. In International Computer Music Conference,

pages 307–310, 2005.

13. M. Kuuskankare and M. Laurson. ENP Note Head Designer. In Journees

d’Informatique Musicale, 2005.

14. M. Laurson and M. Kuuskankare. Instrument Concept in ENP and Sound Syn-

thesis Control. In Journees d’Informatique Musicale, Marseille, 2002.

15. M. Laurson and M. Kuuskankare. PWGL: A Novel Visual Language based on

Common Lisp, CLOS and OpenGL. In Proceedings of International Computer Music

Conference, pages 142–145, Gothenburg, Sweden, September 2002.

16. M. Laurson and M. Kuuskankare. Aspects On Time Modification In Score-Based

Performance Control. In Proceedings of SMAC 03, pages 545–548, Stockholm,

Sweden, August 2003.

17. M. Laurson and M. Kuuskankare. From RTM-notation to ENP-score-notation. In

Journees d’Informatique Musicale, Montbeliard, France, 2003.

18. M. Laurson and M. Kuuskankare. Some Box Design Issues in PWGL. In Proceed-

ings of International Computer Music Conference, pages 271–274, Singapore, 2003.

19. M. Laurson and M. Kuuskankare. Copy-synth-patch: A Tool for Visual Instru-

ment Design. In Proceedings of International Computer Music Conference, pages

47–50, 2004.

20. M. Laurson and M. Kuuskankare. PWGL Editors: 2D-Editor as a Case Study. In

Sound and Music Computing ’04, October 2004.

21. M. Laurson and M. Kuuskankare. Extensible Constraint Syntax Through Score

Accessors. In Journees d’Informatique Musicale, 2005.

22. M. Laurson and M. Kuuskankare. Micro textures with macro-notes. In Proceed-

ings of International Computer Music Conference, pages 717–720, 2005.

Finnish Conference Proceedings

1. H. Penttinen, J. Pakarinen, V. Valimaki, M. Laurson, M. Kuuskankare, and V. No-

rilo. Kiinalaisen kielisoittimen mallipohjainen aanisynteesi. In Akustiikkapaivat,

Jyvaskyla, Finland, November 2005.

26 1. Introduction

Other Publications

1. M. Kuuskankare. Enpaattista nuottikirjoitusta. Tietoyhteys, (2), 2003.

2. M. Kuuskankare and M. Laurson. Expressive Notation Package - a tool for music

theoretical applications. Nordisk Musikforskarkongress, 2004.

3. M. Laurson and M. Kuuskankare. PWGL: A Visual Programming Language for

Computer Aided Composition, Music Analysis and Sound Synthesis. Nordisk

Musikforskarkongress, 2004.

ENP has also been an essential tool in two of the most recent research projects con-

ducted at the Sibelius Academy: Sounding Score - Modeling of Musical Instruments, Vir-

tual Musical Instruments and their Control and CLARA - Control Language for Audio Related

Applications. ENP is among the main tools included in CLARA. It was used in these

projects to produce musical demonstrations of several virtual instruments (e.g., guitar,

clavichord, harpsichord, and guqin) developed by the team members.

The author prepared the WWW pages (http://www.siba.fi/PWGL/) describing both

ENP and PWGL, and also lectured on ENP during the Prisma Course held at the

Sibelius Academy, for example. He has given oral presentations and poster sessions

on ENP and its applications at national and international conferences. These activities

are listed below:

Conference Presentations

1. 2002, International Computer Music Conference, Gothenburg, Sweden.

2. 2002, Symposium on Systems Research in the Arts, Baden-Baden, Germany.

3. 2003, Journees d’Informatique Musicale, Montbeliard, France.

4. 2003, 26th Annual Meeting of the Society for Music Theory, Madison, USA

5. 2003, International Computer Music Conference, Singapore.

6. 2004, 5th International Conference on Music Information Retrieval, Barcelona,

Spain.

1.5 The Scope of this Report 27

7. 2004, International Computer Music Conference, Miami, USA.

8. 2005, Journees d’Informatique Musicale, Paris, France.

9. 2005, International Computer Music Conference, Barcelona, Spain.

10. 2005, 6th International Conference on Music Information Retrieval, London, Eng-

land.

Other Presentations (Posters, etc.)

1. 2004, 14th Nordic Musicological Congress, Helsinki, Finland.

2. 2006, 10th Symposium for Music Research in Finland, Turku.

ENP has been tested primarily through the aforementioned activities. The publica-

tions have provided a scientific framework within which it has undergone the inspec-

tion of an international jury of peers, while both the technical and artistic demonstra-

tions have provided another framework for testing the concepts and methods behind

ENP in practice.

1.5 The Scope of this Report

This report gives an overview of ENP and its applications.

Although the potential of ENP as a music notation program is realized fully when

used in tandem with PWGL, these applications are only partially covered here.

Including all the PWGL applications would have yielded an unreasonably volumi-

nous document, given the nature of this project, and it would have been less straight-

forward to identify the individual contributions of the members of the development

team.

As a result, however, several important applications (e.g., virtual instrument con-

trol) and a considerable amount of the author’s work remain beyond the scope of the

report. In addition to developing ENP and its applications during the four-year project,

28 1. Introduction

the author has also been involved in developing PWGL, its different components, and

the underlying concepts.

This report only covers the notational front-end of ENP. The expression scheme is

of primary importance in several of its applications, and most of the included articles

deal with it in one form or another. The possibilities of annotating ENP scores, both

manually and algorithmically, are discussed in detail. Furthermore, there is a special

focus on a visual tool that could be used to create user-definable expressions. The re-

maining topics include the rule-based scripting language, ENP-Script, and the textual

score description format, ENP-score-notation, both of which are important large-scale

applications inside ENP. The former can be used to algorithmically perform various

operations on an ENP score, and the latter for various purposes, including exchang-

ing, describing and manipulating score-level information. Finally, visualization has

been a major concern throughout the process of developing ENP. These concepts are

present to a greater or lesser degree in all ENP applications.

1.6 The Contents of this Report

This report is divided into two parts. The first part consists of an introductory essay

and the second part of seven articles. The purpose of the introductory essay is to give

supplementary examples in addition to those given in the articles.

The articles, in turn, describe some of the most important concepts behind ENP. The

first one [P1] gives an overview of ENP describing the underlying principles of internal

data representation, the expression scheme, textual score notation, and scripting. The

second article [P2], in turn, discusses the expression scheme in detail and provides sev-

eral practical examples. Thirdly, [P3] gives an initial report of the scripting language,

ENP-Script, and [P4] and [P5] together describe the creation and development of a

text-based score representation, ENP-score-notation. Finally, [P6] and [P7] introduce

two visual tools that can be used to design user-definable extensions to ENP.

1.7 The Structure of this Report 29

The latter two pairs of articles ([P4], [P5] and [P6], [P7]) in tandem give a com-

prehensive overview of their respective topics, ENP-score-notation and visual object

design in ENP, which therefore are not covered in this introductory essay.

1.7 The Structure of this Report

The publications (P1-P7) comprise the major part of this report. Chapter 2 of this In-

troductory Essay gives some complementary information on several aspects of nota-

tion. First, it illustrates—through examples—the notational abilities of ENP, secondly,

it introduces a collection of typical ENP-expressions that show unique behavior when

compared to the traditional approach taken in most notation programs, and thirdly,

it provides several concrete applications of ENP-script. Chapter 3 offers some con-

cluding remarks and suggestions for future directions, and finally, Chapter 4 gives a

summary of the articles and the author’s contribution. The rest of the report consists

of the articles.

1.8 A Note on Terminology

Some essential terms used throughout this report have somewhat different meanings

than their counterparts in the common terminology of Western music. These terms are

clarified here.

Expression The term expression—more specifically ENP-expression—is used here to

denote an ENP-specific device that can be used to represent multipurpose graphical

objects as part of a musical texture. The traditional expression markings, such as artic-

ulations and dynamic markings, form a subset of ENP-Expressions.

Chord and Note The components of an ENP score are enumerated and named in

[P1] and presented here for convenience (see Figure 1.5). It is important to point out

30 1. Introduction

that the terms chord and note are used in the context of object-oriented terminology:

they are two different objects in the hierarchical structure of an ENP score. Chords are

container objects holding an arbitrary number of notes. Notes, in turn, are complex

objects holding, among other things, pitch and timing information. Moreover, both

may contain information about expressions.

SCORE

PART

VOICE(MENSURAL)

VOICE(NON MENSURAL)MEASURE

BEAT

CHORD

NOTE

EXPRESSION

Figure 1.5: An overview of the hierarchical object structure of an ENP score.

Chapter 2

Supplementary Examples

Sections 2.1–2.3 give supplementary examples in three important areas: notation and

music representation (2.1), expressions (2.2) and scripting (2.3).

2.1 Notation

The ENP approach to contemporary—and most specifically graphical—notation is rad-

ically different than the one found in other music notation software. The most signifi-

cant difference is that the additional notational graphics in ENP are genuine objects.

This has several advantages: (1) instead of being passive predefined figures, these

graphical objects can adjust their visual appearance and position according to the mu-

sical context, e.g., pitch, time, loudness; (2) they may carry real musical significance,

because they can contain any kind of data, e.g., annotations, other objects (subimages),

information on timing or pitch; (3) the data they contain can be accessed by the user,

e.g., the graphical information can potentially be converted into playback information.

2.1.1 Modern Notation

By using ENP preferences it is possible to hide many different elements of notation

either locally or globally. Figure 2.1 illustrates this technique. The score is written in

31

32 2. Supplementary Examples

look-alike ’time notation’ using mensural (metric) notation. The score is written using

invisible clefs, bar lines, stems, and beams. Each part is also rotated to create a notation

that is similar to that used in some of the works produced in the 1960’s by Bussotti and

others. It should be noted that, unlike in most other notation programs, the score given

here is fully functional: it can be normally edited and played.

Figure 2.1: An example of modern free notation in a mensural context.

2.1 Notation 33

Figure 2.2 shows an example of relatively complex graphical notation (a similar

notation is to be found in Parentheses by J. Levine). In this case, a special ENP-

expression, called Canvas-expression, is used to display the graphical information.

Canvas-expression is a device that can be used to draw arbitrary graphical objects by

constructing them out of a predefined selection of graphical primitives. Thus, it be-

haves like a miniature drawing program inside the score. Some additional examples

of graphical notation are given in Figures 4 and 7 in [P1].

An interesting detail is that the graphical information shown here is generated auto-

matically. A piece of Lisp code (some 60 lines of code) is used to construct the example

containing a selection of randomly generated and placed graphical primitives, such as

circles or filled polygons. All the other score elements except the Canvas-expression

are hidden using ENP preferences.


Figure 2.2: A modern graphical score created using Canvas-expression (for a similar notationsee, for example, Parentheses by J. Levine).

2.1.2 Orchestral Notation

Figure 2.3 gives an example of a score written for full orchestra. It demonstrates some

of the automatic typesetting features of ENP: the grouping and ordering of the instru-

ments, expression placement, and optimized systems (i.e., empty staves are hidden),

for example. A contemporary look is achieved here by hiding empty measures and

2.1 Notation 35

shortening the staff lines in the case of partial measures. The Appendix in [P1] gives

another orchestral excerpt taken from the literature.

Figure 2.3: An orchestral score written in the modern style with partial and hidden measures(Mika Kuuskankare: Soave per Orchestra da Camera, 1996).

2.1.3 Tape Notation

ENP allows special notational styles to be used to change the visual appearance of the

notation without modifying the underlying data structures. A special notational style

can be applied to both mensural and non-mensural notation. Figure 2.4 shows an


alternative way of displaying normal mensural notation using a special style called

tape notation. The part shown in parenthesis displays the underlying musical structure

with its metrical rhythms and exact pitch.

This kind of approach facilitates writing of music that contains discrete data (pitch

and rhythm/time), but which is visualized using non-traditional or specialized nota-

tion. It also allows the musical data to be represented and manipulated normally: tape

notation can be edited, annotated, and calculated, for example. The same principle

could potentially be applied to shape-note, and tablature notation, for instance.

Figure 2.4: An ENP part written using the tape notation style (top). The same part in normalnotation (bottom).

2.1.4 Music Analytical Notation

ENP also allows for the representation of music analytical information, such as

Schenkerian graphs. It currently follows the fairly traditional scheme according to

which analytical notation is created by inputting the rhythmic and melodic structure

and then embellishing the score with various markings (see Figure 2.5). However, ENP

has in this case the following advantageous features: (1) the markings attached to the

notation (beams, slurs, text), follow the notational objects they are associated with;

(2) the additional markings are genuine objects, i.e., they may also contain additional

information, such as textual notes; and (3) it is possible to algorithmically inspect, gen-

erate and modify the analysis information.

2.2 ENP-Expressions 37

Figure 2.5: A Schenkerian analysis graph realized with the help of ENP (adapted from [5])

It is also possible to split this kind of analysis information into layers, which allows

one to manipulate and view one layer of information at a time and to see the potential

hierarchical structure of the information more clearly. Figure 2.6 gives an example in

which the layers are expanded into two different staff systems.

Figure 2.6: The two layers found in the graph shown in Figure 2.5 displayed in their ownseparate staves.

2.2 ENP-Expressions

This section gives examples of several ENP-Expressions that have unique features

when compared to traditional expression markings (see [P1] for an introduction to

the expression scheme). Figure 6 in [P1] introduces some typical ENP-expressions and

illustrates some important concepts and behavior. [P2] also gives several practical ap-

plications, and a visual editor aimed at creating new expressions is in turn described

in [P6].

Figure 2.7 gives an example that contains various standard expressions. Besides

standard expressions, ENP is also capable of representing different kinds of specialized

expressions. [P1] gives some examples of Score-BPF and Canvas-expression, both of


which are highly specialized expressions that can be used to visualize different kinds

of graphical information as part of music notation.

Figure 2.7: An ENP score containing various standard expressions, such as dynamics, accents,slurs, and tremolos (Mika Kuuskankare: Caracteres II, for string quartet).

One important feature of ENP-expressions is that they can be dynamic, i.e., they

can react to the musical context in several different ways. In its simplest form this may

mean avoiding collision with notes and chords, as is the case with most of the standard

expression markings displayed in Figure 2.7. However, the side effect may also be

more complex, e.g., changing the shape or size of the expression, or the information that

is displayed. Moreover, expressions may also produce side effects in the notation itself.

The following subsections enumerate ENP-expressions that have advanced behavior.

2.2.1 Lyrics

Figure 2.8 gives an example of when the expressions produce side effects in the no-

tational objects to which they are attached. Vocal music is traditionally beamed and

flagged according to the syllables in the text. Through the use of the Lyrics-expression

ENP can automatically beam vocal music according to the traditional rules, as shown

in the second measure of Figure 2.8. Note, that this is melodically and rhythmically

identical to the sixth measure, the only difference being that there is no lyrics informa-

tion in the sixth measure and thus the latter is beamed as in instrumental music.


Figure 2.8: Beams are broken automatically as appropriate when the user enters lyrics infor-mation into the score (Georges Bizet: Carmen).

2.2.2 Dynamic Expressions

Figure 2.9 shows a collection of ENP expressions that dynamically react to their mu-

sical context. They interpret and visualize the data contained in the notational objects

with which they are associated. The PC-Clock, shown in Figure 2.9a, is an expres-

sion that dynamically visualizes the pitch-class set content of a chord using numeric

notation (i.e., c=0, c]=1).1 The clock-face is arranged so that each hour represents a

different pitch-class, 12 o’clock being the first (0) and 11 o’clock being the last (11). The

highlighted (black) hours represent pitch-classes that are present in the chord and the

grayed-out ones, in turn, represent those that are not present.2 The guitar frame (Fig-

ure 2.9b), the trumpet fingering expression (Figure 2.9c) and the harp diagram (Figure

2.9d) are examples of expressions that are both dynamic and contain information about

the associated instrument. The guitar frame is able to dynamically indicate on a fret

diagram on which string and fret a note must be played. Trumpet fingering, in turn,

shows the appropriate combination of three valves needed to play the corresponding

note, and the harp diagram shows the required pedal positions.

Figures 2.9e–h show the same collection of expressions after the score has been

transposed. The dynamic expressions have automatically updated the displayed in-

1For information about pitch-class set theory and related terminology see [3].2Due to pitch-class duplicates (e and e2) the six pitches are reduced to five pitch-classes in the PC-

clock expression.


formation according to the pitch content of the objects to which they are attached.

Figure 2.9: A collection of dynamic ENP-expressions: (a) a PC-clock, (b) a guitar frame, (c)a trumpet fingering, and (d) a harp pedal. The bottom line gives the score after it has beentransposed.

The aforementioned dynamic expressions can also be used to validate instrumental

texture (see the highlighted items in Figure 2.10). The trumpet fingering expression

indicates that the small octave f is not inside the regular range of the instrument (the

C-trumpet in this case).3 The last entry in the harp pedal expression, in turn, indicates

that it is not possible to tune the B-string to both B[ and B\ simultaneously.

Figure 2.10: The encircled items show that there are mistakes in the instrumental notation.

3In this case, the normal range is used. The small octave f] is the lowest natural pitch of the C-trumpet. However, the trumpet, like any brass instrument, is capable of playing so-called pedal tones,that extend its range below the normal.


2.2.3 Expressions Containing Break-point Functions

There are some expressions that contain break-point functions (this information can be

accessed for the purpose of playback, virtual instrument control, or algorithmic com-

position). These expressions include Score-BPF [P1], and all the agogics expressions

(i.e., accelerando, ritardando, and rubato), and continuous dynamics expressions (i.e.,

crescendo and diminuendo). They can visualize themselves according to their current

state. Normally, a crescendo is displayed as a traditional hairpin, and agogics as text, as

shown in Figures 2.11a and b. Figures 2.11c and d, in turn, show the same expressions

with their internal break-point functions visible.

These expressions may also be displayed in an editable state (any object can be

entered in an editable state by double-clicking). In this case the break-point function

can be edited directly in place while the musical material remains in the background

for reference.

Figure 2.11: Two ENP expressions that internally contain a break-point function: (a) ac-celerando, and (b) crescendo. The lower score is the same, but in this case the expressions aredrawn so that their internal break-point functions are shown (c and d).


2.2.4 Instrument-sensitive Expressions

An ENP-expression may have a different graphical representation depending on the

instrument it is written for. These kinds of expressions are called instrument-sensitive.

Figure 2.12 gives an example of when an expression is used to denote that a passage of

music is to be played on a certain string. For example, it is usually written differently

in a guitar part than in a violin part, and if the user changes the instrument of a part,

all of the instrument-sensitive expressions in it adjust their appearance accordingly.

Figure 2.12: An ENP expression that is used to denote a string number for stringed instru-ments is able to adjust its visual appearance according to the instrument it is written for. Thefigure shows the same expression (string number 4) as written for Guitar (top) and Violin(bottom).

2.2.5 Auto-canceled Expressions

Auto-canceled expressions are special kinds of group expressions in ENP. They can be

used to indicate various temporary changes in playing technique (e.g., pizzicato, col

legno) or timbre (e.g., con sordino, sul tasto), for example. Auto-canceled expressions

automatically mark both the beginning and the end of a passage with appropriate tex-

tual information.

Traditionally, this kind of information is manually attached to the beginning of a

given passage. Likewise, the appropriate ’cancelation’ marking is manually written

where the passage ends. ENP, in contrast, requires only the affected passage to be

marked, it then deals with the appropriate cancelation marking. There are several

2.3 ENP-Script 43

advantages in this: it is less prone to errors than the manual approach, and it is possible

to copy part of a passage and have the playing technique copied with it.

Figure 2.13 shows the basic behavior of a pizzicato expression. The ”ord.” instruc-

tion (to change to ’ordinario’ or ordinary playing technique, i.e., bowing, in the case of

the Cello) is the default cancelation marking for this particular expression. It is added

automatically after the effect of the pizzicato expression has ended.

Figure 2.13: Auto-canceled expressions provide a convenient way of indicating different play-ing techniques. ENP takes care of restoring the previous playing technique automatically.

2.3 ENP-Script

ENP-Script is a scripting language that can be used to produce various side-effects on

an ENP score. The scripts can also be applied only to a selection of notational objects

(the selection can be made through the GUI). Its potential applications include:

1. adding expressions,

2. adding analytical information,

3. adjusting timing,

4. changing color, and

5. manipulating pitch and rhythm.

ENP-Script uses a syntax that is identical to the PWGLConstraints pattern-

matching language. The ’ENP-Script’ section in [P1] covers the basic concepts and also

gives a short presentation of the scripting syntax. The syntax is not explained here. In-

stead, the reader is advised to study [38] and [P3]. Furthermore, recent developments


in the syntax are covered in detail in [43], and the examples given below follow the

new syntax presented therein.

The following subsections provide examples of the applications enumerated above.

The first three, given in Subsection 2.3.1, illustrate the use of ENP-Script as an input

tool to automatize repetitive or routine tasks. Subsection 2.3.3 shows how using a script

enables visual aspects, such as size, position and appearance, of the notational objects

to be manipulated, and Subsection 2.3.4 considers ENP-Script as a music analysis tool.

2.3.1 Input Shortcut

Inputting Pitch

The aim in the next example is to enter the first two measures of Etude Op.25, No.10

by Frederick Chopin. The passage contains chromatic texture played in octaves with

several enharmonic alterations. The starting point, given in Figure 2.14, was created

by manually inputting the first voice then duplicating and transposing it to create the

second voice (by default, ENP writes both voices in the first staff of the staff system).

Two scripts are given in Figures 2.16 and 2.17 respectively. The first one maps

through all the chords in the passage (indicated by the :chord keyword in 2.16a). A

new note, being an octave higher with the same enharmonic identity as the existing

one, is inserted in each chord (2.16d–h). The second script, in turn, maps through all

the notes in the score including the ones inserted by the first script. If the pitch of a

note is below a limit given in 2.17c it is assigned to the lower of the two staves (2.17d).4

The resulting score can be seen in Figure 2.15.

In this case, two separate scripts are needed because the operations are not com-

mutable. The properties of the notes cannot be checked before the new ones have all

been inserted into the score, which is why it is imperative that the scripts are executed

sequentially and in the correct order.

4The staff numbering in ENP starts from 0.

2.3 ENP-Script 45

Figure 2.14: An initial input created by inputting the first voice then duplicating it and trans-posing it to create the second voice. The aim is to enter the first two measures of Etude Op.25,No.10 by Frederick Chopin.

Figure 2.15: The finalized input realized with the help of ENP-Script. The octaves and cross-staff changes are created automatically (Frederick Chopin: Etude No.10, Op.25 in B Minor).

a (* ?1 :chord

b (?if

c (when (m ?1 :complete-case? t)

d (mapcar #’(lambda(note)

e (insert-note ?1

f :midi (+ (midi note) 12)

g :enharmonic (enharmonic note)))

h (notes ?1))))

i "Add octave(s) to a chord.")

Figure 2.16: ENP-Script consisting of a scripting rule that inserts an octave in every chord inthe score.


a (* ?1

b (?if

c (when (< (midi ?1) 59)

d (setf (clef-number ?1) 1)))

e "Assign qualifying notes to lower staff.")

Figure 2.17: ENP-Script that assigns notes to the appropriate staves according to their pitchcontent.

2.3.2 Inputting Rhythm

The script in Figure 2.20 can be used to change the time signatures of the score without

changing the face value of the notes. It consists of two scripting rules. The first one

(2.20a–e) sets the denominator of each time signature to half of the original (2.20d),

and the second one 2.20g–k compensates the change of time signature by halving the

unit-lengths of the main-beats (2.20i). The initial score is shown in Figure 2.18, and the

modified result in Figure 2.19.

Figure 2.18: An initial score written with time signatures that have a quarter note as thedenominator.

Figure 2.19: The resulting score, after applying a script, that changes the denominators of thetime signatures and the unit-lengths of the main-beats. This results in a score that has differenttime signatures but the same note face values as the original.

2.3 ENP-Script 47

a (* ?1 :measure

b (?if


d (setf (low ?1) (/ (low ?1) 2))))

e "Halve the denominators of time signatures.")

g (* ?1 :beat

h (?if

i (when (m ?1 :complete-case? t)

j (setf (unit-length ?1) (/ (unit-length ?1) 2))))

k "Halve the unit-length of main-beats.")

Figure 2.20: ENP-Script consisting of two scripting rules (a–e and g–k). The first rule changesthe denominators of the time signatures, while the second one changes the unit-lengths of themain-beats.

Inputting Expressions

It is possible to use ENP-Script as a notational shortcut to insert additional attributes

into the score. This is especially convenient when there is a repeating articulation pat-

tern to be entered. Figure 2.21 shows a passage with a recurring articulation pattern

consisting of two slurred notes followed by four notes with staccatos. Figure 2.22 gives

the corresponding ENP-Script with two scripting rules. The given pattern is repeated

on every downbeat (see 2.22b and g). The benefit of this use of a script is naturally more

pronounced when the musical passage in question is very long.

Figure 13 in [P1] gives an additional example of how to use ENP-Script as a short-

hand to input expressions.

Figure 2.21: The resulting score, after running a script that applies a constant articulationpattern. The slurs and staccatos are added automatically by the script shown in Figure 2.22(Ludwig van Beethoven: String Quartet Op. 18, No. 6).


a (* ?1 ?2

b (?if (when (downbeat? ?1)

c (add-expression ’slur ?1 ?2)))

d "Add a slur to the notes.")

f (* ?1 ?2 ?3 ?4 ?5 ?6

g (?if (when (downbeat? ?1)

h (add-expression ’staccato ?3 ?4 ?5 ?6)))

i "Add a staccato to the last four notes of the group.")

Figure 2.22: ENP-Script showing two scripting rules (a–d and f–i) that define and insert anarticulation pattern into an ENP score.

2.3.3 Score Manipulation

The next example uses the beginning of Prelude Op. 28, No. 8 by Frederick Chopin as

a starting-point. In the original score the 32nd notes in the right-hand staff are written

using a smaller font. ENP accomplishes this by: (1) reducing the size of the beats in the

upper voice, and (2) adjusting the stems so that they align correctly with the note heads

in the lower voice. Making this kind of manipulation using the GUI would prove to be

too laborious. Figure 2.25 gives the script used to perform the aforementioned editing

steps. First, the size of the beats in the first voice (2.25a) is adjusted to 60% of the

original (2.25d), then the second scripting rule 2.25g–k is applied to shift the chords

to the right (2.25j) to align the stems with the note heads in the second voice in the

right-hand staff.

Figure 2.23: A score displaying an excerpt taken from Prelude Op. 28, No. 8 by FrederickChopin. In the original score the fast figures (32nd notes) in the right-hand staff are writtenwith a smaller font.

2.3 ENP-Script 49

Figure 2.24: The score given in Figure 2.23 after running the script. (Frederick Chopin:Preludes Op. 28, No. 8).

a (* ?1 :beat :part (1 1)

b (?if


d (setf (beat-scaling ?1) 0.6)))

e "Set beat size of a main-beat.")

g (* ?1 :chord :part (1 1)

h (?if

i (when (m ?1 :complete-case? t)

j (setf (x-offset ?1) 0.2)))

k "Move chords in the first voice by 0.2 units.")

Figure 2.25: An ENP-Script that performs some score manipulation. The combined effect ofthe two scripting rules (a–e and g–k) is to set the size of the beats in the first voice to 60% of theoriginal and to shift the chords slightly to the right to be aligned with the notes in the secondvoice.

Next, an additional example in the context of non-mensural notation is given.

Figure 2.26 shows an initial score containing chords that internally form written ac-

celerandi and ritardandi gestures. In this example there are six different chord objects.

The chords, in turn, are written so that the notes inside them have positive offset-times.

In mensural notation this operation has no visual impact but in non-mensural notation

the notes with non-zero offset-times are also visually displaced accordingly.

As Figure 2.26 shows, the boundaries between different gestures cannot be easily

perceived. However, in non-mensural notation, it is possible to display optional in-

formation (e.g., different kinds of beams) that makes it easier to identify the internal


structure of the chords.

To make the excerpt given in Figure 2.26 more readable, an ENP-script is used to

find and mark the structurally coherent gestures (see the script in Figure 2.28). Here,

the internal offset-times of the notes are calculated and sorted in ascending order

(2.28e–h). Next, this information is interpreted and the appropriate structural infor-

mation is inserted into the corresponding chord objects: lines 2.28i–j determine if the

ordered list of offset-times result in a decelerating gesture (i.e. the offset-times are in-

creasing) and insert a ritardando beam (:rit-2) if appropriate; lines 2.28k–l do the

opposite, while 2.28m–r inserts a normal beam (32nd beam) in case all the offset-times

inside the chord are within ±10 milliseconds of one another. Figure 2.27 gives the

resulting score.

Incidentally, the approximately constant offset-times found in the last case, could be

used to determine the required number of beams, but three beams are used consistently

in this example.

Figure 2.26: The initial non-mensural score containing six chords (a–f). The chords, in turn,contain notes with individual offset-times (relative to the start-time of the chord) forming ac-celrandi and ritardandi gestures.

2.3 ENP-Script 51

Figure 2.27: The resulting score with some structural information that has been inserted withthe help of ENP-script.

a (* ?1 :chord

b (?if


d (when (rest (notes ?1))

e (let ((dtimes

f (pw::x->dx

g (sort

h (mapcar #’offset-time (notes ?1)) ’<))))

i (cond ((apply #’> dtimes)

j (setf (kind ?1) :rit-2))

k ((apply #’< dtimes)

l (setf (kind ?1) :acc-2))

m ((every #’(lambda(x)

n (<= (- (car dtimes) 0.01)

o x

p (+ (car dtimes) 0.01)))

q dtimes)

r (setf (kind ?1) 3)))))))

s "Determine and mark coherent gestures")

Figure 2.28: ENP-Script is used to insert structural information into a non-mensural scoreaccording to the internal offset-times of the notes.

2.3.4 Analysis and Visualization

Figure 2.29 gives an example of a classic voice-leading assignment that includes paral-

lel fifths (copied from [20], p. 66), and figure 2.30 shows an ENP-script that detects and

marks all parallel fifths in a musical texture.


Let us examine the script in more detail. First, 2.30c–e ensure that both harmonies

are complete, i.e., in the four-part texture there are four notes in each of the two har-

monies. In 2.30f–x the code loops through all the two-voice combinations. Lines 2.30n–

r verify that: (1) both consecutive intervals are perfect fifths; and (2) both voices move

parallel to each other. Finally, 2.30s–x inserts red line-shaped expressions between the

notes in both offending voices. Figure 2.29 shows the automatically annotated score

revealing the parallel fifths.

For further examples of automatic music analysis and visualization see [P2] and

Figure 15 in [P1].

Figure 2.29: A voice-leading assignment containing some (forbidden) parallel fifths (after DeLa Motte). The mistakes are marked directly in the musical texture with the help of a script.

2.3 ENP-Script 53

a (* ?1 ?2 :harmony

b (?if

c (let ((c1 (m ?1 :complete-case? t :object t))

d (c2 (m ?2 :complete-case? t :object t)))

e (when (and c1 c2)

f (loop for p1 from 1 to (1- (length c1)) do

g (loop for p2 from 2 to (length c2)

h for c1/n1 = (find p1 c1 :key #’voicenum)

i for c1/n2 = (find p2 c1 :key #’voicenum)

j for c2/n1 = (find p1 c2 :key #’voicenum)

k for c2/n2 = (find p2 c2 :key #’voicenum)

l while (and c1/n1 c1/n2 c2/n1 c2/n2)

m when

n (and (= (abs (- (midi c1/n1) (midi c1/n2))) 7)

o (= (abs (- (midi c2/n1) (midi c2/n2))) 7)

p (/= (midi c1/n1) (midi c1/n2))

q (= (- (midi c1/n2) (midi c1/n1))

r (- (midi c2/n2) (midi c2/n1)))) do

s (add-expression ’score-expression c1/n1 c2/n1

t :kind :line

u :color :red)

v (add-expression ’score-expression c1/n2 c2/n2

w :kind :line

x :color :red))))))

y "Find and mark parallel fifths.")

Figure 2.30: A relatively complex ENP-Script implementing a voice-leading checker that findsand marks parallel fifths in a given score.


Chapter 3

Conclusions

3.1 Summary

This report provides an overview of ENP, which was developed to meet the needs

of computer-aided composition, music analysis and synthesis control. Its main pur-

pose is to provide a front-end for a visual environment, PWGL. It has a mouse-driven

graphical user interface and enables musical scores to be enriched with various nota-

tional attributes, ranging from standard articulation markings to user definable ones.

It also allows for the generation of scores using its textual score description format. A

score can be converted to the textual format and vice versa. This makes it possible to

use PWGL to manipulate musical data in a flexible way. Moreover, ENP allows script-

ing through a specialized language, ENP-Script. This scripting scheme is useful when

used simply as a speedy entry to attach articulations or articulation patterns, for exam-

ple, but it can also be used to perform more sophisticated operations including music

analysis and visualization.

This ENP project was undertaken to develop methods and algorithms related to

computer-aided music notation. So far it has yielded the following results:

1. the ENP music notation program;

2. a report summarizing the main components of ENP;

55

56 3. Conclusions

3. the ENP Reference Manual;

4. twenty-eight publications;

5. a web page (http://www.siba.fi/PWGL/) describing ENP and its host application

PWGL;

6. applications for computer-aided music notation in the following areas: composi-

tion, music research, music analysis, sound synthesis, and musical performance;

7. the development of new interaction and input strategies for musical notation

dealing with the aforementioned applications;

8. better knowledge and expertise within the research area.

3.2 Future Directions

ENP is a work in progress and it is inevitable that there are missing or incomplete

features. Several improvements, new features and applications are planned.

3.2.1 Short-term Enhancements

Typesetting Engine

There is currently no visual synchronization between the two fundamental notational

styles, mensural and non-mensural, which means that events in two parts written in

mensural and non-mensural notation do not horizontally coincide. This is a visualiza-

tion problem that should be addressed. The next project is to rewrite the typesetting

engine to support synchronization between the notational styles, and to implement an

intelligent page layout scheme.

3.2 Future Directions 57

Optimization

There are also some other design details to consider. ENP currently recomputes the

entire display after every modification made to the underlying musical structures. This

makes the screen update demanding for the computer. Computationally more efficient

incremental display models have been suggested, for example, by Dannenberg ([16,

17]). Although there is always room for optimization, computers are becoming so fast

that radical optimization efforts may soon no longer be relevant.

Missing or Incomplete Features

The Undo feature is currently implemented only for destructive operations (e.g., delet-

ing, changing rhythms, adding and removing expressions). This scheme should be

finalized and stabilized to include transposing and, in general, all the potential UI op-

erations.

There are no supported import formats at the moment, and MIDI import, at least,

is of primary importance. There exist GPL licensed MIDI packages, such as MidiShare

([47] and [23]), and Allegro [18], that implement MIDI file I/O. There is also another

Free Software library, LibMusicXML [22], which implements both MusicXML export

and import (incidentally, this would also allow ENP to export Guido [50] music format).

High-Quality Fonts

There is no native font support in OpenGL. There are basically three approaches to

rendering fonts in OpenGL: bitmap, outline, and texture mapped. The texture map

based technique was used to produce the fonts needed for ENP, but this kind of pro-

prietary approach is both hard to maintain and difficult to interface with other systems,

such as Postscript, that rely on real glyph information. Currently, there are free soft-

ware packages, such as FreeType2 [1], for producing and managing high-quality font

information. FreeType2 is a software font engine that provides an API to access font

58 3. Conclusions

content, and supports a wide array of different font formats including TrueType and

OpenType. FTGL [45], in turn, provides OpenGL bindings to FreeType. FTGL is a li-

brary for rendering TrueType fonts in OpenGL and it uses the FreeType2 API to extract

glyph information from standard fonts. It would need some work to re-implement the

font scheme in ENP in order to take advantage of both FreeType and FTGL. However,

the benefits in moving to these solutions would be significant. As FreeType offers real

kerning information it would be easier to produce EPS or PostScript printouts, for ex-

ample. Both these packages are Free Software, and are also cross-platform compatible.

Expanding the Expression Scheme

Currently the ENP-expressions can be attached only to notes or chords. The present

scheme should be extended to apply to all the objects in the score. This would make

it possible to attach annotations to parts, voices, measures, bar lines, and clefs, for

example, and it would also make the ENP object representation scheme more unified

and coherent.

3.2.2 New Avenues of Research

Computer-Aided Composition

The musical problems that are interesting from the perspective of musical constraints

programming are typically very demanding. Harmony, melody, voice-leading, and

counterpoint provide individual and natural challenges in themselves. Defining a col-

lection of rules to solve a certain compositional problem is a time-consuming task and

requires a lot of musical and programming expertise. The PWGL tool for computer-

assisted composition, PWGLConstraints, provides excellent possibilities for address-

ing these kinds of problems. One area for potential improvement, however, is in how

these rules are written by the user, and ENP could provide an alternative solution. It

would be interesting to see how rules could be described by using the language most

3.2 Future Directions 59

familiar to musicians—music notation.

Computer-Assisted Music Analysis

Computer-assisted music analysis is one of the most interesting ENP applications. It

should be possible to provide intelligent expressions that are especially tailored to a

certain analytical framework (e.g. Schenkerian analysis). Furthermore, for the pur-

poses of computer-aided music analysis a stronger representation of analytical infor-

mation is needed. It should be possible to project different views of a piece of music,

i.e., to allow the notation to be displayed in any number of reductions or alternate

forms. This kind of approach would be a powerful tool in computer-assisted music

analysis.

Computer Assisted Instruction

Computer-Assisted Instruction (CAI) refers to a system of educational instruction per-

formed almost entirely by computer. There are many advantages to using computers

in educational instruction: they provide one-to-one interaction with the student and

instantaneous response to the problem at hand, and they also allow students to pro-

ceed at their own pace. The facilities in ENP, including its integration with PWGLCon-

straints and PWGLSynth, make it an ideal CAI tool. Sophisticated tutors for basic

music theory, solfege, post-tonal music theory, interactive harmony, ear training and

other assignments could potentially be created with the help of ENP to replace the

traditional Drill-and-Practice music theory programs.

Content Authoring Tools

Content Authoring Tools (CAT) provide teachers with the means to create new edu-

cational material (music theory exercises, solfege assignments, interactive tutors, ear

training, etc.). They are used to arrange different kinds of visual and textual compo-

nents to create interactive training content. They usually also support the linking of

60 3. Conclusions

material into modules and provide visual and textual feedback for the user. It is also

important for the modules and objects to be available for reuse in a modular fashion.

ENP could provide such tools, including a language that allows the speedy construc-

tion of interactive pedagogical programs. The language should be simple enough to

be learned by those without a programming background, but also powerful enough to

represent advanced learning modules.

Web applications

There are numerous sites that offer Web applications targeted at basic-level music edu-

cation (e.g., http://www.emusictheory.com/, and http://www.teoria.com/). These applica-

tions usually use Macromedia Flash or Java applets to provide some or all of the user

interface components. There have been a few attempts to create plug-ins for rendering

music notation in an Internet browser: FigureRenderer [36] can be used to display Lily-

Pond as part of a web page; Niffty [2] is is a Java applet that allows NIFF [26] formatted

music notation to be displayed on a Web page; and GUIDO NoteServer [29] displays

GUIDO as common music notation on a Web page. There are also commercial Web

plug-ins, such as musicRAIN 2.0 (http://www.mediarain.com/), to facilitate the viewing

of MusicXML.

It would be an interesting field of study to see how ENP could be integrated with

the World Wide Web. In its simplest form it could mean developing an ENP viewer

plug-in that could be used to represent static notational information generated by ENP.

A more advanced version would obviously involve some level of editability. Making

ENP Web-ready would at least require charting the various Web-based formats (such

as Scalable Vector Graphics, SVG) and technologies to identify the most viable solu-

tions.

Chapter 4

A Summary of the Publications

Publication [P1]

[P1] provides an overview of ENP. The article describes an expression scheme that al-

lows to enrich musical scores with various notational attributes ranging from standard

articulation markings to user definable ones. [P1] also explains the principles of a tex-

tual score description format that allows the generation of scores. A score can be easily

converted into the textual format and vice versa, which makes it possible to use PWGL

to manipulate musical data in a flexible way. [P1] further describes a specialized script-

ing language, ENP-Script. This is a scripting scheme that is useful when used simply

as a speedy entry for attaching articulations or articulation patterns, for example, but

it can also be used to perform more sophisticated operations including music analysis

and visualization. The author of this report wrote the article.

Publication [P2]

[P2] gives an overview of ENP-expressions, and presents a number of built-in expres-

sions that are suitable for visualizing music analytical information as a part of music

notation, for example. A system for creating user-definable annotation information

61

62 4. A Summary of the Publications

is also described, and some sample algorithms are given. Finally, a system for auto-

matically analyzing and annotating an ENP score is illustrated through examples. The

author of this report wrote the text, implemented the expression devices, and prepared

the examples. PWGLConstraints has been augmented with new keywords—allowing

richer access to the score—by Mikael Laurson, in collaboration with the author of this

report.

Publication [P3]

[P3] demonstrated how a constraint-based language, PWGLConstraints, can be

adapted to be used to script ENP scores. The paper also presents and examines in

detail a number of example scripts. The author of this report developed the idea of

rule-based scripting, and wrote all of the paper, except Section 2.

Publication [P4]

The focus in [P4] is on how score information is represented in PWGL. A list-based

notation, used in PatchWork (a predecessor of PWGL), is expanded so as to allow the

generation of very detailed scores that contain not only the basic rhythmic structures

but also additional score information. The author of this report wrote Sections 3 and 4.

Publication [P5]

[P5] describes ENP-score-notation, a text-based score representation format. The ad-

vantages in the text-based representation of a musical score are that the layout of the

notational objects is separated from the representation, and that the textual formats

are human-readable. In this case the textual format provides a way to generate and

manipulate musical data in PWGL. The author of this report conceived ENP-score-

63

notation, which—on the beat level—was derived from the PatchWork RTM-notation

but has been enriched in various different ways. The author of this report also wrote

the paper.

Publication [P6]

[P6] introduces a tool called ENP Expression Designer (ED). It is aimed at visually cre-

ating new notational attributes to be used in ENP. Both visual and text-based interfaces

are provided, and it is also possible to mix these approaches. Furthermore, the new ex-

pressions are viewed and designed in a real musical context. These concepts make

the current system unique compared to other approaches. The author designed and

implemented the ED editor and wrote the paper.

Publication [P7]

[P7] presents a visual editor called ENP Note Head Designer (ND), which was devel-

oped to assist both novice and expert users to design custom note heads in ENP. The

same design principles apply as in the case of ED. The author of this report designed

and implemented the ND editor and wrote the paper.

64 4. A Summary of the Publications

References

[1] Freetype. http://www.freetype.org/.

[2] Niffty. http://sourceforge.net/projects/niffty/.

[3] Allen Forte. The Structure of Atonal Music. Journal of Music Theory, 1973.

[4] C. Ariza. An Open Design for Computer-Aided Algorithmic Music Composition. PhD

thesis, New York University, New York, NY, USA, 2005.

[5] D. Beach, editor. Aspects of Schenkerian Theory. Yale University Press, 1983.

[6] A. Belkin. Macintosh notation software: Present and future. Computer Music Jour-

nal, 18(1):53–69, 1994.

[7] P. Bellini, P. Nesi, and G. Zoia. Symbolic music representation in mpeg. IEEE

MultiMedia, 12(4):42–49, 2005.

[8] D. Blostein and L. Haken. Justification of printed music. Communications of the

ACM, 34(3):88–99, 1991.

[9] R. S. Brindle. The New Music : The Avant-garde since 1945. Oxford University Press,

1987.

[10] W. Buxton, S. Patel, W. Reeves, and R. Baecker. Scope in interactive score editors.

Computer Music Journal, 5(3):50–56, 1981.

[11] D. Byrd. Music Notation by Computer. PhD thesis, Computer Science Department,

Indiana University, 1984.

65

http://www.freetype.org/

http://sourceforge.net/projects/niffty/

66 References

[12] D. Byrd. User interfaces in music-notation systems. In Proceedings of International

Computer Music Conference, pages 145–151, 1986.

[13] D. Byrd and E. Isaacson. A music representation requirement specification for

academia. Computer Music Journal, 27(4):43—57, 2003.

[14] A. Cooper. About Face. The Essentials of User Interface Design. IDG Books, Foster

City, CA, 1995.

[15] R. Dannenberg. Music Representation Issues, Techniques, and Systems. Computer

Music Journal, 17(3):20–30, 1993.

[16] R. B. Dannenberg. A structure for representing, displaying and editing music. In

Proceedings of International Computer Music Conference, pages 153–160, 1986.

[17] R. B. Dannenberg. A Structure for Efficient Update, Incremental Redisplay and

Undo in Graphical Editors. Software-Practice and Experience, 20(2):109–132, 1990.

[18] R. B. Dannenberg. Allegro Music Representation Language. http://www.cs.cmu.

edu/~rbd/software/allegro.htm, 2000.

[19] R. B. Dannenberg, D. Rubine, and T. Neuendorffer. The resource-instance model

of music representation. In Proceedings of the 1991 International Computer Music

Conference, pages 428–432, 1991.

[20] D. de la Motte. Harmonielehre. Barenreiter-Verlag, 1976.

[21] G. R. Diener. Modeling Music Notation: A Three Dimensional Approach. PhD thesis,

Stanford University, 1990.

[22] D. Fober, S. Letz, and Y. Orlarey. Open source tools for music representation and

notation. In Sound and Music Computing ’04, 2004.

http://www.cs.cmu.edu/~rbd/software/allegro.htm

http://www.cs.cmu.edu/~rbd/software/allegro.htm

REFERENCES 67

[23] D. Fober, Y. Orlarey, and S. Letz. Midishare joins the open source softwares. In

ICMA, editor, Proceedings of the International Computer Music Conference, pages 311–

313, 1999.

[24] M. Good. Musicxml for notation and analysis. The Virtual Score: Representation,

Retrieval, Restoration, pages 113–124, 2001.

[25] J. S. Gourlay. Spacing a line of music. Technical Report OSU-CISRC-10/87-TR35,

Ohio State University, Columbus, Ohio, 1987.

[26] C. Grande. Beyond MIDI The Handbook of Musical Codes, chapter The Notation

Interchange File Format: A Windows-compliant Approach, pages 491–512. The

MIT Press, Cambridge, Massachusetts, 1997.

[27] H. Honing. Issues in the representation of time and structure in music. Contempo-

rary Music Review, 9:221–239, 1993.

[28] H. Hoos, K. Hamel, K. Flade, and J. Kilian. Guido - music notation - towards an

adequate representation of score level music. In Journees d’Informatique Musicale,

1998.

[29] H. H. Hoos, K. A. Hamel, and K. Renz. Using Advanced GUIDO as a Notation

Interchange Format. In Proceedings International Computer Music Conference, pages

395–398. International Computer Music Association (ICMA), San Francisco, CA,

1999.

[30] D. Huron. Music information processing using the humdrum toolkit: Concepts,

examples, and lessons. Computer Music Journal, 26(2):15–30, 2002.

[31] J. T. M. III and S. M. Ornstein. Mockingbird: A composer’s amanuensis. BYTE,

9(1):384–401, 1984.

[32] E. Isaacson. What you see is what you get: On visualizing music. In International

Symposium on Music Information Retrieval, pages 389–395, London, UK, 2005.

68 References

[33] E. Karkoschka. Das Schriftbild der Neuen Musik. Hermann Moeck Verlag, Celle,

1966.

[34] K. Kuitunen, M. Kuuskankare, and M. Laurson. Large Scale Duration Organiza-

tion in ‘Hodoi to Ergo’. In Proceedings of International Computer Music Conference,

pages 713–716, Barcelona, Spain, 2005.

[35] M. Kuuskankare and M. Laurson. ENP2.0 A Music Notation Program Imple-

mented in Common Lisp and OpenGL. In Proceedings of International Computer

Music Conference, pages 463–466, Gothenburg, Sweden, 2002.

[36] C. Lamb. FigureRender. http://www.chris-lamb.co.uk/code/figurerender/.

[37] K. Lassfolk. Music Notation as Objects. PhD thesis, University of Helsinki, 2004.

[38] M. Laurson. PATCHWORK: A Visual Programming Language and some Musical Ap-

plications. Studia musica no.6, doctoral dissertation, Sibelius Academy, Helsinki,

1996.

[39] M. Laurson, C. Erkut, V. Valimaki, and M. Kuuskankare. Methods for Modeling

Realistic Playing in Acoustic Guitar Synthesis. Computer Music Journal, 25(3):38–

49, Fall 2001.

[40] M. Laurson and M. Kuuskankare. PWSynth: A Lisp-based Bridge between Com-

puter Assisted Composition and Sound Synthesis. In Proceedings of the Interna-

tional Computer Music Conference, pages 127–130, Havana, Cuba, 2001.

[41] M. Laurson and M. Kuuskankare. PWGL: A Novel Visual Language based on

Common Lisp, CLOS and OpenGL. In Proceedings of International Computer Music

Conference, pages 142–145, Gothenburg, Sweden, 2002.

[42] M. Laurson and M. Kuuskankare. Some Box Design Issues in PWGL. In Proceed-

ings of International Computer Music Conference, pages 271–274, Singapore, 2003.

http://www.chris-lamb.co.uk/code/figurerender/

REFERENCES 69

[43] M. Laurson and M. Kuuskankare. Extensible Constraint Syntax Through Score

Accessors. In Journees d’Informatique Musicale, Paris, France, 2005.

[44] M. Laurson and M. Kuuskankare. Micro Textures with Macro-notes. In Proceedings

of International Computer Music Conference, pages 717–720, Barcelona, Spain, 2005.

[45] H. Maddock. FTGL. http://homepages.paradise.net.nz/henryj/index.html.

[46] H.-W. Nienhuys and J. Nieuwenhuizen. LilyPond, a system for automated music

engraving. In XIV Colloquium on Musical Informatics (XIV CIM 2003), Firenze, Italy,

2003.

[47] Y. Orlarey and H. Lequay. MidiShare: a Real Time multi-tasks software module

for Midi applications. In ICMA, editor, Proceedings of the International Computer

Music Conference, pages 234–237, 1989.

[48] B. W. Pennycook. Computer-music interfaces: A survey. Computing Surveys,

17(2):267–289, 1985.

[49] G. Read. Music Notation. Victor Gollancz Ltd., 1982.

[50] K. Renz. Algorithms and Data Structures for a Music Notation System based on GUIDO

Music Notation. PhD thesis, Technischen Universitat Darmstadt, 2002.

[51] K. Renz. An Improved Algorithm for Spacing a Line of Music. In Proceedings

of International Computer Music Conference, pages 475–481, Gothenburg, Sweden,

2002.

[52] C. Roads. The Computer Music Tutorial. The MIT Press, Cambridge, Massachusetts,

London, England, 1996.

[53] Sibelius Software Ltd. Using the ManuScript Language.

[54] L. Smith. Beyond MIDI The Handbook of Musical Codes, chapter SCORE, pages 252–

280. The MIT Press, Cambridge, Massachusetts, 1997.

http://homepages.paradise.net.nz/henryj/index.html

70 References

[55] G. L. Steele. Common LISP : The Language. Digital Press, 2nd edition, 1990.

[56] K. Stone. Music Notation in the Twentieth Century: A Practical Guidebook. W. W.

Norton & Company, 1980.

[57] University of Leeds. Case Study: Representation of Musical Structure for Music Soft-

ware, Leeds, 16-17 September 2003. Music Notation Workshop: XML based Music

Notation solutions.

[58] T. Weyde and J. Wissmann. Visualization of musical structure. In Proceedings of

the Conference on Interdisciplinary Musicology, Graz, 2004.

[59] M. Woo, J. Neider, T. Davis, and D. Shreiner. OpenGL Programming Guide. Addison

Wesley, Massachusetts, USA, 3rd edition, 1999.

Part II

Publications

71

Changes in [P6] and [P7]

The possibility of using textures and transparency—when designing note heads as

described in [P7]—is removed from ENP, because of incompatibility with PostScript.

PostScript does not support true alpha-blending. Furthermore, ENP cannot currently

convert OpenGL textures to PostScript.

Since the introduction of the two editors described in [P6] and [P7] there have been

some changes in terms of appearance and functionality. Currently, these tools are

highly interactive and provide instant visual feedback. Any change made is imme-

diately reflected in the appearance of the designed object. When the user changes the

code in the code-view, the effect is shown in the preview score/canvas. The user is not

required to perform manual updates, as was the case before. The tools also provide

feedback as to the syntactic correctness of the code.

73

74

Errata

In [P4], page 5, first paragraph, “... without scarifying the functionality” should be

“sacrificing”

In [P7], page 6, section “Conclusions”, first paragraph, “... any of the build-in ones.”

should be “built-in”

75

Publication [P1]

[1] M. Kuuskankare and M. Laurson. Expressive Notation Package. Computer Music

Journal, 30(4):67–79, Winter 2006. (in press).

77

Kuuskankare and Laurson

Mika Kuuskankare* and Mikael Laurson†

*Department of Doctoral Studies in MusicalPerformance and Research†Centre for Music and TechnologySibelius AcademyHelsinki, Finland{mkuuskan,laurson}@siba.fi

Expressive NotationPackage

67

At present, there are a large number of commercialand free programs dealing with music notation. Twoof the most notable free software projects are Lily-Pond (Nienhuys and Nieuwenhuizen 2003) andGuido (Hoos et al. 1998; Renz 2002). Both of theseprograms are LaTeX-like languages that are used todescribe the contents of a musical score in a textualform and not through a point-and-click user inter-face. Furthermore, a Web site dedicated to musicnotation software, ace.acadiau.ca/score/others.htm,lists a large number of other commercial and freemusic notation programs including simple notationeditors (e.g., abc) for typesetting relatively simplenotation; special-purpose editors like Grégoire fortypesetting Gregorian chant, Django for typesettingtabulature, and GOODFEEL for Braille notation;and full-featured programs like Berlioz, Igor (pro-grammed in Lisp), Nightingale, and SCORE. Finally,the two de facto commercial notation programs areFinale and Sibelius. Recently, Web-viewable appli-cations have also started to emerge. There are a fewcommercial approaches such as Scorch by SibeliusSoftware. ScoreSVG (Bays 2005), in turn, is a free al-ternative based on the Scalable Vector Graphics, alanguage for describing two-dimensional vectorgraphics in XML.

A handful of Lisp-based applications also existthat are aimed at representing musical data, such asCommon Music Notation (CMN; Schottstaedt1997), the Rhythm-Editor of PatchWork (RTM;Laurson 1996), and the musical editors in OpenMu-sic (Assayag et al. 1999). One of the earliest experi-ments, MUZACS (Kornfeld 1980), was even writtenfor a Lisp machine. Of these editors, RTM is prima-rily intended to represent musical raw material, andthus the editing capabilities are limited. This is alsothe case with Open Music’s notation editors. CMNis powerful typesetting package but lacks a graphi-cal user interface (GUI).

Expressive Notation Package (ENP; Kuuskankareand Laurson 2002) is a music notation program thathas been developed to meet the requirements ofcomputer-aided composition, music analysis, andsynthesis control. ENP is programmed with Lisp-Works Common Lisp (www.lispworks.com). Lisp-Works, in turn, is a Lisp implementation that issource-code compatible across Windows, Linux,Mac OS X, and some UNIX platforms. ENP is pri-marily intended to represent Western musical nota-tion roughly from the 17th century onward with astrong emphasis on modern notation. ENP is not afull-featured music typesetting program. However,it is designed to produce automatic, reasonable mu-sical typesetting according to the common prac-tices described, for example, in Stone (1980) andRead (1982).

ENP is used as a notational front end in a visualprogramming language called PWGL (Laurson andKuuskankare 2002). PWGL is a combination of sev-eral software packages build on top of CommonLisp. In addition to ENP, the components include,for example, a rule-based programming languagecalled PWGLConstraints (Laurson 1996), and asound-synthesis program called PWGLSynth (Laur-son et al. 2005). The development of PWGL is fo-cused toward the Mac OS X operating system, butit currently runs also under Windows XP. A Linuxversion is under consideration. PWGL is freeware,and it can be downloaded from our Web site atwww.siba.fi/PWGL/.

ENP has a sophisticated GUI based on direct edit-ing, a rich set of musical primitives, an extendedconcept of expressions, and algorithmic control overscores. It contains a unique set of features that cannotbe found elsewhere, at least to this extent of integra-tion in one program. ENP allows one to analyze, mod-ify, view, and annotate scores in many different ways;a text-based format can be used to generate scores al-gorithmically; it is possible to modify data containedby the objects in a score by using a scripting language;our rule-based programming language can be used

Computer Music Journal, 30:4, pp. 67–79, Winter 2006© 2006 Massachusetts Institute of Technology.

to calculate the pitch content of an ENP score; and fi-nally, ENP can be used to represent and generate con-trol information for our software synthesizer. Owingto its dynamic nature, ENP is primarily intended forcomposers and music analysts. The system has alsoproven useful in various research activities dealingwith physics-based musical instrument design andcontrol. Furthermore, potential applications that areof interest in the near future are various education-ally related activities that require intelligent feed-back from the computer, such as computer-aidedteaching of counterpoint, harmony, and solfege.

The present article summarizes some of the mostimportant concepts behind ENP. We concentrate onfour main topics: the underlying data representa-tion, the expression scheme, the ENP-score-notation format, and the ENP-Script language.These topics have also been discussed in separatepublications. The data representation has been cov-ered briefly in Kuuskankare and Laurson (2002).The expression scheme has been discussed in gen-eral in Kuuskankare and Laurson (2003, 2005). ENP-score-notation has been reported in both Laursonand Kuuskankare (2003) and Kuuskankare and Laur-son (2004b). Finally, ENP-Script is introduced in Ku-uskankare and Laurson (2004a). This article intendsto elaborate on these topics and to provide largerand more comprehensive examples.

Music Representation in ENP

Understanding the structure of an ENP score is ofprimary importance, because it is reflected in everyaspect of ENP: its GUI, expression scheme, script-ing, etc. There is an excellent introduction to manydifferent representation schemes in Selfridge-Field(1997). Music representation issues in general havealso been discussed, for example, in Diener (1990),Huron (1992), Honing (1993), Dannenberg (1993),Roads (1996), and Lassfolk (2004).

An ENP score is built out of hierarchical objectstructures. There are two fundamental notationalstyles: mensural and non-mensural notation. Typi-cally, a mensurally notated ENP score contains anumber of parts; each part contains voices; eachvoice contains measures; and each measure can

contain any number of beats. Beats, in turn, cancontain other beats (to create complex rhythmicstructures) or chords. Finally, chords can contain anarbitrary number of notes. Notes and chords canalso contain information about additional attributessuch as expressions.

Figure 1 shows a short musical excerpt written ina mensural style along with the corresponding hier-archical tree structure of the score. In the tree view,objects are represented by their ENP class name(e.g., NOTE means an ENP note object). The numbersfollowing the class names have two different mean-ings. In case of beats, they indicate the proportionalunit length of the beat, and in the case of notes,they indicate the MIDI note number. The last beatin the score and the corresponding part in the objectstructure are both marked with a gray rectangle.

The beat-level ENP score representation is basedon the concept of PW-beat (Laurson 1996). PW-beatitself is a hierarchical structure that can be used torepresent complex metrical rhythms. Each beat hastwo important properties: the unit-length and po-tential sub-beats. By manipulating these properties,it is possible to represent a comprehensive set ofrhythms sufficient for most practical purposes. Fig-ure 2 gives detailed examples of some rhythms andtheir corresponding structural representations.Again, the unit lengths of the beats are written in-side parentheses after the class name of the object.The main unit in this case is always a quarter note.(See the first beat on the top left.)

Although the beat representation is closely relatedto the one used in PatchWork, the score representa-tion, in general, is considerably different from that ofPatchWork. ENP object structure now reflects thetraditional structure of a (paper) score, with parts,voices, instruments, key signatures, expressions,etc. Moreover, the rhythmic structures have beenaugmented to include grace notes, for example.

The difference in non-mensural notation com-pared to mensural notation is that internally, a non-mensural voice contains a collection of chordswith absolute start-times and durations. The non-mensural notation can be used, for example, whenwriting contemporary “time-notation” (proportionalnotation). Figure 3 gives an example of an ENPscore that is written using non-mensural notation.

68 Computer Music Journal


The fundamental notational styles in ENP behavedifferently from the approach typically used inother notation software, whereby the user writestime notation in the normal mensural context. Inthe latter case, only the appearance of a time nota-

tion is achieved by removing certain graphical com-ponents of the notation, such as stems, bar lines,etc. The internal structure of the data does not re-flect the fact that the notation is free of any metri-cal structures. In ENP, by contrast, the structure of

69

Figure 1. An ENP score(bottom) and the corre-sponding tree view of theENP score structure (top).(Igor Stravinsky, Rite ofSpring, reproduced by kindpermission of Boosey andHawkes Publishers, Ltd.)

Figure 2. Two different rep-resentations of three rhyth-mic structures in ENP.

Figure 1

Figure 2

the data is reflected directly in the notational out-put. It also determines how the user interface be-haves: There are no measure boundaries, and thenotational objects can be freely dragged in time.However, by adjusting ENP preferences, it is pos-sible to hide many different elements of notation ei-ther locally or globally. Figure 4 gives an exampleusing this technique: a score written in “time nota-tion” using mensural (metric) notation. The line-shaped expressions found in this example areattached to notes that they connect. Thus, the ex-pressions logically follow the pitch contour if thescore is, for example, transposed. This kind of be-havior is also different from the existing approacheswhere graphical information is often consideredpassive in nature.

Next, we briefly demonstrate the concept ofvoices in ENP. As discussed previously, a part cancontain an arbitrary number of voices. A part can bedrawn using one of the available systems, which, inturn, can contain single or multiple staves, eachhaving their own clefs, etc. A voice can then bedrawn either on one individual staff, or it can tem-porarily cross staves within the system. Figure 5shows an organ part containing four voices that arewritten in a typical organ system consisting of threestaves. Two voices share the topmost staff, and theremaining two occupy the other staves. The Appen-dix gives an example written for full orchestra: thefirst page of Elliot Carter’s composition A Mirror onWhich to Dwell.

Expression Scheme

The notational attributes, ENP-Expressions, are ofprimary importance in terms of our software. Here,we use the term expression in a broader sense thana traditional expression marking. ENP-Expressionsare multipurpose visualization devices that can beused to represent complex dynamic Lisp-based ob-jects as part of a musical texture. In addition to

their traditional use, ENP-Expressions can be usedin a wide range of applications: display of music-theoretical analysis information, such as annotationof motives and harmonic progressions; visualizationof specialized analytical information, such asSchenkerian graphs and pitch-class-set theoreticalinformation; attachment of arbitrary textual anno-tations, names, or comments to objects; and dy-namic inspection and visualization of datacontained by the notational objects, such as pitch,interval duration, velocity, and onset time. Theycan also be used to modify the notational output it-self. (See Figure 4 for an example.)

ENP provides a predefined set of both standardand non-standard expressions. Standard ENP-Expressions include, for example, staccatos, slurs,accents, and playing styles. Non-standard expres-sions include, among others, Group, Canvas-expression, and Score-BPF (Kuuskankare andLaurson 2003). Of these, Canvas-expression is amultipurpose graphical device that can be used todraw graphical objects in a score. Score-BPF, in turn,is a specialized group expression that can representbreak-point functions as a part of a musical texture.ENP also offers a visual editor that can be used tocreate new expressions through a GUI. (See the sec-tion “ENP Expression Designer.”)

ENP-expressions can be dynamic, i.e., they canadjust their visual appearance according to the nota-


Figure 3. A score written incontemporary time nota-tion. There is no underlyingmetric structure present;instead, musical events arenotated with absolute on-set times and durations,proportionally spaced.

Figure 4. An example ofmodern time notation in amensural context. A line-shaped expression is usedto connect the invisible

(hidden) note heads tomimic the string glissandowriting of, for example,Krzysztof Penderecki.


tional context. Expressions are also editable directlyin the score; therefore, adjusting the slope of a sluror repositioning an articulation can be done interac-tively. Figure 6a gives a simple score containing var-ious ENP-Expressions. Figure 6b, in turn, shows thesame score after it has been transposed up by a mi-nor sixth.

Canvas-Expression is a device that can be used todraw graphical objects by constructing them out ofa predefined selection of graphical primitives. Cur-rently, these primitives include text objects, lines,arrows, filled or framed polygons, and circles.Canvas-Expressions behave as miniature drawingprograms inside the score. Figure 7a shows an ex-ample of a Canvas-Expression that is in an editablestate, i.e., all the shapes inside the dark canvas canbe moved, reshaped, added, and deleted. Figure 7b,in turn, shows how relatively complex graphical no-tation can be entered with the help of Canvas-Expression. This particular example mimics thewriting that can be found, for example, in nonstopfor piano (1960) by Boguslaw Shaeffer.

Score-BPF is a specialized form of a group expres-sion. It is a graphical object that can represent anynumber of break-point functions as a part of a musi-cal texture. Score-BPFs can be used to describe pitchcontours, tempo functions, envelopes, dynamiccurves, etc. Potentially, the same mechanism couldbe used to represent samples, sonograms, and evenspatial paths. Score-BPF can be edited directly in thescore. It also allows the contained break-point func-tions to be modified and generated algorithmically.

Figure 8 shows an application of Score-BPF. Here,control information specialized to a virtual Guqininstrument model has been drawn in the score withthe help of several Score-BPFs. The example alsocontains special tabulature markings (below thestaff) that are used to indicate the string numbersone to seven. (The Guqin is a traditional Chinese

instrument that has seven strings.) These charactersare based on a new computer-based coding systemdeveloped by Henbing Li (2005). This coding sys-tem, in turn, is based on an ancient Chinese tabula-ture notation called Jian Zi Pu (JZP-notation;Changlin 1989).

ENP Expression Designer

Modern music in particular often requires specialexpression markings. Many historical playing tech-niques exist, and new ones are invented all thetime. For example, Karkoschka (1966) and Risatti(1973) alone list thousands of different modern no-tational devices. Against this background, it be-comes evident that it is an impossible task for amusic notation program to make available a com-prehensive collection of expression markings. It ismore reasonable to provide an interface that lets theuser build custom expressions.

Two of the most widely used commercial nota-tion programs, Finale and Sibelius, provide the pos-sibility to create user-defined expressions. In CMN,it is also possible to define new expressions usingLisp (here, user-definable graphical entities arecalled sundries). Furthermore, LilyPond includes a\markup facility—a typesetting mini language—and it also allows users to define their own expres-sions in C++. However, when using these tools, it isnot possible to view and design the expressions inthe context of music notation, i.e., to see—duringthe design process—how the new expression is go-ing to appear in the score.

ENP Expression Designer (ED) is a visual tool thatcan be used to create custom expressions in ENP.ED provides a protocol that helps users build new

71

Figure 5. A part contain-ing four voices that aredistributed in three differ-ent staves (Max Reger,Phantasie und Fuge über

B-A-C-H, Op. 46, copyrightUniversal Edition A.G.,Wien / UE 1222, www.universaledition.com).

Figure 6. (a) Score contain-ing a collection of ENP-Expressions, e.g., (1)Pitch-Class Clock, (2)Marker, (3) Score-BPF(breakpoint function), and(4) Group; (b) the same

material transposed a mi-nor sixth higher. The ENP-Expressions automaticallyadjust their position andthe displayed informationaccordingly.

(a) (b)

expressions. This approach enables the following:(1) expressions can be made dynamic, because theyare defined using Lisp code; (2) expressions are dis-played in a fully functional score that allows theuser to observe how the expressions behave, e.g.,when the music is transposed (an approach differentfrom any existing one); and (3) it is possible to com-bine both algorithmic (code-based) and visual(graphical) approaches.

ED is part of an increasing collection of visualtools inside ENP. Currently, these tools also includeColor Editor, Expression Tool Editor, and NoteHead Designer. The purpose of these tools is to pro-vide a visual interface for constructing complexmusic-notation-related Lisp objects and to provideassistance to those users who do not possess theneeded programming skills to use the strictly text-based approach. The visual approach, however, isnot for inexperienced users only. Some objects maybe realized more easily by using a purely graphicalapproach and others, in turn, by using a program-ming language.

Uniform design principles have been applied toall the aforementioned tools. The tools share thesame kinds of editors, components, and functional-ity, which makes them easier to learn. Most editorsalso contain an interactive view providing visuallysynchronized feedback.

The use of visual editors in our case has severaladvantages: (1) the underlying Lisp syntax can behidden to allow the user to concentrate on the taskat hand; (2) the system automatically handles suchtasks as class and method definitions; and (3) thisapproach is less prone to errors, because most of theunderlying code is generated automatically.

Figure 9 shows an example of a typical ED ses-sion. Here, we have used the code-based approach todefine an expression. (See the view labeled “Code

view.”) The expression is constructed out of sixBézier curves that form an opening fan that fills thehorizontal space reserved for the note.

ENP-Score-Notation

Many different text-based formats have been devel-oped for representing the information in a musicalscore. Some typical examples are CMN, GUIDO,MusiXTEX (Taupin, Mitchell, and Egler 2005),GNU LilyPond, and MusicXML (Good and Actor2003). Some Lisp-based representations are also ableto preserve only a subset of musical information,such as the rhythm; these include the notation for-mats of both PatchWork and OpenMusic. The ad-vantages in text-based representation of a musicalscore are that the layout of notational objects is sep-arated from the representation, and that the textualformats are human-readable. In our case, the textualformat also provides a way to generate and manipu-late rich musical data in PWGL.

ENP allows users to describe scores in a specialtext-based format called ENP-score-notation (Laur-son and Kuuskankare 2003). It is offered as an inter-mediate step between the score and the low-levelfile format. By using ENP-score-notation, a scorecan be converted into a more readable form that canalso easily be converted back to an ENP score. Thiskind of symmetry is beneficial, as it allows users toview and modify score information without losinginformation.

The structure of ENP-score-notation reflects thescore-hierarchical structure of the score, and thesyntax is similar to Lisp list syntax. Every level inthe hierarchy is collected into a list. ENP-score-notation offers a large subset of notational attri-butes. Through keyword arguments, the systemalso allows users to add new features easily.

Next, we give an example of ENP-score-notationand the resulting ENP score (see Figure 10). We havechosen here a slightly unorthodox visualizationscheme to be able to align the corresponding ele-ments of the two representations. The score is dis-played on the right, rotated 90° clockwise, and theENP-score-notation code is shown in the middle.On the left is a rhythmic reduction, resulting in


Figure 7. (a) Canvas-Expression shown here isin an editable state, show-ing the collection of avail-able tools; (b) a graphicalscore, in the style of Bogus-law Shaeffer, created usingCanvas-Expression.

(a) (b)

Kuuskankare and Laurson 73

Figure 8. Score-BPF expres-sion provides the possibil-ity to draw piecewiselinear functions as a partof a musical texture. Inthis example several Score-

BPFs are used to describepitch glides for a Chinesestring instrument calledGuqin. Note also the Chi-nese tabulature markingsbelow the staff.

Figure 9. ED is used to de-fine an expression (seeCode view) that representsa fanning glissando inwhich six different instru-

ments start from unisonand make gestures that ap-proximately follow theshape of the expression.

Figure 8

Figure 9

the unit-lengths of the main-beats. This is alsoaligned with the corresponding elements in the tex-tual notation.

Figure 11 attempts to illustrate two importantconcepts concerning ENP-score-notation: defaultsand optional attributes. Here, an ENP-score-notation fragment taken from Figure 10 (lines 3–10)is divided into three columns. The basic idea behindthis visualization is that, by starting from therhythmic skeleton shown in the leftmost column,the score can be described in more detail by addingthe information found in the two succeeding col-umns. The result of the rhythmic skeleton alone(specifying only rhythmic elements, including po-tential rests and ties) is shown in the bottom of thefirst column.

As can be seen, the score already contains in-formation about pitch, tempo, and other attri-butes that are not indicated in the textualdescription. ENP automatically substitutes miss-ing properties with default values (e.g., the pitch is


Figure 10. ENP-score-notation example(middle); the correspon-ding score (right); therhythmic reduction (left).The structural elements inthe ENP-score-notation are

aligned with and con-nected to the matchingsections in the musicalscore. (The alto saxophoneis notated here at soundingpitch.)

Figure 11. A fragment ofENP-score-notation is bro-ken into three differentsegments. The aggregateeffect of adding optionalattributes is shown in the

music notation below. (a)Rhythmic skeleton; (b)pitch information; and (c)optional attributes, suchas expressions and enhar-monic spelling.


middle C, the clef is a treble clef, and the tempo isM. M. = 60). Furthermore, some properties (such astime signatures) are always calculated automati-cally by ENP.

The second column (b) adds pitch information tothe rhythmic skeleton. The more detailed result canbe seen again in the bottom of the column. For ex-ample, there is now a note with an alteration (sharp)as ENP spells all altered notes as sharps unless spec-ified otherwise.

Finally, in the last column (c), the information inthe score is completed with some additional attri-butes and ENP-Expressions. The score fragment inthe last column shows the aggregate of all three col-umns. Here, the :enharmonic attribute is used toindicate that the associated note is written using itslowered enharmonic equivalent. The result can beseen in the corresponding score fragment: The A-sharp found in the previous column has beenchanged to B-flat.

ENP-Script

Most professional music notation programs providea scripting language or plug-in interface that allowsthe user to modify notational information con-tained in the score. In a music notation program, ascript can typically be used to apply a certain articu-lation pattern to a passage of music or to recalculatethe enharmonic identity of selected notes. Sibelius,for example, provides a built-in scripting languagecalled ManuScript. Finale, on the other hand, pro-vides an interface that allows the user to write plug-ins using C++. It is also possible to use the scriptinglanguage called FinaleScript. LilyPond provides aplug-in functionality through its built-in Schemeinterpreter.

ENP-Script (Kuuskankare and Laurson 2004a) is ascripting language that uses a pattern-matchingsyntax identical to the one used by PWGLCon-straints. The use of the pattern-matching languageas the basis of our scripting language offers severaladvantages: (1) complex musical patterns can easilybe defined; (2) the syntax is compact, powerful, andeasy to learn; (3) typically there is no need to writeany control structures (e.g., loops) owing to the ba-

sic ability of the system to map through the nota-tional objects in the score; and (4) recently intro-duced enhancements to the pattern-matchingsyntax (Laurson and Kuuskankare 2005) allowENP-Script to map through different types of ob-jects in the score in addition to notes (e.g., mea-sures, beats, chords, and harmonies). Furthermore,ENP-Script is intended to be open, allowing full ac-cess to the notational structures of the score.Scripts can access and manipulate a rich set ofmelodic, rhythmic, and harmonic informationfound in the score.

Next, we give two examples of how ENP-Scriptcan be used in a musical context. The completesyntax of the language is not presented here; in-stead, the basic concepts are covered to make theincluded scripting examples more comprehensible.The syntax of the pattern-matching language iscovered in detail in Laurson (1996) and Laurson andKuuskankare (2005).

In the first example, we use an excerpt taken fromArnold Schönberg’s Suite, Op. 25, as a startingpoint. We begin with a score that contains onlypitch and rhythm information. The aim is to fillthis excerpt with some of the articulations that arefound in the original score with the help of ENP-Script. To make the example more straightforward,we exclude all articulations other than grave ac-cents and slurs. We attempt to determine the typeand position of the inserted articulations by access-ing the notational information contained in the mu-sical objects. This is done according to the followingtwo features: (1) the size of the melodic movement;and (2) the beaming level. The original score fea-tures a slur between two notes if they form an inter-val larger than an octave and if there is not a restbetween them. Also, each sixteenth note has agrave accent.

Without going into details, we give the definitionof an ENP-Script consisting of two scripting rules.Figure 12a shows the first scripting rule that trav-erses all adjacent melodic note pairs in the scoreand determines if a slur should be inserted accord-ing to the user-defined conditions. Figure 12b, inturn, gives the second scripting rule that handlesgrave accents. In both cases, the lines starting withadd-expression insert the appropriate expression

75

three different shapes: circle, rectangle, and triangle.)These shapes are used to distinguish between con-secutive symmetric harmonies that potentiallyshare notes. Thus, for example, the third harmony(marked with triangles) shares two notes (found inthe middle voices) with the next harmony that, inturn, is marked with rectangles.

Future Work

ENP is a work in progress, and several features arestill missing or incomplete. There is currently novisual synchronization between the mensural andnon-mensural notation. This means that, for ex-ample, events in two parts written in mensural andnon-mensural styles do not coincide even if theyhave exactly the same start-time. Also, the page-layout capabilities of ENP are currently quite primi-tive. The next project is to rewrite the notationengine to support synchronization between the no-tational styles and to implement an intelligentpage-layout scheme.

ENP allows users to export several graphics andmusic file formats, including EPS, PS, MIDI, ETF,LilyPond, and MusicXML. LilyPond and MusicXMLformats are currently implemented through FO-MUS (Psenicka in press) by David Psenicka. How-ever, there are no supported import formats at themoment; here, MIDI is of primary importance. Cur-

into the score. Figure 13 shows the score after thescript has been applied.

Next, we show how to automatically analyze andannotate some harmonic and melodic properties ofa choral piece, in this case Erotessa by the Finnishcomposer Kimmo Kuitunen, who writes:

The cycles of three four-voiced symmetricchords and their inversions (the consecutiveset-class identities are 4–7, 4–7 and 4–9 respec-tively) create the harmonic basis. The pitchclasses remain the same within each four repe-titions of the cycle, but the octave positionschange. These three chords (F–A–E–A-flat, D–B-flat–E-flat–B, and D-flat–G-flat–G–C) alsoform a matrix that fills the chroma (set class12-1). The melodic line of each vocal part re-peats the three-note cells derived from the ma-trix. (Set classes are either 3-3a or 3-3b). Eachvoice also uses all twelve pitch classes, thusfilling the chroma. (Kuitunen 2006)

This kind of analysis requires a high level of ex-pertise. It is also very time-consuming and prone toerrors. Here, instead of doing the analysis by hand,we use ENP-Script and a set of specialized graphicaldevices to visualize the analysis information di-rectly in the score. (See the result in Figure 15.) Weconcentrate on two aspects of the analysis givenabove. Figure 14a shows the scripting-rule to locateand visualize the symmetric harmonies, and Figure14b analyzes the set-class identity of consecutivethree-note cells. The first of these scripting-rules isseemingly more complex than the ones in the previ-ous case, owing to the visual appearance of the anal-ysis information that is calculated inside thescripting-rule. (The appearance is cycled through


Figure 12. An ENP-Script,consisting of two scriptingrules, is used to insert ar-ticulations into a score ac-cording to (a) pitch and (b)

beaming information con-tained by the notationalobjects. (See the commentson the right followingsemicolons.)

Figure 13. Slurs and graveaccents found in this ex-ample are inserted by anENP-Script. The type andplacement of the expres-sions has been determinedaccording to the beaminginformation and the size ofthe melodic interval. The

example contains only asubset of the articulationsfound in the original score(Arnold Schönberg, Suite,Op. 25, copyright 1925 byUniversal Edition A.G.,Wien/UE 7627, www.universaledition.com).

Changlin, C. 1989. “Preliminary Application of the Com-puter Technique to GuQin Music Research.” Journal ofComputer Science and Technology 4(1):86–96.

Dannenberg, R. 1993. “Music Representation Issues.”Computer Music Journal 17(3):20–30.

Dannenberg, R. 2000. “Allegro Music RepresentationLanguage.” Available online at www.cs.cmu.edu/~rbd/software/allegro.htm/ (accessed May 25, 2006).

Diener, G. R. 1990. “Modeling Music Notation: A Three-Dimensional Approach.” Ph.D. Thesis, Stanford Uni-versity.

Fober, D., S. Letz, and Y. Orlarey. 2004. “Open-Source Toolsfor Music Representation and Notation.” Paper presentedat Sound and Music Computing ’04, Paris, 21 October.

Good, M., and G. Actor. 2003. “Using MusicXML for FileInterchange.” Third International Conference on WEB

rently, we are considering Allegro (Dannenberg2000), which is a text-based representation that, inconjunction with a converter tool (allegroconvert)could potentially be used to both export and importMIDI files. There exists also a library, LibMu-sicXML (Fober, Letz, and Orlarey 2004), that is li-censed under the GNU Lesser General PublicLicense (LGPL) and suitable for both exporting andimporting MusicXML format. (Incidentally, itwould also support Guido.)

Acknowledgment

The work of Mikael Laurson has been supported bythe Academy of Finland (SA 105557).

References

Assayag, G., et al. 1999. “Computer-Assisted Composi-tion at IRCAM: From PatchWork to OpenMusic.”Computer Music Journal 23(3):59–72.

Bays, G. 2005. “ScoreSVG: A New Software Frameworkfor Capturing the Semantic Meaning and GraphicalRepresentation of Musical Scores Using Java2D, XML,and SVG.” Master’s Thesis, College of Arts and Sci-ences, Georgia State University.

Kuuskankare and Laurson 77

Figure 14. An ENP-Scriptwith two scripting rules isused to insert (a) verticaland (b) horizontal analysisinformation into a score.

Figure 15. All symmetricharmonies are displayedautomatically in the scorewith the help of connectedshapes. In addition, theset-class content (3-3a or 3-3b) of consecutive three-

note cells is shown beloweach part (KimmoKuitunen, Erotessa, withlyrics by Eino Leino, formixed choir, 2001, repro-duced by kind permissionof the composer).

Delivering of Music. New York: Institute of Electricalland Electronics Engineers, p. 153.

Honing, H. 1993. “Issues in the Representation of Timeand Structure in Music.” Contemporary Music Review(9):221–239.

Hoos, H. H., et al. 1998. “The GUIDO Music NotationFormat—A Novel Approach for Adequately Represent-ing Score-level Music.” Proceedings of the 1998 Inter-national Computer Music Conference. San Francisco,California: International Computer Music Association,pp. 451–454.

Huron, D. 1992. “Design Principles in Computer-BasedMusic Representations.” In A. Marsden and A. Pople,eds. Computer Representations and Models in Music.London: Academic Press.

Karkoschka, E. 1966. Das Schriftbild der Neuen Musik.Celle: Hermann Moeck Verlag.

Kornfeld, W. 1980. “Machine Tongues VII: LISP.” Com-puter Music Journal 4(2):6–12.

Kuitunen, K. 2006. E-mail correspondence with the au-thor, 14 June 2006.

Kuuskankare, M., and M. Laurson. 2002. “ENP2.0: A Mu-sic Notation Program Implemented in Common Lispand OpenGL.” Proceedings of the 2002 InternationalComputer Music Conference. San Francisco, Califor-nia: International Computer Music Association,pp. 463–466.

Kuuskankare, M., and M. Laurson. 2003. “ENP-Expressions, Score-BPF as a Case Study.” Proceedingsof the 2003 International Computer Music Conference.San Francisco, California: International Computer Mu-sic Association, pp. 103–106.

Kuuskankare, M., and M. Laurson. 2004a. “IntelligentScripting in ENP using PWConstraints.” Proceedings ofthe 2004 International Computer Music Conference.San Francisco, California: International Computer Mu-sic Association, pp. 684–687.

Kuuskankare, M., and M. Laurson. 2004b. “Recent Devel-opments in ENP-score-notation.” Paper presented atSound and Music Computing ’04, Paris, 21 October.

Kuuskankare, M., and M. Laurson. 2005. “ENP Expres-sion Designer: A Visual Tool for Creating User-Definable Expressions.” Proceedings of the 2005International Computer Music Conference. San Fran-cisco, California: International Computer Music Asso-ciation, pp. 307–310

Lassfolk, K. 2004. “Music Notation as Objects.” Ph.D.Thesis, University of Helsinki.

Laurson, M. 1996. “PATCHWORK: A Visual Program-ming Language and Some Musical Applications.” Ph.D.Thesis, Sibelius Academy, Helsinki.

Laurson, M., and M. Kuuskankare. 2002. “PWGL: ANovel Visual Language based on Common Lisp, CLOSand OpenGL.” Proceedings of the 2002 InternationalComputer Music Conference. San Francisco, Califor-nia: International Computer Music Association,pp. 142–145.

Laurson, M., and M. Kuuskankare. 2003. “From RTM-notation to ENP-score-notation.” Paper presented atJournées d’Informatique Musicale 2003, Montbéliard,France, 4 June.

Laurson, M., and M. Kuuskankare. 2005. “ExtensibleConstraint Syntax Through Score Accessors.” Paperpresented at Journées d’Informatique Musicale 2005,Paris, 2 June.

Laurson M. , V. Norilo, and M. Kuuskankare. 2005.“PWGLSynth: A Visual Synthesis Language for VirtualInstrument Design and Control.” Computer MusicJournal 29(3):29–41.

Li, H. 2005. “The Encoding of Guqin Jian Zi Pu Tablaturefor Computer Input.” Unpublished manuscript, Insti-tute for Psychoacoustics and Electronic Music, Depart-ment of Musicology, Ghent University, Ghent, Belgium.

Nienhuys, H.-W., and J. Nieuwenhuizen. 2003. “Lily-Pond: A System for Automated Music Engraving.” XIVColloquium on Musical Informatics. Firenze, Italy:Centro Tempo Reale.

Psenicka, D. (in press). “FOMUS, a Music Notation Pack-age for Computer Music Composers.” Proceedings ofthe International Computer Music Conference.

Read, G. 1982. Music Notation: A Manual of ModernPractice. New York: Taplinger.

Renz, K. 2002. “Algorithms and Data Structures for a Mu-sic Notation System based on GUIDO Music Nota-tion.” Ph.D. Thesis, Technischen UniversitätDarmstadt.

Risatti, H. A., 1973. New Music Vocabulary: A Guide toNotational Signs for Contemporary Music. Champaign-Urbana: University of Illinois Press.

Roads, C. 1996. The Computer Music Tutorial. Cam-bridge, Massachusetts: MIT Press.

Schottstaedt, B. 1997. “Common Music Notation.” In E.Selfridge-Field, ed. Beyond MIDI: The Handbook ofMusical Codes. Cambridge, Massachusetts: MIT Press.

Selfridge-Field, E., ed. 1997. Beyond MIDI: The Hand-book of Musical Codes. Cambridge, Massachusetts:MIT Press.

Stone, K. 1980. Music Notation in the Twentieth Cen-tury: A Practical Guidebook. New York: Norton.

Taupin, D., R. Mitchell, and A. Egler. 2005. MusiXTex.Available online at directory.fsf.org/MusiXTex.html/(accessed October 19, 2005).



Appendix

A page of an orchestral score created with ENP (El-liot Carter, A Mirror on Which to Dwell, 1975) isshown here as Figure 16.

79

Figure 16. Typeset with theENP software, this scoreexcerpt shows measures 1–2 of Elliott Carter’s A Mir-ror on Which to Dwell forsoprano and chamber or-chestra, a setting of poems

by Elizabeth Bishop. Copy-right 1976 (renewed) byAssociated Music Publish-ers, Inc. (BMI). Interna-tional copyright secured.All rights reserved.Reprinted by permission.

Publication [P2]

[P2] M. Kuuskankare and M. Laurson. Annotating Musical Scores in ENP. In Inter-

national Symposium on Music Information Retrieval, London, UK, 2005.

91

ANNOTATING MUSICAL SCORES IN ENP

Mika KuuskankareDepartment of Doctoral Studies

in Musical Performance and ResearchSibelius Academy

[email protected]

Mikael LaursonCentre for Music and Technology

Sibelius AcademyFinland

[email protected]

ABSTRACT

The focus of this paper is on ENP-expressions that can beused for annotating ENP scores with user definable infor-mation. ENP is a music notation program written in Lispand CLOS with a special focus on compositional and mu-sic analytical applications. We present number of built-inexpressions suitable for visualizing, for example, musicanalytical information as a part of music notation. A Lispand CLOS based system for creating user-definable anno-tation information is also presented along with some sam-ple algorithms. Finally, our system for automatically ana-lyzing and annotating an ENP score is illustrated throughseveral examples including some dealing with music in-formation retrieval.

Keywords: Music representation, annotating, symbolicnotation.

1 OVERVIEW

Expressive Notation Package (ENP, [1, 2]) is a music no-tation program that is designed for displaying scores usingthe common Western music notation. ENP has been usedin several research projects ranging from computer aidedcomposition to controlling virtual instruments. A spe-cial focus has been given to compositional and computer-assisted music analysis applications.

ENP has a graphical user interface that allows musi-cal objects to be edited directly with the mouse. It sup-ports two fundamental notational styles, i.e., mensural andnon-mensural notation, and a number of special notationalstyles such as time notation, frame notation, etc.

Representation of musical units must offer ways ofmaking annotations, giving names, making comments,adding images or diagrams, providing links to informa-tive resources on the web, etc [3]. ENP provides a col-lection of standard and non-standard notational attributes

Permission to make digital or hard copies of all or part of thiswork for personal or classroom use is granted without fee pro-vided that copies are not made or distributed for profit or com-mercial advantage and that copies bear this notice and the fullcitation on the first page.

c©2005 Queen Mary, University of London

(e.g, articulations) called ENP-expressions. Furthermore,it offers a set of attributes that can be used to representanalytical information or other user-defined annotationsas a part of a musical texture. In addition to their tra-ditional use, ENP-expressions can be used in wide rangeof applications: (1) display music theoretical analysis in-formation, e.g, annotate motives, harmonic progressions,etc; (2) visualize specialized analytical information, suchas Schenker graphs, or pitch-class set theoretical informa-tion; (3) attach arbitrary textual annotations, names, orcomments to objects; (4) dynamically inspect and visu-alize data contained by notational objects, i.e., duration,velocity, start-time, etc.; (5) add instructions to tutorials,documentation, or presentations, etc.

All ENP-Expressions can access the data containedby the notational objects they are associated with. Thisallows to design dynamic expressions that can automat-ically display relevant information about themselves andtheir musical context. It is also possible to use a scriptinglanguage called ENP Script [4] as an algorithmic comple-ment to the manual approach where the user inserts ENP-expressions by hand. This is useful when building, for ex-ample, computer-assisted music analysis applications orautomatically annotating a musical score.

In the following, we present and discuss the annota-tion possibilities of ENP in more detail. We start with abrief introduction of ENP-Expressions and the notationaloutput of ENP. Section 3 is the main part of this study.Here we present a wide range of annotation devices alongwith some example scores. Section 4 gives a brief look atthe possibilities of automatic music information retrievaland annotation. Section 5 presents some conclusions andideas for future work.

2 ENP-EXPRESSIONS IN BRIEF

Every expression is attached to some musical object or toa group of objects. Currently these objects can be eithernotes or chords (see [5] for a description of the object hier-archy in ENP). All the expressions are aware of their mu-sical context and can automatically adjust their positionand graphical appearance accordingly. New expressionscan be created through a textual interface using Lisp andCLOS or by using a set of specialized editors inside ENP.

Figure 1 gives a concise overview of the notationalpossibilities of ENP in a modern context. The example is

written in non-mensural notation (time notation) and con-tains various expressions, special note heads and playingstyles.

Figure 1: An example of the notational output of ENP.

3 ANNOTATIONS OVERVIEW

3.1 Groups

Groupsare multi-purpose expressions that can be used tomark continuous passages in music. The visual appear-ance of the groups can be adjusted to suit the current ap-plication. Groups can also contain user definable textualinformation. The next example (Figure 2) shows one usefor groups to annotate motives in a score. The groups, inthis case, are represented as brackets with labels.

Figure 2: Motivic analysis information inserted in thescore with the help of groups (J.S. Bach: Invention no.1).

3.2 Text Expressions

Textual information can be attached to both chords andnotes. The expressions can be either static or dynamic.These expressions typically contain some written instruc-tions or annotations made by the user. Dynamic ones, inturn, can adjust the visual appearance and printed infor-mation by analyzing their musical context.

3.2.1 Static Text Expressions

Next, we give two examples of static text expressions. InFigure 3 the user has inserted text expressions to the notesand entered the note names accordingly. This kind of be-havior is acceptable when the data contained by the notesis not expected to change (e.g., through transposing). Inthis case the expressions show their association with thenotes by drawing a dotted line connecting the expressionto the corresponding note.

Figure 3: Some static text expressions inserted in a score.The association between the expression and the notationalobject is displayed by a dotted line

Another example of a static expression is thewindowexpression. This device is useful for giving focus to anabject at any position on the screen. These expressionsapply only to the current window so they are suitable forinserting markings only to one static page at a time, as incase of tutorials or presentations.

Figure 4: Window expressions can be used to mark anyposition in the score.

3.2.2 Dynamic Text Expressions

Dynamic expressions can adjust their visual representa-tion (e.g, position, shape, color, text) automatically. Thisensures that the score is always up-to-date. This is alsouseful when dealing with analytical problems: the analy-sis information remains correct even if the music is edited.For this purpose we introduce a special expression calledannotation. Dynamic expressions are also useful to dis-play relevant information accessed from the objects theyare attached to. Almost any attribute of the notational ob-jects can be made visible in this way. The attribute can beselected from an automatically generated list that is spe-cialized for both chords and notes. The current set in-cludes, for example,midi, start-time, diatone, duration,velocity, channel, etc.

Figure 5 gives one example of the behavior of dynamicannotation. The data displayed is read directly from theobjects. In this case the data shown is the value found inthemidi slot of the note.

Figure 5: Dynamic expressions can adjust their visual ap-pearance automatically.

The user can also write additional methods for dis-playing the result of some user-definable code with thehelp of annotation expressions. ENP provides anadd-to-annotation-librarymacro that can be used to add user de-finable annotations to a special library. These annotationsare immediately usable and can be inserted into the score

through context sensitive menus. The parameters of add-to-annotation-library are: (1) the type of the object the ex-pression is associated with, (2) the name of the expression,and (3) the function that constructs the displayed data.

Below we give two relatively simple examples of dy-namic text annotations created with the help of the add-to-annotation-library macro.

First, we define a new annotation that can be attachedto notes (1). In (2) we define the name of the annotation(note-name) and in (3) we give the function that constructsthe textual information depending on the midi value of thenote. Figure 6 gives the corresponding score.

(add-to-annotation-library:note ;(1)note-name ;(2)#’(lambda(note) ;(3)

(format () "note name: ˜a"(midi-to-note-name (midi note)))))

Figure 6: A dynamic text annotation, displaying the notename, created with the help of the add-to-annotation-library macro.

As can be seen in Figure 7, the needed user-interfacecomponents (context sensitive menus) are also added au-tomatically. The annotations can be inserted in the scoreas any built-in ENP-expression (see the ”User methods”menu in Figure 7). The two menus marked as ”Methods”and ”Slots” contain a list of available slots and predefinedmethods applicable to the object in question.

Figure 7: Menus for user definable annotations are addedautomatically by the system.

The second example deals with displaying some pitch-class set theoretical analysis information. Here we use adynamic text expression to mark the pitch-classes abovethe individual notes in a twelve-tone row. In Figure 8 wehave two twelve-tone rows. The former one is the rowused in Alban Berg’s Lyrische Suite in original form, andthe latter one, in turn, is its transposed version.

The expression definition is given below. There areonly few changes when compared to the previous exam-ple. The most important one is that in this case we takethe modulo 12 of the midi value contained by the note, ascan be seen in (1).

(add-to-annotation-library:notepc-name#’(lambda(note)

(format () "˜a"(mod (midi note) 12)))) ;(1)

Figure 8: The twelve-tone row of Alban Berg’s LyrischeSuite (above) and its transposed version (below). Note,that the pitch classes shown by the dynamic annotations(pc-name) are displayed correctly in the transposed rowform.

3.3 Score Expressions

Score-expressionscan be used to visualize discontinuousinformation in the score (Figure 9). This is a convenientway to display information that is scattered across differ-ent parts. Arbitrary vertical relations can be made visiblefor analytical purposes. The shape of the score expressioncan also be selected. The next example illustrates the useof the box-shaped score-expression to draw a focus aroundtwo chords in the score. This kind of an expression is use-ful for revealing groups containing several entries.

Figure 9: Focus drawn around two chords using abox-shaped score-expression (Arnold Schoenberg: SechsKleine Klavierstucke, op. 19).

A line-shaped score-expression is best suited to indi-cate a relation between two notational objects as can beseen in the Figure 10. Here some music analytic informa-tion (cross relation) is inserted in the score. In addition tothe line connecting the two notational objects the expres-sion can contain some user definable text.

Figure 10: Voice-leading analysis information displayedusing the line-shaped score-expression.

3.4 Pitch-class Clock Face

The Pitch-class clock is a highly specialized expressionthat can be used to visualize set theoretical information asa part of the musical texture. It is a dynamic graphical ob-ject that displays the pitch-class set of a selected chord asa pitch-class clock face. The pitch classes that are presentin the set are highlighted (Figure 11).

Figure 11: A pitch-class clock face is a specialized expres-sion for visualizing pitch-class set theoretical information.

3.5 Highlighting

Highlighting is a visualization device that can be usedfor demonstration purposes. Relevant information in thescore can be highlighted to focus the attention to a specificdetail in the score (Figure 12). The areas outside the high-lighted area are dimmed but still clearly visible, thus pre-serving the view to the whole musical context at the sametime. The highlighted area can be moved and re-sized bythe user.

Figure 12: A highlighted area created in the bassoon partof Rite of Spring by Igor Stravinsky.

4 AUTOMATICALLY ANNOTATING ASCORE

In this section we briefly describe how to automaticallyanalyze and annotate an ENP score by using a built-inscripting language, ENP script. A script is usually used toautomate complex or advanced tasks within the program.In a music notation program it is useful to allow frequentlyrepeated operations (e.g., applying an articulation pattern)to be automated by using scripts.

The scripting examples in this section are explainedonly briefly. It is not our intention to explain the syntax ofENP Script but to illustrate the potential of the system ingeneral.

In our first example we examine the vertical aspect ofa piece of music by performing a simple harmonic analy-sis.1 The result of the analysis, the name of the pitch-classset of each prevailing harmony, is printed on the bottomof the score. The naming of the pitch-class sets followsthe conventions introduced by Allen Forte [6]. Figure13 gives a score with the set theoretical analysis informa-tion. The script used to generate the information is shownabove the score. This kind of use of a script resembles thecontainer-iterator idea found, for example, in CPNView[7, 8]. Here we iterate through every harmonic situation

1Here we consider the term harmony to be a result of one,two or more notes being sounded simultaneously. Usually thesmallest possible harmony contains at least two notes. For sim-plicity, monophonic situations are regarded as special cases.

in the score (see the :harmony keyword in (1) in the ex-ample script). A special Lisp functionadd-expression(2)is used to insert into the score a group expression display-ing the analysis information. The expression is attachedto the lowest note of the chord (3). In (4) the built-in set-theoretical functionsc-nameis used to identify the pitch-class set name of the current harmony.

As can be seen in Figure 13 there are three pitch-classset names below the first note in the second measure (E[).This is due to the fact that there are three different har-monies that include the note in question. The set-classnames are, from the bottom up: (1) 1-1, which is the re-sult of the E[ sounding alone, (2) 2-5, which is the resultof the E[ and two-line B[ in the right hand staff soundingtogether, and (3) 2-6, resulting from the low E[ and theone-line A in the right hand.( * ?1 :harmony ;(1)

(?if(when (complete-chord? ?csv)

(add-expression ;(2)’group (give-bass-item ?csv) ;(3):info (sc-name (m ?1)))))) ;(4)

Figure 13: Vertical pitch-class set theoretical analysis in-formation displayed in the score with the help of ENPScript and groups (Arnold Schoenberg: Suite fur Klavier,op. 25).

The second example deals with horizontal aspects ofmusic. The script in this case is used to calculate and dis-play the interval chain of a melodic line. The script isgiven below and the resulting score can bee seen in Figure14. In this case we are interested in the relation betweentwo consecutive notes (see (1) in the script). Ananalysis-text-groupexpression is used in this case (2). It is posi-tioned above the associated objects (see Figure 14). Theexpression is also displaced horizontally (3) so that it ispositioned between the notes. The printed information iscalculated by subtracting two consecutive pitches (4).( * ?1 ?2 ;(1)

(?if(add-expression ’analysis-text-group ?1 ;(2)

:user-position-X-correction ;(3)(- (/ (width ?1) 2.0) 1.0):info (- (m ?2) (m ?1))))) ;(4)

Figure 14: The interval chain of the melodic line dis-played above the staff (Anton Webern: Zwei Lieder, op.8).

Finally, we give an example dealing with n-grams. N-grams have been widely used in text retrieval, where the

sequence of symbols is divided into overlapping constant-length sub-sequences [9]. The use of n-grams in MIR hasalso been considered by several authors (see for example[10]).

Let us examine the script in more detail. It is dividedinto two parts. The first part (A) calculates di-grams andthe second part (B) calculates tri-grams. The Lisp func-tion, add-expression, is again used to insert the analysisinformation into the notes (2). This time the informationis attached to multiple notes (see the di-gram informationand the corresponding brackets in Figure 15). Also a moresubtle bracket style is utilized (3). In (4) and (5) the inter-val information is calculated and formatted (the specificsof the Lisp’s format function are not in the scope of thispaper. See [11] for detailed information). The second partof the script (B) is otherwise identical, when compared tothe first one, except that we now consider four consecutivenotes (6) forming three intervals. The printed informationis again calculated in (7-9).A.

( * ?1 ?2 ?3 ;(1)(?if

(add-expression ’group ?1 ?2 ?3 ;(2):kind :bracket-at-end ;(3):info (format () "˜3,@d |˜3,@d" ;(4)

(- (m ?2) (m ?1)) ;(5)(- (m ?3) (m ?2))))))

B.

( * ?1 ?2 ?3 ?4 ;(6)(?if

(add-expression ’group ?1 ?2 ?3 ?4:kind :bracket-at-end:info (format () "˜3,@d |˜3,@d |˜3,@d"

(- (m ?2) (m ?1)) ;(7)(- (m ?3) (m ?2)) ;(8)(- (m ?4) (m ?3)))))) ;(9)

Figure 15: Di- and tri-grams displayed as a part of a mu-sical texture (J.S. Bach: Musikalisches Opfer).

5 CONCLUSIONS

A comprehensive set of different annotation devices inENP were discussed. Also a scheme to automatically an-alyze a score and to visualize analytical data was intro-duced.

Currently the ENP-expressions can be attached only tonotes or chords. The present scheme should be extendedto apply all the objects in the score. This way it wouldbe possible to attach annotations to parts, voices, mea-sures, bar lines, clefs, etc. It is possible, however, to insertcomments to measures, for example, but these annotationshave different status than the regular ENP-expressions. Byaddressing this problem it would also make the ENP ob-ject representation scheme more unified and coherent.

Displaying MIR results in a score is suitable only forcertain types of data. To bring ENP closer to practicalMIR applications it would be beneficial to integrate a sta-tistical package, such as R [12], into the environment (R isa language and environment for statistical computing andgraphics). This would allow to represent, for example,pitch histograms and other similar information retrievedfrom an ENP score.

ACKNOWLEDGEMENTS

The work of Mikael Laurson has been supported by theAcademy of Finland (SA 105557).

REFERENCES

[1] Mika Kuuskankare and Mikael Laurson. ENP2.0A Music Notation Program Implemented in Com-mon Lisp and OpenGL. InProceedings of Interna-tional Computer Music Conference, pages 463–466,Gothenburg, Sweden, September 2002.

[2] Mika Kuuskankare and Mikael Laurson. ExpressiveNotation Package - an Overview. InInternationalSymposium on Music Information Retrieval, 2004.

[3] Tillman Weyde. Case Study: Representation of Mu-sical Structure for Music Software.

[4] Mika Kuuskankare and Mikael Laurson. IntelligentScripting in ENP using PWConstraints. InProceed-ings of International Computer Music Conference,pages 684–687, 2004.

[5] Mika Kuuskankare and Mikael Laurson. Recent De-velopments in ENP-score-notation. InSound andMusic Computing ’04, October 2004.

[6] Allen Forte. The Structure of Atonal Music.Journalof Music Theory, 1973.

[7] DonnchaO Maidın. Common Practice NotationView: a Score Representation for the Constructionof Algorithms. Proceedings of International Com-puter Music Conference, pages 248–251, 1999.

[8] DonnchaO Maidın and Margaret Cahill. Score Pro-cessing for MIR.International Symposium on MusicInformation Retrieval, pages 59–64, 2001.

[9] Shyamala Doraisamy.Polyphonic Music Retrieval:The N-gram Approach. PhD thesis, University ofLondon, 2004.

[10] J. S. Downie.Evaluating a Simple Approach to Mu-sic Information Retrieval: Conceiving Melodic N-grams as Text. PhD thesis, University of WesternOntario, 1999.

[11] Guy L. Steele.Common LISP : The Language. Dig-ital Press, 2nd edition, 1990.

[12] Francisco Cribari-Neto and Spyros G. Zarkos. R:Yet another econometric programming environ-ment. Journal of Applied Econometrics, 14:319–329, 1999.

Publication [P3]

[P3] M. Kuuskankare and M. Laurson. Intelligent Scripting in ENP using PWCon-

straints. In Proceedings of International Computer Music Conference, pages 684–687,

Miami, USA, 2004.

97

Intelligent Scripting in ENP using PWConstraints

Mika KuuskankareDocMus, Sibelius Academy

[email protected]

Mikael LaursonCMT, Sibelius Academy

[email protected]

Abstract

In this paper we are going to focus on how a constraint-based language, PWConstraints, can be adapted so that iscan be used to script our music notation package, called Ex-pressive Notation Package (ENP). We briefly introduce thebasic concepts behind PWConstraints and give an overviewof the scripting properties of ENP. We also present and exam-ine in detail a number of example scripts.

1 Background

Scripting languages are programming languages specificto a particular program. They are usually weakly typed oruntyped and interpreted rather than compiled. A scriptingprogram, a script, is normally used to automate complex oradvanced tasks within the program. Scripts typically interacteither with other programs or with a set of functions providedby the interpreter. Scripts are usually stored in ASCII format,thus they can be distributed across different computer plat-forms. This is an important feature for cross-platform com-puter applications.

A practical application of a script is to program eitherrepetitive or otherwise laborious tasks so that they can be ex-ecuted in a single pass. In a music notation program sucha task could be to apply a certain articulation pattern to apassage of music, to recalculate the enharmonic identity ofselected notes, etc.

There are some commercial music notation programsthat provide a scripting language and/or plug-in interfacethat allows the user to modify the information in the score.In Sibelius, for example, the user can write scripts by us-ing the built-in scripting language called ManuScript. TheManuScript syntax resembles, in many aspects, Java andC++.

ENP (Kuuskankare and Laurson 2002) is a music notationprogram that we have developed in order to meet the require-ments of computer aided composition, music analysis and vir-tual instrument control. ENP is programmed with Lisp andOpenGL (Woo et al. 1999). It’s purpose is to display scores

using the traditional western notation. Furthermore, ENPprovides a rich set of both standard and user definable nota-tional attributes, called ENP-expressions, which range fromstandard articulation marks to a fully editable multi-purposegraphical expression, called score-BPF. All ENP-expressionsare dynamic, i.e, they adjust their visual appearance accord-ing to their notational context.

PWConstraints (Laurson 1996), in turn, is a family ofconstraint-based programming tools that are aimed at solvinga wide range of constraint satisfaction problems. Constraintsatisfaction is a general problem in which the goal is to findvalues for a set of variables that will satisfy a given set of con-straints (Leler 1988). The musical applications of PWCon-straints are closely related to ENP and PWConstraints canaccess the data structures behind the ENP notational front-end.

In the following we compare some important featuresfound in other scripting languages, such as ManuScript, toour scripting system in ENP. (1) Usually, when writing ascript, the user has to write code to handle looping throughthe notational objects in the score. In our approach the usernever sees the ’main loop’ as it is handled by PWConstraints.(2) In case of ManuScript, for example, the scripts are notundoable. In order to revert the effect of the script one mustmanually remove the attributes one by one. In ENP, on theother hand, the system records knowledge about the individ-ual runs so that the user can later identify them and removethe attributes in any combination (selective undo). (3) In mostof the music notation programs the notational attributes arenot dynamic. Let us assume a case where the user adds someanalysis information to the score with the help of a script.Whenever the related musical information changes the analy-sis results will become invalid. Normally, the user is requiredto remove the old attributes by hand and run the script again.In ENP, however, the attributes are dynamic (when applica-ble). Once inserted they adjust themselves according to thechanges made in the music notation. (4) Most scripting lan-guages lack the ability of PWConstraints to easily define andidentify complex musical patterns with the help of a pattern-matching syntax.

The rest of the paper is structured as follows. First we

give an overview of PWConstraints and the pattern-matchinglanguage. Next we discuss the scripting in ENP and give anoutline of the scripting properties. We end the paper withsome example scripts and concluding remarks.

2 Overview of the PWConstraints

PWConstraints is a general-purpose rule-based program-ming language where the user writes rules to describe theend result from a number of different viewpoints. A search-space is defined as a set of search-variables. Each search-variable has a domain containing a list of values. In a rulea pattern-matching language is used to extract relevant infor-mation from a potential solution. This information is given toa Lisp test function that either accepts or rejects the currentchoice made by the search-engine (i.e., the rules return a truthvalue).

A PWConstraints rule consists of two parts: the pattern-matching part and the Lisp-code part. A special symbol?ifis used in the rule to distinguish between the pattern-matchingpart and the Lisp-code part.

The pattern-matching part consists of several differentpattern-matching variables. The variables are named as fol-lows:

1. Named variable(e.g.,?1, ?note1, ?note2) is used toextract single search-variables from a partial solution.The name (the portion of the text after the questionmark) is user definable.

2. Anonymous variable(?) is used when a search-variableis required to define a pattern but its value can be ig-nored.

3. Wild card (*) matches any continuous part of a partialsolution. In the pattern-matching part the wild card canoccur only once.

4. Index variable(e.g.,i1, i12) extracts search-variablesfrom an absolute position in the partial solution.

The following gives an example of a PWConstraints rule.This rule forces an ascending series of numbers:

(* ?1 ?2 (?if (< ?1 ?2)))

The pattern-matching part contains a wild card and twonamed variables. The rule loops through every adjacent pairof search-variables (indicated by?1 and ?2)) in the givensearch-space. The Lisp-code part, in turn, applies a test func-tion (< ?1 ?2) to each pair and returns a truth value.

PWConstraints contains also a collection of more spe-cialized tools that extend the behavior of the basic schemedescribed above (Laurson and Kuuskankare 2001). These

tools include Score-PMC, Texture-PMC, and MA-PMC (MAstands for music analysis). Score-PMC is used to solve poly-phonic search problems. An input score (an ENP score withthe rhythmic structure prepared in advance) is given as anargument to the search-engine. The search, in turn, aims atfilling the input score with pitch information according to thegiven rules. Texture-PMC, in turn, is used to produce rhythmsand textures. Finally, MA-PMC is used to analyze existingrhythmic, melodic and harmonic material in a score. Unlikethe former cases MA-PMC does not use backtracking.

In all aforementioned tools a search-variable represents anote in the input score. In these cases the search-variablescontain knowledge about pitch, rhythm, harmony, etc. Thisknowledge can be accessed in a rule with a set of specializedLisp-functions.

In this study, MA-PMC is used as a starting point to im-plement our scripting language. This approach is further ex-plained in the following Section.

3 Scripting in ENP

The scripting language of ENP uses MA-PMC to providethe basic ability to map through the notational objects in thescore. ENP script can contain an arbitrary number of script-ing rules. In case of the scripting rule the pattern-matchingsyntax does not require any changes when compared to anordinary PWConstraints rule. The Lisp-code part, however,does not return a truth value. Instead it contains a list of Lispexpressions that perform various side effects to the score suchas adding expressions. The approach described here has sev-eral advantages:

1. The syntax of the pattern-matching language is com-pact and powerful.

2. There is a built-in mechanism in MA-PMC to mapthrough all the notational objects in an ENP score.There is no need for the user to write tedious loop con-structs.

3. MA-PMC provides rich knowledge about the melodic,rhythmic and harmonic properties of the score. Thisinformation can be used in the scripts.

ENP provides a set of specialized Lisp-functions that canbe used to manipulate the appearance and content of ENP ob-jects. Virtually every notational attribute can be accessed andmanipulated with the scripts. Among the scripting propertiesof ENP are: color; channel and velocity; note-head type; en-harmonic identity; time-signature, key-signature and tempo;start-time and duration; x-offset and y-offset, etc.

4 Some Scripting Examples

In this section we examine some ENP scripts in detail.Due to space limitations the user-interface is not shown here.The ENP script-editor, however, contains a text editor to inputthe script code, a few options to choose from (e.g., if the scriptis applied to a selection only) and a list of the scripts alreadyapplied with the possibility for selective undo.

4.1 Adding an Articulation Pattern

In our first example we use a script to insert a repeat-ing articulation pattern to a passage of sixteenth notes. Inthis case the selected articulation pattern isslur-staccato-staccato. This particular example underlines the fundamentalidea behind scripting: to perform a potentially laborious andrepetitive task. We begin by defining the script:

(* ?1 ?2 ?3 ?4 ;1(?if (when (downbeat? ?1) ;2

(add-expression ’slur ?1 ?2) ;3(add-expression ’staccato ?3) ;4(add-expression ’staccato ?4)))) ;5

The first row of the script (1) shows the pattern-matchingpart containing a wild card and four named variables(?1, ?2, ?3 and?4). The second row (the beginning of theLisp-code part), in turn, checks if the first of the four notesfalls on a downbeat. Provided that it does, the next row (3)of the script adds a slur to the first two notes,?1 and?2, andthe remaining two rows (4 and 5) add staccatos to the last twonotes,?3 and?4, respectively. Figure 1 gives the output of thescript. The benefits of this kind of a script are naturally moreobvious when the musical passage in question is very long.

Figure 1: An articulation pattern is applied to passage of mu-sic with the help of a script.

4.2 Adding Analysis Information

The second script analyzes the given melodic line by cal-culating the set class identity of every group of three con-secutive notes (see Figure 2). This example demonstrates theability of the script to access the information contained by thenotational objects. The script is defined as follows:

(* ?1 ?2 ?3 ;1(?if (add-expression ’group ?1 ?2 ?3 ;2

:info (set-class-name ?1 ?2 ?3)))) ;3

In this case, the pattern-matching part (1) shows a wildcard and a group of three named variables (?1, ?2 and?3).The first row of the Lisp-code part (2) adds a special expres-sion, calledgroup1, to the three notes. The set-class identityof the notes is supplied as a parameter for the group indicatedby the keyword:info (3).

Figure 2: The melodic excerpt is analyzed by the script andthe analysis information is added to the score accordingly(Berg: Lyrical Suite).

It is to be noted, however, that in this case the informa-tion added to the score is static. Should the user, for example,transpose any of the notes in Figure 2, the analysis informa-tion shown by the expressions would become incorrect. ENPprovides dynamic expressions for this kind of analytical pur-poses, and they should be preferred over the static ones inpractical applications. The next two examples show the ad-vantage of dynamic expressions used along with scripting.

4.3 Adding Trumpet Fingerings

There is sometimes the need to explicitly display the fin-gering information of different instruments in a score. Thiscould be done for pedagogical purposes or for providing con-trol information for a virtual instrument model (see for exam-ple Laurson et al. 2001)

The script for adding trumpet fingerings for each note inthe score is defined as follows:

(* ?1 (?if (add-expression ’trumpet-fingering ?1)))

Figure 3 shows the output of the script. This may seemtrivial at first but the complexity of the case, however, be-comes apparent after the user changes the musical informa-tion, for example, by transposing. In Figure 4 the user hastransposed the musical material by a whole-step. This showsthat the fingering expressions inserted by the script are dy-namic, i.e., they adjust their visual representation accordingto the notational objects that they are attached to. There isno need to double check the notation or re-run the scripts inorder to keep the score up to date.

1A group is a specialized expression that displays any user definable tex-tual information as a part of the music notation. The extent of the group isalso visually indicated in the score.

Figure 3: Trumpet fingerings inserted into the score by thescript.

Figure 4: Fingering expressions are dynamic, so when themusical material is transposed the fingerings are updated ac-cordingly.

4.4 An Advanced Example

We conclude with a relatively complex example. Herewe use an excerpt from opus 25 by Arnold Schoenberg asa starting point. The aim is to fill a score with some of thearticulations that are found in the original score with the helpof a script. To make the example more straightforward weconcentrate on two types of articulations, grave accents andslurs. All other attributes that are found in the original areomitted. In the original score there are always slurs betweentwo notes if they form a large interval (the adjacent notes aremore than an octave apart) provided that there is not a restbetween them. Also, each sixteenth note has a grave accent.In this case we use the musical knowledge of PWConstraintsand ENP to determine the type and position of the insertedarticulations. This is done according to the following twoprinciples: the size of the melodic movement or the beaminglevel of the notes.

The result, which is shown in Figure 5, can be accom-plished with the following script containing two scriptingrules:

(* ?1 ?2 (?if (when (and ;1(> (abs (- (m ?2) (m ?1))) 12) ;2(not (prev-rest? ?2))) ;3

(add-expression ’slur ?1 ?2)))) ;4

(* ?1 (?if (when (= (beaming-level ?1) 2) ;5(add-expression ’accent-grave ?1)))) ;6

The first row of the script shows a wild card and twonamed variables (?1 and?2). In the next two rows wecheck if the size of the melodic movement is greater thanan octave (2) and also that there is no rest between the twonotes (3). If both conditions are met we insert a slur expres-sion (4). The latter part of the script is quite straightforward.

If the beaming of the note has two levels (5) a grave accent isadded (6).

Figure 5: The slurs and grave accents inserted by a script. Thetype and placement of the expressions has been determinedby the script according to the duration of the notes or the sizeof the melodic movement.

5 Conclusions

This paper demonstrated how ENP can be scripted usingthe PWConstraints pattern-matching language. The powerof the pattern-matching language combined with the intelli-gent expression handling of ENP provides a powerful toolfor different kinds of compositional and analytical applica-tions. The scripting scheme presented here is useful whenused simply as a speedy entry to attach, for example, articu-lations or articulation patterns. It can also be used to performmore sophisticated operations including music analysis andvisualization.

ReferencesKuuskankare, M. and M. Laurson (2002, September). ENP2.0 A

Music Notation Program Implemented in Common Lisp andOpenGL. InProceedings of International Computer MusicConference, Gothenburg, Sweden, pp. 463–466.

Laurson, M. (1996).PATCHWORK: A Visual Programming Lan-guage and some Musical Applications. Studia musica no.6,Sibelius Academy, Helsinki.

Laurson, M., C. Erkut, V. Valimaki, and M. Kuuskankare (2001,Fall). Methods for Modeling Realistic Playing in AcousticGuitar Synthesis.Computer Music Journal 25(3), 38–49.

Laurson, M. and M. Kuuskankare (2001). A Constraint BasedApproach to Musical Textures and Instrumental Writing. InCP01 workshop on Musical Constraints, Cyprus.

Leler, W. (1988).Constraint Programming Languages, TheirSpecification and Generation. Addison-Wesley PublishingCompany.

Woo, M., J. Neider, T. Davis, and D. Shreiner (1999).OpenGLProgramming Guide(3rd ed.). Massachusetts, USA: Addi-son Wesley.

Publication [P4]

[P4] M. Laurson and M. Kuuskankare. From RTM-notation to ENP-score-notation.

In Journees d’Informatique Musicale, Montbeliard, France, 2003.

103

From RTM-notation to ENP-score-notation

Mikael Laurson1 and Mika Kuuskankare2

1Center for Music and Technology,2Department of Doctoral Studies in Musical Performance and Research.

Sibelius Academy, P.O.Box 86, 00251 Helsinki, [email protected], [email protected]

Abstract

This paper discusses some recent developments within a compositional environment calledPWGL. Our focus is to present how score information is represented in PWGL. We give somebackground information concerning the rhythmic notation that was used in PatchWork (apredecessor of PWGL). After this we show how this notation has been expanded so that itallows to generate very detailed scores that can contain besides the basic rhythmic structuresalso other information such as grace-notes, instrumentation, pitch and expressions.

1 IntroductionPWGL (Laurson and Kuuskankare 2002) is a visual programming language based on Lisp,CLOS and OpenGL. PWGL is tool for computer assisted composition and sound synthesis.PWGL is based on similar concepts than PatchWork (PW, Laurson 1996) such as directrelation to its base languages Lisp and CLOS, musical objects and visual editors. The maindifference between PW and PWGL is that the graphics part of the implementation is basednow on OpenGL and not on QuickDraw as was the case with PW. OpenGL (Woo et al. 1999)has many attractive features and offers several advantages when compared to QuickDraw (formore details see Laurson and Kuuskankare 2002).

In this text we will concentrate on the way complex musical scores are represented inPWGL. Scores are of primary importance in our system and they can be used in manycompositional and analytical applications such as to produce musical material forinstrumental music. Scores have been also used successfully to produce control informationfor physical models of acoustical instruments (see for instance Laurson et al. 2001 andVälimäki et al. 2003).

The Score-editor of PWGL utilizes internally the ENP2.0 music notation package. ENP2.0(Kuuskankare and Laurson 2002) is based on the former ENP user library of PW(Kuuskankare and Laurson 2001). The Score-editor combines all PW music notation relatededitors (i.e. chord-editor, chord-sequence-editor, RTM-editor, PolyRTM-editor) into onepackage. ENP2.0 supports both mensural and non-mensural notation. Also ENP2.0 has beencompletely rewritten and is now based on OpenGL. The purpose of ENP2.0 is to provideprofessional notational capabilities, a graphical user interface and powerful object structuresfor compositional use. As PWGL, ENP2.0 is programmed with LISP and CLOS thus beingextendible and open. It also provides full access to the musical structures behind the notationallowing ENP2.0 to be controlled algorithmically. Furthermore, ENP2.0 provides bothstandard and user definable expressions.

Algorithmic control requires a high-level representation of the complex structuralinformation behind the visual front-end of a musical score. To be useful this representationhas to fulfil the following criterias. First, it has to be compact so that we are able to handlelarge scores efficiently. Second, it should be understandable to a human reader so that it couldbe used directly as input to build scores. Third, it should be easily translatable into objectstructures of the final score, and vice versa, any score should be translatable back to the high-level representation. Finally, it should be extendible in order to meet future improvements andextensions of the current environment.

The RTM-package (Laurson 1996) of PW allowed to generate complex rhythms out of alist representation for beats. Henceforth we will call this list representation RTM-notation. Inthe current PWGL implementation the RTM-notation representation has been augmented inseveral ways.

In the following we first introduce the PW RTM-notation and describe its most importantfeatures. After this we go over to the current implementation and show step by step how it canhandle scores with increasing complexity. We discuss also one elaborate score example andend with some concluding remarks.

2 RTM-notation in PWThe section gives a synopsis of the RTM-notation syntax used in PW to represent scores. Westart with Lisp lists that are converted to metrical rhythms. Lisp lists is a natural choice asthey allow easily to build hierarchical structures needed for our purposes. In the following weshow the basic components of the system and after this discuss several examples that can befound in the western notation practice (Read 1980).

2.1 Beat-listThe starting-point for a Lisp representation of a beat is a list. This list is called a beat-list andhas two components: beat-count and rtm-list. Beat-count is a positive number, usually aninteger. rtm-list, in turn, is a list of numbers. The numbers, called rtm-values, are usuallypositive integers, but can also be negative integers or floats. Let us assume the followingexample (Figure 1):

Figure 1: Some basic rhythms with their beat-list representations.

Figure 1 gives a measure containing four beats. Below each beat is the corresponding beat-list. Thus, for example, for the first quarter-note, the beat-list is (1 (1)). This detail is givenalso in Figure 2:

Figure 2: A quarter-note and its beat-list representation.

beat-count indicates how many units of the basic pulse is needed for a beat. The basicpulse, in turn, can be inferred from the denominator of the time signature. Thus, in theexample above, the basic pulse is a quarter-note. As the beat-count is 1, the length of the beatin Figure 2 will be one quarter-note.

The rtm-list consists of proportional values and has three properties: (1) the sum of allnumbers in the rtm-list, (2) the length of the rtm-list and (3) the values of the distinctnumbers. The sum gives the number of units into which the beat must be divided. The sub-unit of the beat equals 1 divided by the sum. Next, the length of the rtm-list determines thenumber of notes inside a beat. Finally, the rtm-value n indicates that n sub-units are to beassigned to the corresponding note. As the rtm-list in our example consists of only the number1, we have only one note with a length of a quarter-note.

Figure 3 gives the second beat of Figure 1, with the beat-list (1 (1 1)). The beat lasts forone quarter-note (beat-count = 1). As the sum of the rtm-list elements is 2, the sub-unit is 1/2.The beat contains two notes, both being 1/2 of the length of the beat, i.e. eighth-notes (1/4 *1/2 = 1/8).

Figure 3: Two eighth-notes and the corresponding beat-list.

In the third beat of the example in Figure 1 above, the beat-list is (1 (2 1)). The beat willlast for one quarter-note (beat-count = 1). As the sum of the rtm-list is 3, the sub-unit is 1/3 ofthe length of the beat. The length of the rtm-list is 2. The length of the first note is 2/3 (2 *1/3) of the beat, that of the second note 1/3 (1 * 1/3). Thus, the first note is a "sixth-note" (1/4* 2/3), the second a "twelfth-note" (1/4 * 1/3).

In the fourth beat of Figure 1 the rtm-list is (1 2), as opposed to the (2 1) of the previousbeat. Consequently, the length of the first note will be 1/3 of the length of the beat, that of thesecond 2/3.

2.2 Simple RhythmsThe examples below (Figures 4 - 6) show beat-list representations describing rhythms withdifferent sub-units. The sub-units range from 1/4 to 1/6:

Figure 4: sub-unit = 1/4.



Figure 7 gives an example with beat-count 2. Hence, each beat gets two units (2 * quarter-note) of the basic pulse:

Figure 7: beat-count = 2.

2.3 Embedded RhythmsUntil now all the rtm-lists have been simple lists, that is, only one level deep. To be able torepresent embedded rhythms, we add more levels to the rtm-list. In Figure 8, the top examplestarts with a simple beat-list (1 (1 1)). Then, in the right-hand side of the top example wereplace the second 1 of the rtm-list with a beat-list (1 (1 1 1)). The result is a new embeddedbeat-list: (1 (1 (1 (1 1 1)))). The second example starts with a beat-list (2 (1 1 1)). We replaceeach 1 of the rtm-list with (1 (1 1 1 1 1)).

Figure 8: The construction of embedded rhythms.

2.4 Compound RhythmsIn compound rhythms the grouping of notes is controlled by the beat-count. In Figure 9 thetime signature is 8/8 and the grouping is 3 + 3 + 2.

Figure 9: Compound rhythms.

2.5 Rests and Tied NotesUntil now, all the rtm-list examples contained only positive integers. To represent rests andtied notes, we allow the numbers inside the rtm-list to be negative numbers or floats. If anumber in a rtm-list is negative, it is considered to be a rest. If it is a float, it represents a tiednote. The rhythm in Figure 10 shows a combination of note-heads, rests and tied notes.

Figure 10: A combination of note-heads, rests and tied notes.

3 ENP-score-notation in PWGLIn this section we present a new and enriched list notation called ENP-score-notation. Wedescribe how the ENP-score-notation concept differs from the RTM-notation presentedabove. ENP-score-notation is based on some important concepts of RTM-notation, such asthe hierarchical PW beat-list. For several reasons, this approach was adapted also in ENP.First, we wanted to ensure basic backwards compatibility between PW music editors andthose of PWGL. Second, the list representation is uniform and proven to be powerful inpractical use. And third, it was easy to extend the expressiveness of the syntax, as required byENP, without scarifying the functionality or backwards compatibility.

In ENP the musical structures of PW have been modified and extended to add newfunctionalities and hierarchies (Kuuskankare and Laurson 2001). Thus, also the ENP-score-notation is enriched to reflect these changes. The starting point in the current implementationis to use the PW beat-list representation. Using the notation given above it is possible tocreate complex beat structures. To create higher-level structures (i.e., measures, voices, parts,scores) we collect lower-level structures as lists. For instance, beat-lists can be used to buildmeasures. The measures, in turn, can be used to build even higher-level structures.

3.1 ENP Score StructureAn ENP score is built out of hierarchical object structures. Thus a score consists of a list ofparts, a part consists of a list of voices, a voice consists of a list of measures and finally ameasure consists of a list of beats. For example we can create a score out of the following listrepresentation that reflects directly the ENP score structure (note that both measures consistsof beat-lists as described in Section 2).

( ; score

( ; part1

( ; voice1

((1 (1)) (1 (1 1 1)) (1 (1 2 1))) ; measure1

((1 (1 2 2)) (1 (1 2)) (2 (-1 2 1))) ; measure2

)

)

)

Figure 11 shows in practice how the previous ENP-score-notation example is used inPWGL to build scores:

Figure 11: A PWGL patch constructing scores.

3.2 Keyword ExtensionsThe basic ENP-score-notation system described until now can be augmented by insertingkeywords inside the list structure. This allows to define more precisely the resulting score. Atypical example is for instance a measure that should have knowledge of its time signature in

order to work properly. If no information is given then default values are used. For instance inthe example of Figure 11, the default denominator value of the time signature (or :low) is aquarter-note or 4 (the nominator value is calculated automatically using the beat-count valuesof the beat-lists), the default pitch value (or :midi) of a note is middle-C or 60, and so on.Typically the used keywords are slot-names of the resulting CLOS objects. This feature isuseful as it allows to add new slots in the existing musical objects without having to modifythe source code that takes care of the translation of the ENP-score-notation to an ENP score.

Each level of the ENP score structure has its own set of keywords. For instance, followingkeywords can be used: :instrument, :staff, :clef-number, :low, :metronome-value, :start-time,:notes, :expressions, :midi, :velocity and :channel.

Here are some ENP-score-notation examples dealing with various keywords accompaniedwith the resulting score (Figures 12 - 14):

((:instrument "Piano" :staff piano-staff ((:low 8 :metronome-value 72 (1 (1 1 1)) (1 (1 (1 :notes ((60 :clef-number 0) (60 :clef-number 1)))))))))

Figure 12: A score with keywords for instrument, staff, time signature, clef-number andmetronome.

(((((4 ((1 :notes (72 64)) (1 :notes (72 65)) (1 :notes (72 67)) (1 :notes (71 65)))))) (((4 (2 (2 :notes (62))))))))

Figure 13: A two-voice part example with pitch information.

(:Title "Duo" :spacing 0.7 (:instrument "Flute" (((1 ((1 :notes (72)))) (1 ((1 :notes (72) :expressions (:staccato)) (1 :notes (74) :expressions (:staccato)) (1 :notes (76) :expressions (:staccato)) (1 :notes (77) :expressions (:staccato)) (1 :notes (78) :expressions (:staccato))) 'grace-beat) (3 ((1 :notes (79) :expressions (:trillo))))))) (:instrument "Violoncello" :staff bass-staff (((1 ((1 :notes (36 43) :expressions (:accent :f)))) (3 ((1 :notes (52 60) :expressions (:accent))))))))

Figure 14: A two part example with instrument, staff, pitch and expression information.

3.3 Special ExtensionsThe PWGL ENP-score-notation system allows also to give extra information that defines thetype of a beat-object. One example of these special structures is the grace-beat. Grace-beatsare used to define grace-chords or grace-notes in an ENP score. An example of this can befound in Figure 14 where the upper part contains an ascending grace-note sequence. Grace-beats are given by inserting at the end of the beat-list definition the symbol grace-beat. Thefollowing expression:

(((((1 (1 1) 'grace-beat) (1 (1 1 1)) (1 ((1 (1 1)) (1 (1 1 1) 'grace-beat) (1 (1))))))))

gives as a result the score that can be seen in Figure 15:

Figure 15: An example with two grace-beats.

A similar extension allows to create accelerando- or ritardando-beats. The followingexample shows how to define these special beats (see also Figure 16):

(:spacing 0.33 :zoom 2.0 :pan-y -55.0 ((((1 (1 1 1 1 1 1 1) 'accelerando-beat) (1 (1 1 1 1 1 1 1) 'ritardando-beat)))))

Figure 16: An example containing one accelerando-beat and one ritardando-beat.

3.4 Group-expressionsENP expressions can be applied to a single note or chord or to a group of notes or chords. Wehave already seen several examples of the former expression type. This subsection discussesthe latter case which is called a group-expression. Group-expressions can overlap freely in thescore. They are interesting for our study as typically they do not obey the strict metricalhierarchy imposed by the beat-list and measure notation. Thus we need a special syntax togenerate them in a metric context.

Group-expressions can be inserted to a chord using an appropriate keyword to define theclass and instance identity of the expression in question. The keyword consists of two parts. Itbegins with an ENP class-name (e.g., 'group', 'slur', 'bpf', etc.). In addition of this, there can bean optional numeral that defines a group identity. The group identity numeral is used todistinguish between different instances of group-expressions. Thus, ':group1' would denote an

ENP object of class 'group' with identity value '1'. In this case, all subsequent references to':group1' would point to the same instance.

Furthermore, when an expression is introduced for the first time, any number ofinitialization-keywords can be supplied. In this case, the expression is given as a listconsisting of the expression keyword and the required initialization-keyword/value pairs, forexample: (:group1 :print-symbol "3-11a"). Below we find an example with three group-expressions:

(((((1 ((1 :notes (73) :expressions (:slur1)) (1 :notes (75) :expressions (:slur1 (:group1 :print-symbol "M1") )) (1 :notes (76) :expressions (:slur1 :group1)) (1 :notes (78) :expressions (:slur1 :group1 (:group2 :print-symbol "M2"))))) (1 ((1 :notes (74) :expressions (:slur1 :group1 :group2)) (1 :notes (76) :expressions (:slur1 :group1 :group2)) (1 :notes (77) :expressions (:slur1 :group1 :group2)) (1 :notes (79) :expressions (:slur1 :group1 :group2)))) (1 ((1 :notes ((75 :enharmonic 1)) :expressions (:slur1 :group1 :group2)) (1 :notes (77) :expressions (:slur1 :group2)) (1 :notes (79) :expressions (:slur1 :group2)) (1 :notes ((80 :enharmonic 1)) :expressions (:slur1 :group2))))))))

Figure 17: Group-expressions example containing a slur and two overlapping groups called'M1' and' M2'.

4 Score exampleWe end with a complex ENP-score-notation example which combines several featuresdescribed in the previous subsections. The resulting score is found in Figure 18.

(((((2 ((1 :notes (52) :expressions ((:slur1 :slope -6.0) :pp)))) (1 ((1.0 :notes (52) :expressions (:slur1)) (1 :notes ((58 :enharmonic 1)) :expressions (:slur1)) (1 :notes (59) :expressions (:slur1)) (1 :notes (62) :expressions (:slur1)))) (1 ((1 :notes ((63 :enharmonic 1)) :expressions (:slur1 :crescendo1)) (1 :notes (65) :expressions (:crescendo1)) (1 :notes ((66 :enharmonic 1)) :expressions (:crescendo1)) (1 :notes ((68 :enharmonic 1)) :expressions (:crescendo1)) (1 :notes (69) :expressions (:crescendo1)))) (1 ((1 :notes ((72)) :expressions (:slur2 :f)) (1 :notes ((73 :enharmonic 1)) :expressions (:slur2) :x-offset 1.0) (6 :notes (79) :expressions (:fp :crescendo2) :x-offset 2.0)))) ((1 ((1.0 :notes (79) :expressions (:crescendo2)) (-1 :notes (72) :expressions (:fz)))) (2 (-1 (1 :notes ((72 :note-head :x)) :expressions (:accent-grave :ff (:group1 :print-symbol "slap" :expression-fore-color 0))) (1 :notes ((69 :note-head :x)) :expressions (:accent-grave :group1)) (1 :notes ((73 :note-head :x)) :expressions (:accent-grave :group1)) (1 :notes ((68 :note-head :x)) :expressions (:accent-grave :group1)) (1 :notes ((66 :note-head :x)) :expressions (:accent-grave :group1)))) (1 ((1 :notes (77) :expressions (:glissando1 :fp :crescendo3)) (1 :notes ((80 :note-head :invisible)) :x-offset 1.0))) (1 ((1 :notes (83) :expressions (:glissando1 :fz :staccato :crescendo3))))))))

Figure 18: A complex example containing dynamics, single expressions, group-expressionsand special note-heads.

5 ConclusionsThis paper gave an overview of some recent PWGL developments dealing with a novel scorerepresentation called ENP-score-notation. We first introduced the PW beat-list protocol whichis used to construct metric structures. After this we augmented the beat-list scheme withkeywords and special extensions in order to be able to build scores containing parts, voices,measures, chords and notes. The system allows also to define additional notational attributessuch as expressions, grace-notes and group-expressions.

ReferencesKuuskankare M. and M. Laurson. 2001. "ENP, Musical Notation Library based on Common Lisp and CLOS".

In Proc. of ICMC’01, Havana, Cuba, pp. 131-134.Kuuskankare M. and M. Laurson. 2002. "ENP2.0 A Music Notation Program Implemented in Common Lisp and

OpenGL". In Proc. of ICMC'02, Gothenburg, Sweden, pp. 463-466.Laurson M. and M. Kuuskankare. 2002. “PWGL: A Novel Visual Language based on Common Lisp, CLOS and

OpenGL”. In Proc. of ICMC'02, Gothenburg, Sweden, pp. 142-145.Laurson M., C. Erkut, V. Välimäki, and M. Kuuskankare. 2001. "Methods for Modeling Realistic Playing in

Acoustic Guitar Synthesis." Computer Music Journal 25(3).Laurson, M. 1996. PATCHWORK: A Visual Programming Language and Some Musical Applications. Doctoral

dissertation, Sibelius Academy, Helsinki, Finland.Read, G. 1980. Modern Rhythmic Notation. London. Victor Gollanz Ltd.Woo M., J. Neider, T. Davis, and D. Shreiner. 1999. OpenGL Programming Guide. Addison Wesley, 3rd edition,

Massachusetts, USA.Välimäki V., Laurson M., and C. Erkut. 2003. "Commuted Waveguide Synthesis of the Clavichord." To be

published in Computer Music Journal.

Publication [P5]

[P5] M. Kuuskankare and M. Laurson. Recent Developments in ENP-score-notation.

In Sound and Music Computing ’04, Paris, France, 2004.

113

RECENT DEVELOPMENTS IN ENP-SCORE-NOTATION

Mika KuuskankareDocMus

Sibelius [email protected]

Mikael LaursonCMT

Sibelius [email protected]

ABSTRACT

In this paper we describe the recent developments on atext-based score representation format called ENP-score-notation. ENP-score-notation is used to describe scores inExpressive Notation Package (ENP). We introduce the ba-sic concepts behind ENP-score-notation and discuss sev-eral new features in detail along with some notational ex-amples. We also examine some syntactic changes andpoint out the differences between the new and the old syn-tax. Furthermore, we propose a new interface to PWGL,including several new boxes, which allows to use ENP-score-notation to construct musical objects in a PWGLpatch.

1. BACKGROUND

Many different text-based formats have been developedfor representing musical information. Some typical ex-amples are Common Music Notation [10], GUIDO [3],LilyPond [9], and MusicXML [2]. There are also someLisp-based representations that are able to preserve onlya subset of the musical information, such as the rhythm.These include the RTM-notation of both PatchWork [6]and OpenMusic [1].

There are several advantages in text-based representa-tion of a musical score: (1) the layout of notational ob-jects is separated from the representation, (2) the musicaltypesetting is improved when the typesetting program im-proves, and (3) textual formats are human readable.

ENP-score-notation [8] is a text-based format to repre-sent ENP scores. ENP [4] is a LISP-based music notationprogram. It currently runs on MacIntosh OS X platform.The main purpose of ENP is to represent musical data forcompositional and music analytical applications. ENP-score-notation is based on the hierarchical score structureof ENP. ENP-score-notation offers a large subset of defin-able attributes. The syntax is designed so that it also al-lows to add new features easily. The current subset, how-ever, should be large enough for most practical purposes.The Lisp list format allows a straightforward way to gen-erate and manipulate musical data. ENP-score-notation

M.Kuuskankare and M.Laurson, Recent Developments in ENP-score-notation, inSound and Music Computing 04, October 2004.http://smc04.ircam.fr/

can also be used in PWGL to represent musical objects.PWGL [7] is a multi-purpose visual programming envi-ronment specialized in music and sound related applica-tions. PWGL uses ENP as its notational front end.

The rest of the paper is organized as follows. Firstly,we briefly present the overall data structure of an ENPscore. Secondly, we discuss ENP-score-notation in gen-eral. Next, we point out some syntactic changes in ENP-score-notation. Finally, we introduce a set of new PWGLboxes, relating to ENP-score-notation, along with someexample patches.

2. ENP SCORE STRUCTURE IN BRIEF

ENP scores are internally represented as hierarchical treestructures. For example, a score consists of a list of partsand parts, in turn, consist of list of voices, etc. In general,any higher-level structures are constructed by collectinglower-level structures as lists. Figure 1 gives an overviewof the score structure.

Figure 1. An example of an ENP score (above) and itsobject hierarchy displayed as a tree structure (below).

On the beat-level ENP score representation is based onthe concept of PW-beat which can be represented as a Lisplist. This list, called abeat-list , has two main com-ponents:beat-count andrtm-list . beat-countis a positive number, usually an integer.rtm-list , inturn, contains a list of numbers, calledrtm-values .The type of an individualrtm-value has the followingmeanings: (1) positive integer represents a note, (2) neg-ative integer represents a rest, and (3) floating-point num-ber indicates a note that is tied to the previous one.

Figure 2 gives some typical examples of thebeat-list notation along with their notational counter-parts. More detailed explanation of PW-beat can be foundin [6]. ENP score structure, in turn, is discussed in greaterdetail in [8].

Figure 2. Some simple rhythms demonstrating the use ofrtm-values in thebeat-list .

3. ENP-SCORE-NOTATION

When we use a text-based format to describe the complexstructural information contained by a musical score thereare several things to be considered. The syntax has to be:(1) compact so that we are able to handle large scores ef-ficiently, (2) understandable to a human reader so that itcould be used directly as input to build scores, (3) easilytranslatable into the score representation (and vice versa),and (4) extensible in order to meet future extensions andimprovements.

3.1. ENP-score-notation Syntax

The structure of ENP-score-notation [8] reflects thehierarchical structure of the score. The syntax is similarto the LISP list syntax and also resembles CLOS in theway the optional keyword arguments are given. Thebasic syntax of ENP-score-notation describes only therhythmic skeleton along with the overall structure ofthe score (i.e., parts, voices, measures, etc.). All otherinformation (including pitches) is defined as optionalkeyword arguments. For example, in its simplest form,a triplet is notated as(1 (1 1 1)) 1. The pitchinformation2 can be added by using a keyword:notes ,e.g., (1 ((1 :notes (60 64 67)) (1 :notes(60 64 67)) (1 :notes (60 64 67)))) .The :notes keyword refers to an actual slot in an ENPchord object.

There are several advantages in this approach:(1) Since all the attribute names refer to actual slots inCLOS objects, features can be added in most cases with-out any additional coding. (2) Syntax is uniform and sim-ple: everything else, besides rhythm, is defined by addingoptional attributes. (3) Unless a particular attribute isgiven it is set to some reasonable default. (4) The hier-archical relations of notational objects can be determinedby examining the textual representation. (5) The format

1Even if not explicitly indicated the resulting triplet would in thiscase also contain some pitch information. The default pitch, unless statedotherwise, is c1 (= middle c).

2Pitches in ENP are generally represented as midi values, 60 beingc1, i.e., middle C. However, ENP can also represent microtones. Theyare notated by adding the relevant decimal fraction to the midi value.Thus, 60.5 would be c1 raised by 1/4 (half a semitone).

maps exactly to the score hierarchy and is therefore easilytransformable back to an ENP score.

Below we give a simplified but complete example ofthe structure of ENP-score-notation along with the result-ing score (Figure 3):

Figure 3. A simplified ENP-score-notation example (left)and the resulting score (right).

One of the problems with strict hierarchical represen-tation is how to express entities that do not obey the hier-archy. ENP-expressions are a typical group of objects thatbreak the otherwise hierarchical representation describedabove. In the next subsection we briefly describe howENP-expressions are represented in ENP-score-notation.

3.2. Syntax of ENP-expressions

Currently ENP-expressions [4, 5] can be attached to eithersingle chords or notes (single-expressions) or to groupsof chords or notes (group-expressions). Expressions canbe incorporated into ENP-score-notation by using a key-word :expressions followed by a list of expressions andpossible initialization options. Expressions are identifiedby using an appropriate keyword indicating the class ofthe expression. Among valid expression keywords are,for example,:group, :slur, :bpf, :accent,:staccato , etc. In case of group expressions the key-words usually contain an additional numeral that indicatesthe instance identity of the expression. The instance iden-tity numeral is used to distinguish between different in-stances of group-expressions (in case of overlapping slurs,for example). Thus,:slur1 indicates an ENP expres-sion of class ’slur’ with the identity value ’1’. All subse-quent references to:slur1 point to the same instance.Furthermore, when an expression is introduced for thefirst time, any number of initialization options can be sup-plied. This information consists of keyword and valuepairs defining any additional properties of the expression.The components of an expression keyword are as follows:

expressionclass

�� : slur1 -

�� instanceidentitynumeral

4. CHANGES IN ENP-SCORE-NOTATIONSYNTAX

In this Section we enumerate some syntactic changes inENP-score-notation and also present some new function-alities, such as the possibility to describe non-mensuralnotation.

4.1. Changes in Beat Types

Formerly, the type of the resulting beat was indi-cated by inserting an extra symbol as the last el-ement in the beat-list. For example, a grace-beatwas created by using the symbol’grace-beat , e.g.,(1 (1) ’grace-beat) . The new syntax, how-ever, unifies the use of these kinds of optional at-tributes. When using the current version of ENP-score-notation the grace-beat is defined as follows:(1 (1) :class :grace-beat) . Notice the in-clusion of a keyword:class . The beat list canalso have several other optional attributes. The fol-lowing values for the:kind keyword are currentlysupported: :accelerando, :ritardando , and:feathered . The distinction between:class and:kind keywords also allow to mix these attributes, e.g.,to create a grace-beat with accelerando beaming.

Figures 4 demonstrates the use of the:kind keywordand Figure 5, in turn, gives an example of a grace-beatcreated with the keyword:class .

(((((1 (1 1 1 1 1 1) :KIND :ACCELERANDO)))))

Figure 4. A beat with accelerando beaming.

(((((1 ((1 (1 1) :CLASS :GRACE-BEAT) (-1)))))))

Figure 5. A grace-beat

4.2. Non-mensural notation

The current version of ENP-score-notation allowsto represent non-mensural notation. The syntax isotherwise identical to that of mensural-notation, butinstead of using proportional beat-counts and rtm-values, non-mensural notation uses absolute start-timesand relative offset-times. Thus a mensurally notatedchord, (1 ((1 :notes (60 64 67)))) , couldpotentially be translated to non-mensural notation as(0.0 :notes (60 64 67) :duration 1.0) ,where the first floating point number represents thestart-time of the chord.3 Each note can also have anindependent offset-time relative to the start-time of thechord. Next we give a complete example of non-mensuralnotation and the resulting score (Figure 6):

3We assume that, in the mensural case, the time-signature is 4/4 andtempo is 60 beats/minute. Thus, the duration of one quarter-note is 1second.

(((((0.2 :NOTES ((60 :OFFSET-TIME -0.15)(66 :OFFSET-TIME 0.0)(71 :OFFSET-TIME 0.15))

:EXPRESSIONS ((:crescendo/1:initial-dynamics :p :end-dynamics :f)))

(0.5 :NOTES (77) :EXPRESSIONS (:crescendo/1))))))

Figure 6. A non-mensural score described with ENP-score-notation.

4.3. Changes in ENP-expression Syntax

When using the old ENP-score-notation syntax the in-stance identity numeral is attached directly to the end ofthe expression keyword, i.e.,:bpf1 . This creates a po-tential conflict when used with expressions whose namescontain a numeral, such as the string number expressions(:string1 , :string2 , etc.). The new syntax allowsto separate the expression name and the instance identitynumeral more precisely by using a slash (/). Now differentinstances of, for example, string number expressions canbe denoted as:string1/1 and :string1/2 . Thisapproach clarifies ENP-score-notation syntax. The oldsyntax, however, is still valid and can be used alongsidethe new one.

4.4. Score-BPF notation

The Score-BPF [5] is multipurpose graphical object thatcan represent breakpoint functions as a part of a musicaltexture. breakpoint functions, in turn, are piece-wise lin-ear functions. It can contain an arbitrary number of break-point functions which, in turn, contain a list of points withx and y values. The x values are timing values, in seconds,relative to the start-time of a Score-BPF in the score.

The new version of ENP-score-notation supportsScore-BPF expressions. Currently there are two ways todefine a point in a breakpoint function:

(1) By using absolute values. In this case a point cansimply be indicated by a list of two numbers, e.g.,(0.0 1.0) . The first number in the list definesthe position of the point in time relative to the start-time of the Score-BPF. The second value is a y valuethat should fall inside the user definable height of theScore-BPF.

(2) By using values that are proportional to the length (intime) and height of the Score-BPF. The point is writ-ten as a list of two numbers ranging form 0.0 to 1.0. Inthis case it is required to include an optional keywordargument,:scaled t , indicating that the incomingvalues are to be scaled inside the Score-BPF (Figure7).

Moreover, the user can define a number of others at-tributes, such as type, name and height of the Score-BPF,the name and color of the breakpoint function, etc.

(((((1 ((1 :notes (72):expressions ((:bpf1 :points (((0.0 0.5)

(1.0 0.75):scaled t:name "acc.")))))

(1 :notes (74) :expressions (:bpf1))(1 :notes (76) :expressions (:bpf1))(1 :notes (77) :expressions (:bpf1))(1 :notes (78) :expressions (:bpf1))))))))

Figure 7. A Score-BPF expression (:bpf1) inserted in thescore with the help of ENP-score-notation. Notice alsothe use of the extra arguments:name and:scaled .

5. PWGL INTERFACE TOENP-SCORE-NOTATION

In this Section we present the interface between ENP-score-notation and PWGL. Due to its list based rep-resentation ENP-score-notation provides a natural wayto construct musical structures in a PWGL patch. Weintroduce next three new boxes that allow the userto generate and manipulate ENP objects in a PWGLpatch. Two of the boxes,enp-constructor andenp-object-composer , are used to convert ENP-score-notation to ENP objects. The third complementarybox,enp-score-notation , is in turn used to convertan ENP score to ENP-score-notation.

5.1. enp-constructor -box

enp-constructor -box is used to create a single kindof ENP object. The main-box consists of two requiredinput-boxes: (1) Selector-input-box which is used to se-lect the type of the resulting object. The box dynamicallychanges the number and type of required input-boxes asthe user changes this value. (2) One or two value-input-boxes that accept as input either ENP-score-notation ex-pression or ENP objects. (Figure 8)

Any additional properties can be defined by extendingthe enp-constructor -box. Extending reveals onespecial input-box pair at the time. The collection of pos-sible input-box pairs depends on the type of the resultingobject. The input-box pairs, in turn, contain two input-boxes arranged in a row. The first input-box,selector-input-box, is used to select an attribute keyword. A changein the selector-input-box performs a side effect to the sec-ond input-box, calledstate-input-box. The state-input-boxdynamically changes its type according to the value ofthe selector-input-box. If the value of the selector-input-box is, for example,:start-time , the state-input-boxis of type float-input-box. On the other hand, if thevalue of the selector-input-box is:expressions , thestate-input-box is an expression menu-input-box contain-ing all the possible ENP-expressions. Figure 8 shows an

enp-constructor -box for a beat object. The box isextended to add a class attribute (i.e., :class and :grace-beat). The lowest row of the box shows the correspondinginput-box pair.

Figure 8. enp-constructor -box is used to create agrace-beat.

5.2. enp-object-composer -box

enp-object-composer -box can be used to con-vert any lower level object to any of the higher levelobjects (see the score hierarchy in Section 2). Thus,a note could potentially be converted to a score, etc.enp-object-composer -box has several useful prop-erties: (1) it allows fast visualization of any musical object(e.g., convert a note directly to a score), (2) it can handleeither single objects or lists of objects, (3) it can be used asa shorthand to create any higher level objects out of anycombination of lower level objects (types can be freelymixed), and (4) the input format is similar to the hierarchi-cal score representation and can be used to further modifythe structure of the resulting object. Figure 8 shows oneapplication toenp-object-composer -box. In thiscase a grace-beat is converted directly into a score.

Let us next examine a more complex patch given inFigure 9 (see Appendix). The main idea behind this patchis to demonstrate how to create different score structuresfrom the same input data just by adding one extra paren-thesis each time. The original input in this case is a listof two notes. In the first case (a) this input creates a scorewith two parts each containing one note. In (b) one extraparenthesis is added around the two notes. This, in turn,creates a score with one part containing two voices. Eachvoice again contains one note (the stems of the notes aredrawn in different directions). In (c) two extra parenthe-sis are added around the the notes and the resulting scorecontains a part with two measures. In the last case (d)there is a total of four parenthesis around the notes. Theresulting score contains one part with a measure contain-ing two notes. This procedure can be continued to createeven deeper structures.

5.3. enp-score-notation -box

enp-score-notation -box is used to convert a scoreto ENP-score-notation expression. Unless stated oth-erwise, this box collects all the optional attributes foreach object in the score.4 It is, however, possible tohave control over which attributes are collected. Theenp-score-notation -box contains a menu-input-box and a dialog-input-box for this purpose (see Figure10). The menu-input-box can be used to choose between:exclude and:include keywords. The dialog-input-box, in turn, can be used to select the set of attributes froma dialog containing all the possible attributes. Accord-ing to the value of the menu-input-box these attributes arethen either included in or excluded from the result.

Figure 10, in the Appendix, gives an example of theextreme effects of:exclude and:include . The top-most score is the starting point containing a short pas-sage of music with some expression markings and spe-cial note-heads. Theenp-score-notation -box onthe left (a) has the:exclude parameter set toNILmeaning that nothing is excluded from the result. Thiscreates an exact copy of the original score (b). Theenp-score-notation -box on the right (c) has the:include parameter set toNIL . In this case none of theoptional attributes is included. Only the rhythmic skeletonis collected as shown in (d).

6. CONCLUSIONS

This paper presented some recent developments in ENP-score-notation. Several new features were introduced.Also some syntactic changes were proposed that fur-ther clarified or extended the functionality of ENP-score-notation. Furthermore, a set of dedicated PWGL boxeswere introduced. These boxes provide a bridge betweenENP-score-notation and PWGL and allow an algorithmiccontrol over ENP-score-notation.

There is still some room for improvement. The twoways to describe breakpoint functions, presented in thispaper, are quite primitive. There is no apparent relationbetween the points and the notational objects (except forthe start and end points). The syntax should be expandedso that it would allow the notational objects to be refer-enced by a symbol or an index.

4There is a relating mechanism in ENP called automatic source codegenerator. The purpose of the automatic source code generator is to builda Lisp representation of a CLOS object. It can then be used to re-createan exact copy of the object. However, in order to optimize the Lisp rep-resentation, the automatic source code generator also compares the prop-erties of the CLOS objects to their default values and determines if thevalue of the particular property needs to be included. This mechanism isquite sophisticated and can also handle arbitrary deep hierarchical struc-tures.

The same kind of strategy is used in the case of ENP-score-notation.The enp-score-notation -box actually collects only those at-tributes that are different from their default value. For example, let ussay that there is a note whose duration is 1.0 seconds. This is also thedefault duration of a note object. In this case the:duration keywordand the corresponding value are not included in the ENP-score-notationexpression.

The current inclusion functionality ofenp-score-notation -box is somewhat am-biguous. As explained in the footnote 4 the inclusion ofa certain attribute is dependent not only on the choicesmade by the user but also the underlying decisions madeby the system. There should be a mechanism to forcecertain attributes to be present in the ENP-score-notationexpression (e.g., force-include feature). This would en-sure that the needed attributes would always be collected.The information could then be used to construct otherobjects that explicitly rely on the existence of such data.Instead, the current implementation requires that the userhas knowledge about the default values of the variousattributes.

7. ACKNOWLEDGMENTS

The work of Mikael Laurson has been financed by theAcademy of Finland.

8. REFERENCES

[1] Gerard Assayag, Camillo Rueda, Mikael Laurson,Carlos Agon, and Olivier Delerue. Computer As-sisted Composition at IRCAM: From PatchWork toOpenMusic.Computer Music Journal, 23(3):59–72,Fall 1999.

[2] M. Good and G. Actor. Using MusicXML for FileInterchange. InThird International Conference onWEB Delivering of Music, page 153, Los Alamitos,CA, September 2003. IEEE Press.

[3] H. H. Hoos, K. A. Hamel, K. Renz, and J. Kil-ian. The GUIDO Music Notation Format - A NovelApproach for Adequately Representing Score-levelMusic. In Proceedings of International ComputerMusic Conference, pages 451–454, San Francisco,1998.


[5] Mika Kuuskankare and Mikael Laurson. ENP-Expressions, Score-BPF as a Case Study. InPro-ceedings of International Computer Music Confer-ence, pages 103–106, Singapore, 2003.

[6] Mikael Laurson.PATCHWORK: A Visual Program-ming Language and some Musical Applications.Studia musica no.6, Sibelius Academy, Helsinki,1996.

[7] Mikael Laurson and Mika Kuuskankare. PWGL:A Novel Visual Language based on Common Lisp,CLOS and OpenGL. InProceedings of Interna-tional Computer Music Conference, pages 142–145,Gothenburg, Sweden, September 2002.

[8] Mikael Laurson and Mika Kuuskankare. FromRTM-notation to ENP-score-notation. InJournees

d’Informatique Musicale, Montbeliard, France,2003.

[9] Han-Wen Nienhuys and Jan Nieuwenhuizen. Lily-Pond, a system for automated music engraving. InXIV Colloquium on Musical Informatics (XIV CIM2003), Firenze, Italy, May 2003.

[10] Bill Schottstaedt. Comon Music Notation. InBeyondMIDI, The Handbook of Musical Codes. MIT Press,Cambridge, Massachusetts, 1997.

Appendix

Figure 9. An Example showing the behavior of theenp-constructor -box. By manipulating the input data, differentkinds of score structures can be created.

Figure 10. An example showing the two extremes ofenp-score-notation -box. On the left: an exact copy is made.On the right: only the rhythmic skeleton is retained.

Publication [P6]

[P6] M. Kuuskankare and M. Laurson. ENP Expression Designer - a Visual Tool for

Creating User Definable Expressions. In International Computer Music Conference,

pages 307–310, Barcelona, Spain, 2005.

121

ENP EXPRESSION DESIGNERA VISUAL TOOL FOR CREATING USER DEFINABLE EXPRESSIONS

Mika KuuskankareSibelius Academy

[email protected]

Mikael LaursonSibelius Academy

[email protected]

ABSTRACT

In the present paper we introduce a new tool called ENPExpression Designer (ED). It is aimed at visually creatingENP-expressions. Currently, ENP allows to create newexpressions by using Lisp and CLOS. The strictly textualinterface can, however, be demanding to those users thathave limited or no experience in programming. ED pro-vides an easy and straightforward way to define new ex-pressions through a graphical user interface.

Through inheritance the ED allows to reuse the built-inproperties of the existing ENP-expressions. We also pro-vide a set of graphical primitives for drawing lines, text,etc. Additionally, more experienced users can take fulladvantage of the OpenGL graphical language.

1. INTRODUCTION

Especially in modern music notation there is the needfor special expression markings. There is an increasingamount of instrument specific playing techniques and newones are invented all the time. These markings can also beinvented even piece per piece basis. This makes it difficultfor a music notation program to provide a comprehensivecollection of expression markings. Thus, it is more rea-sonable to provide an interface for the user that (s)he canuse to build custom expressions.

Two of the most widely used commercial notation pro-grams, Finale [1] and Sibelius [2], provide the possibilityto create user defined expressions. In Finale it is possi-ble to create either text expressions or graphic expressions(shapes in Finale terminology). In Sibelius, in turn, theformer group of expressions is called Symbols and the lat-ter one Lines. Both programs provide tools to create bothtypes of expressions. However, when using these toolsit is usually not allowed to view the expressions in theirnatural context—music notation.

ENP [3] is a music notation program that has been de-veloped in order to meet the requirements of computeraided composition, music analysis and virtual instrumentcontrol. It’s purpose is to display scores using the com-mon western notation. Furthermore, ENP provides arich set of both standard and user definable notational at-tributes, called ENP-expressions. ENP is programmedin Common Lisp (LispWorks by Xanalys) and uses theOpenGL API to produce the notational graphics.

ED is a visual tool that can be used to create customexpressions in ENP. ED provides a simple protocol thathelps to define the basic properties of the user definableexpressions. It also allows to define the visual appear-ance using Lisp. There are several advantages in the ap-proach presented in this paper. One advantage that ENPprovides is that the defined expressions can contain userdefinable data. This means that the data can be read andwritten both through the graphical user interface or algo-rithmically. The data can then be used in, for example,compositional or music analytical applications. Second,the expressions can be dynamic since they are definedusing standard Common Lisp. Third, in ED the expres-sions are displayed in a fully functional ENP score, thusthe user can observe how the expressions behave, for ex-ample, when the music is transposed or when the layoutof the score is changed. This approach is different formany of the existing ones. For example, in Finale, the userdefinable expressions (shapes) are designed without thepresence of any musical context.

The rest of the paper is structured follows: first, wegive a brief overview of ENP-expressions. Next, we intro-duce the ENP Expression Designer and give a few prac-tical examples of its use. We end the paper with someconclusions and suggestions for future work.

2. OVERVIEW OF ENP-EXPRESSIONS

ENP-Expressions are divided into two main categories:(1) standard expressions, including articulations and dy-namics, and (2) non-standard expressions which include,for example, groups and Score-BPF’s [3]. Every expres-sion is attached either to a single notational object (singleexpressions) or to a group of notational objects (group ex-pressions).

ENP provides a collection of standard tempo-,dynamics- and articulation-symbols. Furthermore, it in-cludes a set of non-standard notational attributes. Newexpressions can be created through inheritance.

Next, we present some of the basic components ofENP-Expressions.

2.1. The Basic Components of ENP-Expressions

All ENP-expressions share some common properties.These include, for example, print symbol, default print

position, user definable offset, and font scaler.While this is not a complete list these properties al-

ready define a bulk of the visual appearance, placement,and behavior of the expressions. The specific functions ofeach of the properties are further discussed in subsections3.2 and 3.3. Figure 1 shows these properties in a musicalcontext.

Figure 1. The components of an ENP-expression. On theleft the graphical representation of an accent. On the right,some common attributes to all ENP-expressions.

3. ENP EXPRESSION DESIGNER

ED is a visual tool for rapid expression designing and pro-totyping. The workflow typically consists of three steps:(1) defining class and inheritance, (2) defining proper-ties, and (3) defining the visual appearance (coding). Allthese steps can be executed in a single ED window. TheED Window, in turn, contains the following components:class view, properties view, code view, and preview score(an example ED window is shown in Figure 3).

Expressions are defined by first entering a unique classname and its superclass. Next, some additional propertiesare defined. These include among other the font typeface.Finally, the graphical outlook of the expression is definedby entering some Lisp code in thecode view. The visualappearance of the new expression is shown in thepreviewscoreat the bottom of the ED Window.

There are also buttons for displaying the preview andgenerating the expression definition in Lisp. Thepreview-button refreshes thepreview scoreto give the updated vi-sual representation of the expression. Theprint source-button, in turn, generates a text document containing theLisp code needed to store and restore the user defined ex-pression.

Change in any of the aforementioned properties (ex-cept change in the Lisp code in the code view) providessynchronized feedback in the preview score. In case ofthe code view the user is required to press thepreview-button to update the contents of thepreview score.

In the following Subsections we examine in detail thecomponents of the ED Window.

3.1. Class view

In theclass viewthe user must specify a unique class nameand a superclass. By default all the user defined expres-

sions inherit from the built-inexpression -class. Thisis the fundamental class that provides the necessary datastructures and behavior of all ENP-expression.

3.2. Properties view

Theproperties viewprovides the following options:

(1) Menu-category. All user-defined expressions areautomatically added in the appropriate ENP menuswhere they are, in turn, arranged in groups accordingto the type and/or purpose. Here the user can eitherchoose one of the existing menu categories or createa new one by providing a new category name.

(2) Print symboldefines the string or the character whichis used as the printed representation of the expression.

(3) Print position. There are several predefined printpositions which, in turn, are used to determine thedefault print position. The current choices are::on-stem , :above-staff , :below-staff ,:above-chord , and:notehead . The individualeffects of these values can be seen in Figure 2.

Figure 2. Five accent expressions attached to the chord.Each accent is given a differentprint position. The effectcan be seen in the example along with the respective printposition keyword.

(4) Font can currently be either:times or :sonata .

There are also some check boxes in the properties view.These can be used to determine whether the expressionis standard notation, and whether the expression is savedalong with the score or not.

3.3. Code view

In the code viewthe user can write standard Lisp code(see Figure 3). The code defines how the expressionis drawn in the score. The users can utilize any ofthe OpenGL functions provided by LispWorks. Thereare also some special graphical primitives provided byENP that can be used to draw, for example, lines, poly-gons, circles, etc. The following list gives the completeset of graphical primitives currently available: draw-2D-arrow, draw-2D-box, draw-2D-bracket, draw-2D-circle,draw-2D-line, draw-2D-lines, draw-2D-point, draw-2D-polygon, draw-2D-quads, draw-2D-shape, draw-2D-text,draw-2D-texture, and draw-2D-triangles.1

1 The list of the graphical primitives is quite self explanatory and it isnot further discussed in this paper.

There are also some special variables that can be re-ferred to in the code. The following variables containsome useful properties of the expressions:(1) x andy give the absolute position of the expression.

The are calculated according to the position of the as-sociated notation object and other constraints. Thesevalues are an aggregate of the default position (cal-culated according to the print position) and the userdefinable offset discussed above (see Subsection 3.2).

(2) width gives the horizontal distance from the currentnotation object to the next one.

(3) print-symbol contains the string or character de-fined by the user in theproperties view.

(4) font gives the font name as a keyword. Currently,the value of this variable is either:times (for read-able text) or:sonata (for music notation).

(5) instrument variable contains an ENP instrumentobject that, in turn, contains information about thename, range, etc.

(6) expression is the current ENP expression objectthat is being drawn. This object can also be queriedabout extra information such as any user definabledata stored in the expression.

(7) self , in this case, refers to the notational object theexpression is associated with (i.e., a note or a chord).Different properties of the notational object, includingpitch, start-time and duration, can also be used in theexpression code.

3.4. Preview score

Thepreview scoredisplays the user defined expression asit would appear in a regular ENP score. Thepreview scoreis also fully editable so the user can observe how the ex-pression behaves, for example, when the music is trans-posed, etc. The score can also be panned and zoomedfreely so that the expressions can be examined and alignedin great detail.

There is one additional feature to this view. As de-scribed above, the default expression position is calculatedaccording to the selectedprint positionand an additionaluser definable offset. This offset can be adjusted in thepreview scoreby dragging the expression with the mouse.

4. SOME EXPRESSION DESIGNER EXAMPLES

In this Section we give two examples on how the ED canbe used to create new expressions. In the first example,wirebrush, we build a simple percussion mallet symbol[5]. This example demonstrates the use of some of thegraphical primitives. In the second example, in turn, wecreate an expression that uses the knowledge about thespacing of the associated notational objects to fill a givenspace in the score.

Due to space limitations the examples are given with-out any detailed explanation. The key points, however, arediscussed along each example. Some knowledge about

Lisp is beneficial to thoroughly understanding the pro-vided code examples.

4.1. A Percussion Mallet Symbol: Wirebrush

As our first exercise we create a simple percussion malletsymbol denoting a wirebrush (see Figure 3).

We begin the expression definition by providing aunique class name, ’wirebrush’ (1). In this case thereis no need for any additional inheritance other than thebasic expression behavior. Thus the default value (’ex-pression’) displayed in the inheritance menu is sufficientfor our purposes (2). The menu category is chosen to be’User’ (3). Print symbol and font typeface, in this exam-ple, do not have any use since the expression in questionis purely graphical (4). In music notation these kinds ofinstructions are usually written above the staff [4] and theprint position is selected accordingly (5). In code viewthere are three lines containing some drawing instructions(6). Finally, the graphical representation of the expressioncan be seen in the preview score (7). The second note inthis example is added to demonstrate how the user-definedexpression behaves when the associated notational objectis transposed. Note, how the expression avoids collidingwith music notation.

Figure 3. The wirebrush mallet symbol created with ED.

4.2. A Playing Technique: Scrape

The second ED example focuses on a graphic expressionthat can be used to denote scraping an instrument [5]. Aswe draw the scraping symbol in the score it is requiredhave knowledge about the length or extent of the asso-ciated notational object (Figure 4). This information isprovided with the special variable calledwidth .

Next, we briefly describe the steps needed to define thescraping expression. The class name and inheritance areas shown in (1) and the menu category is again chosen tobe ’User’ (2). In this case, it is required to define the printsymbol and font typeface since the expression containssome written instructions along with the graphical infor-mation (3). The expression print position is ’above-staff’

Figure 4. A playing technique expression created withED.

as indicated in (4). The lisp code part is once again offeredwithout any detailed explanation (5). Suffice to say thatthe x and y -shapes are first defined and then drawn witha graphics primitive calleddraw-2d-shape . Addition-ally, the user defined print symbol is also drawn. Finally,the preview score (6), demonstrates how this particular ex-pression reacts to the duration of the associated notationobject due to thewidth -parameter (see code view).

5. CONCLUSIONS AND FUTURE WORK

The current version of ED is already quite functional andcan be used to quickly prototype and create new expres-sions.

Some of the benefits in this approach are that it does notrequire extensive programming experience and the limitedset of graphics primitives is quite easy to learn. It allows tocreate new expressions without the need to reproduce thelow-level lisp code for each one. The number of lines ofcode the user is required to write is reduced to a fractioncompared to the entirely text based approach. Further-more, the synchronized visual feedback offered by ED isa valuable property as it allows the user to design and viewthe expressions in their native context.

There are, however, several additional features and en-hancements planned to make ED more useful as a generalpurpose tool: First, it is currently not possible to creategroup expressions. Group expressions require more op-tions and possibly a dynamic property view. There arealso a handful of additional methods that are user defin-able which in turn require more code view type of inputs.A dedicated group expression tab would be one solution tolay out the ED Window to facilitate this feature. Second,it should also be possible to use graphical tools for draw-ing the expression definition. This feature would require agraphics canvas and a set of graphical tools, such as lines,polygons and circles. It would also bring the ED closer tothe more traditional tools offered by, for example, Finale.However, allowing to combine both algorithmic and ’handdrawn’ representations would make this approach more

flexible. Third, there is currently no means to add slotsto the user defined expression classes through ED. Thisis of primary importance when designing more complexobjects that, for example, have different kinds of graphi-cal representations depending on attributes selected by theuser. Finally, although ED currently creates the menus forinserting the expression in the music notation there shouldbe the possibility in ED to define the appropriate contextsensitive menus for the expression itself. This would re-quire a dedicated menu editor to be embedded in ED.

The ED project was launched only recently and as canbe seen in the missing features list above there is still quitea lot of work to be done. The current version is more of aproof of concept at this time. It requires some redesigningand additional coding to integrate all the listed features inED. However, the fully mature application should proveto be beneficial not only for casual users but also for themore experienced ones.

6. ACKNOWLEDGMENTS


7. REFERENCES

[1] Finale User Manual.

[2] Sibelius3 User Guide.

[3] Kuuskankare, M. and M. Laurson. ”ENP2.0A Music Notation Program Implemented inCommon Lisp and OpenGL”,Proceedings ofthe International Computer Music Confer-ence, Gothenburg, Sweden, 2002.

[4] Read, G. Music Notation. Victor GollanczLtd., 1982.

[5] Risatti, H. New Music Vocabulary. A Guideto Notational Signs for Contemporary Music.Univ. of Illinois Press, Urbana, 1973.

Publication [P7]

[P7] M. Kuuskankare and M. Laurson. ENP Note Head Designer. In Journees

d’Informatique Musicale, Paris, France, 2005.

127

ENP NOTE HEAD DESIGNER

Mika KuuskankareDepartment of Doctoral Studies

in Musical Performance and ResearchSibelius Academy

[email protected]

Mikael LaursonCentre for Music and Technology

Sibelius AcademyFinland

[email protected]

ABSTRACTThis paper presents a new visual editor called ENP NoteHead Designer (henceforward ND). ND is aimed at as-sisting both novice and expert users to design custom noteheads in ENP. Both visual and text-based interfaces areprovided. Using the latter approach it is possible to takeadvantage of the power of Common Lisp, CLOS, andOpenGL (the base languages of ENP). The former ap-proach, in turn, allows to use a set of graphical tools todefine the note heads visual appearance. It is also possibleto mix these approaches. Furthermore, the note heads areviewed and designed in an actual musical context. Theseconcepts make the current system unique compared to theexisting approaches.

1. BACKGROUND

ENP Note Head Designer (ND) is part of an increasingcollection of visual tools inside ENP [3]. Currently, thesetools include (1) Color Editor, (2) Expression Designer,and (3) Note Head Designer. The purpose of these toolsis to provide a visual interface for constructing complexmusic notation related CLOS objects and to provide assis-tance to those users that do not possess the needed pro-gramming skills to use the strictly text-based approach. Itis important to realize, however, that the visual approach isnot exclusively for the unexperienced users. Some objectsmay be realized more easily by using a purely graphicalapproach while the others may be more suitable to be de-fined using a programming language. Thus, a text-basedapproach is provided along with the graphical one.

Uniform design principles have been applied to allaforementioned tools. The tools share same kinds of ed-itors, components and functionality which makes themeasier to learn. Most editors also contain an interactivefeedback view. This makes it possible to design objectsin their native environment, e.g, the designer sees the notehead as it would appear in the score. Furthermore, vi-sual synchronized feedback is one of the most importantdesign concepts behind ENP. Every applicable GUI oper-ation behaves this way and thus all ENP tools share thesame design principles.

The use of visual editors in our case has several advan-tages. First, the underlying CLOS syntax can be hiddento allow the user to concentrate on the task in hand (e.g.,

designing note heads). Second, following the purely text-based scheme involves routinely repeating several manda-tory low level program components (i.e., class definitions,specialized method definitions, etc.). This kind of repet-itive work load is considerably reduced. Finally, this ap-proach is also less prone to errors since the syntax andmost of the underlying CLOS code is handled automati-cally.

Most of the existing commercial notation programs,such as Finale[1] or Sibelius[8], provide some kind of in-terface for the user to change the appearance of the noteheads. The most fundamental and the most frequent ap-proach is that the user can select a note head character anda font typeface. On the other hand, Finale additionallyprovides some graphical tools allowing to define the notehead appearance by drawing arbitrary shapes in a graphicscanvas. However, in this case the note heads are designedwithout the presence of any musical context. In CommonMusic Notation (CMN, [11]) it is also possible to definenote heads algorithmically by using Common Lisp. Fi-nally, LilyPond [9] allows to change the note head typesalgorithmically, by using Scheme programming language.

ND is a visual tool that allows to design and view user-definable note heads in ENP. ND provides an approachthat mixes both the power of textual programming and theuse of traditional graphical tools. All the needed programcomponents and user-interface components (e.g., menus)are prepared automatically by ND. The note heads are alsoviewed and designed in a musical context. Moreover, thedesigned note heads are ready to be used in the currentENP session and can be saved in a file to be loaded eitherautomatically or by demand.

In the following sections we first describe the note headscheme used in ENP. Then we present the ND windowand its components. After this we examine in detail theprogramming interface. Finally, we give a comprehensiveset of examples prepared with the help of ND. The paperends with some concluding remarks.

2. THE ENP NOTE HEAD SCHEME

In this Section we describe the internal representation ofnote heads in ENP. We enumerate the different types ofnote heads and also give examples of their visual counter-parts. Internally the note heads can be represented in the

following ways:

(1) Strings or characters. In this case the object is usu-ally the literal representation of the note head, i.e., thestring or character is drawn as is using the default mu-sic notation font. The set of standard ENP note headsare defined in this way. Figure 1 gives a set of somecharacter-based note heads in ENP.

Figure 1. Character-based note heads in ENP.

(2) Keywords act as symbolic references to specificCLOS methods that, in turn, draw the note heads ac-cording to the provided code. This approach is suit-able for more complex and usually, but not necessar-ily, static note heads, which cannot usually be rep-resented with a simple character or string. Figure 2gives an example of one such note head, denoting atambura effect used in guitar music.

Figure 2. A tambura note head.

(3) Objects. When note heads are expressed as objectsevery note head can have a different visual appear-ance and an individual set of data and properties. Thisallows, for example, to use different kinds of user ed-itable shapes as note heads.

The internalbpf note headis an example of this kindof an object (see Figure 3). In this case the note headis represented as a break-point function. It can be usedfor several different purposes, among others, to de-fine the visual appearance of the note head itself byproviding the note head shape or allowing the user todefine one, or to contain control information for play-back (e.g., envelope). The break-point function is alsofully editable through the GUI.

Figure 3. Two bpf note heads. The latter one is in an ed-itable state with the movable break-points shown as blackrectangles.

The note heads constructed with ND usually belong tothe category (1) or (2). The third option (note head ob-jects) relies on inheritance and this is not yet supported byND.

3. THE ENP NOTE HEAD DESIGNER

There are only a handful of things that need to be de-fined in order to create a new note head using ND. Therequired minimum is: Unique name, which in this case isa Lisp keyword (e.g., :tambura, :sprechstimme, etc.), andthe note head definition code in Lisp and/or hand drawngraphics composed out of the provided graphical prim-itives. All additional code and the UI components, i.e,classes, methods, and contextual menu entries, are pro-vided automatically by ND.

3.1. The Components of the Note Head Designer

ND is represented by an ND window, which, in turn, con-tains several specialized components. As can be observedin Figure 4, on the top of the window there is a proper-ties view (1) containing inputs (e.g., note head name andfont). Next there is a code view (2) that can be used toenter the note head definition in Lisp. At the bottom ofthe window there is a graphical tool palette (3), and a pre-view score/graphics canvas (4). The preview score is fullyeditable. Additionally, there are also some push buttons todeal with the preview score and the note head definitionsource code.

The ruler above the staff entitled ’width’ shows the ex-tent of the note. This can be used as a visual aid whendrawing the note head definition by hand.

Figure 4. The components of the ND window. (1) theproperties view, (2) the code view, (3) the graphical tools,and (4) the preview score/graphics canvas. The ruler (la-beled ’width’) showing the extent of the note is displayedabove the staff. Also, (5) shows the stem of the chord thedesigned note is associated with.

There are basically two ways to define a note head inED. First, by using the text-based or code-based approachthe user can define the the note heads appearance with thehelp of the provided set of drawing commands and spe-cial variables. Second, drawing by hand using a specialgraphics canvas and a set of graphical tools.

3.2. Code View

In the code view the user can write standard Lisp code(see Figure 4). The users can utilize any of the OpenGL[12] functions provided by LispWorks. Additionally, ENPprovides also a set of graphical primitives, e.g., draw-2D-arrow, draw-2D-box, draw-2D-bracket, draw-2D-circle,draw-2D-line, draw-2D-lines, draw-2D-point, draw-2D-polygon, draw-2D-quads, draw-2D-shape, draw-2D-text,draw-2D-texture, draw-2D-triangles, etc.

To be able to use these primitives in a meaningful way,some knowledge about the musical context of the notehead is needed. Access to this information is providedthrough some special variables and objects which are enu-merated in the following list:

(1) x andy . These variables give the corresponding po-sition of the note head in the horizontal and verticalaxis (see Figure 5).

(2) notehead . A character or string defined by the user.

(3) width . This variable represents the length or extentof the notehead. As can be seen in Figure 5 this ismainly determined automatically according to the du-ration of the associated note. It can also be edited bythe user through the GUI or algorithmically.

(4) height . Some note heads (e.g., bpf note head) pro-vide information as to their height. For the others thisattribute is set to 1.0. This parameter is usually user-editable through the GUI.

(5) stem-down-p flag is true if the chord containingthe note has the stem downward.

(6) pitch gives the midi value of the note. It can alsocontain a fractional part in case of micro-intervals.

(7) augmentation . Optional augmentation (e.g.dots). These are calculated automatically by ENP ac-cording to the duration/rhythm of the note.

(8) self . This variable gives the actual note object.There are several attributes (in addition to the onesgiven above) that can be read directly from it. In mostcases it is not required to access any information inthis way. It is nevertheless provided for advanced ap-plications.

Figures 6 and 7 illustrate the use of thewidth param-eter. The two lines of code in the code view both draw asingle line segment. The first line segment is drawn from{0.0, 0.0} to a fixed end point at{2.0, 2.0}. The end points

Figure 5. The attributes of ENP note heads: (1) x posi-tion, (2) y position, and the widths of two notes of differ-ent lengths (3) and (4).

of this line are always drawn relative to the position of thenote head. The second line of code, on the other hand,draws a line segment from{0.0, 0.0} to a variable endpoint at{width, 2.0}. The behavior of these two line seg-ments can be observed by comparing the preview scoresfound in Figures 6 and 7. In the latter score the measureis considerably wider than in the former one, thus givingmore room to the note. Notice, how the shorter line seg-ment preserves its length and position but the longer oneadjusts itself according to the new width of the note.

Figure 6. Two line segments drawn in the ND canvas.Both have one fixed end point at{0.0, 0.0}. The formerone (1) has a fixed end point at{2.0, 2.0} and the latterone (2) has a variable end point at{width, 2.0}.

Figure 7. The variable end point{width, 2.0} of thelonger line segment dynamically follows the width of thenote head.

3.3. Graphical Tools

In ND there is also a collection of graphical tools thatcan be used to compose the visual appearance of the noteheads. The currently available set of tools consist of thefollowing:

(1) Line segments, line strips, arrows, filled or framedpolygons, and circles. These tools are quite self ex-planatory. In general they behave as the ones foundin any graphically oriented software. There are, how-ever, some differences. In case of most of the primi-tives the user can, for example, add and delete points(or vertices) to form more or less complex shapes.

(2) Text. This tool allows to add textual information. Theuser can select the color, transparency, size, and fontface. The currently available font faces are Times andSonata.

(3) Line thickness, and stipple can be set to most ob-jects. There is a set of predefined stipple patterns andthe user can also enter any arbitrary pattern (a 16-bitpattern determines which fragments of a line will bedrawn).

(4) Color and transparency. Color and transparency val-ues can be chosen from a palette of some predefinedcolors and levels of transparencies. Additionally, anycolor provided by the system color chooser dialog canbe used.

(5) Textures can be applied to polygons. Currently thetexture coordinates automatically map to the vertexcoordinates of the polygon (see [12] for further refer-ence on texture coordinates).

In the following (see Figures 8 and 9) we give an exam-ple how the ND graphics canvas behaves. The note headappearance is always drawn relative to the origo, i.e., thex/y coordinate of the note head. If the note head of thechord containing the note head is transposed or otherwisedisplaced in the score the graphical objects follow accord-ingly. This allows to check the behavior of the note headin several different positions.

Figure 8. The ND graphics canvas shows some graphicalobjects with the editable handles shown: (1) polygon, (2)line strip, (3) circle, (4) text, and (5) filled triangle. Theorigo of the drawing canvas is given in (6). All the graph-ical objects are drawn relative to this point.

4. SOME NOTE HEAD DESIGNER EXAMPLES

In this section we use ND to design several note heads.We begin by defining a simple character-based note head.Then we move to more complex cases as we define threehand drawn note heads. The first two are static ones,

Figure 9. The chord (containing the designed note head)has been manually displaced both in x and y dimensions(transposed up and dragged to the right). The graphicalobjects adjust their position accordingly.

meaning that they retain their shape regardless of theirmusical context. The third one, in turn, is a dynamic oneas it adjusts its visual appearance according to the extentof the note. As the final example we define an algorithmicnote head that graphically visualizes the production of arattling sound.

4.1. A Character-based Note Head

This approach is the most simple one. The user can definea character found in either Times or Sonata font faces tobe used as the note head. In this example we have selecteda ’ c©’ symbol to create a note head that could potentiallybe used, for example, to represent a pitch-wise constrainednote (see [6]). Figure 10 gives the corresponding ND win-dow. Although the definition of the note head itself seemstrivial there are nevertheless several things that are han-dled on the behalf the user. Among other things the draw-ing method is defined along with the needed menu com-ponents.

Figure 10. A simple character based note head createdwith ND.

4.2. A Hand-Drawn Note Head

Using the set of graphical tools described in section 3.3makes it possible to draw the note head shapes as they

would appear in the score.As our first hand drawn example we define a simple

note head composed out of some characters and an ar-row (Figure 11). This particular note head means scrapingalong the stings with the fingernail [10]. The arrow indi-cates the direction. In addition to the actual appearanceof the note head the user needs to define a unique name(’scrape’).

Figure 11. A note head with a character component andsome simple graphics.

Next we use a filled polygon tool to draw a triangleshaped note head (Figure 12). The default polygon objectcontains four vertices so one vertex is removed simply byselecting and deleting it. One of the remaining verticescan now be dragged to form a triangle shape. Our newnote head denotes an inhaling sound as used by, for exam-ple, Ligeti in ’Aventures’.

Figure 12. A static graphical note head. As the previewscore is fully editable the second note head has been in-serted in the score by copying.

The latter 1/8 note, in this example, was inserted in thepreview score to illustrate the fact that the note heads areready to be used in the current ENP session. They behaveexactly like the built-in ones and can be copied, inserted,and edited even in the preview score.

The two previous examples have dealt with relativelystatic note heads. However, it is sometimes convenientthat the note head scales itself according to the space re-served for it in the score. In ENP the note heads can bemade to fill the space between two notes or to justify itself

to some proportion of it. The Figure 13 shows an exampleof an object of this kind. The note head could be inter-preted to denote, for example, an evolving cluster (e.g.,ala Penderecki).

Figure 13. A dynamic graphical note head containingsome proportionally spaced vertices.

This note head is drawn using the filled polygon tool.The polygons, in turn, are constructed out of a set of ver-tices. A vertex can have two different modes: a normalmode and a proportional mode. Vertices that are in anormal mode have absolute x positions in the coordinatespace, i.e., a vertex placed in{1.0, 1.0} remains in thatposition unless moved explicitly by the user. The propor-tional vertices, however, calculate their x position accord-ing to the space given to the note (width ). Note, thatthe y position of both kinds of vertices is always calcu-lated relative to the y position (read: pitch) of the note,as explained above. Figure 14 gives a closer view of thisparticular example with the proportional vertices shown.

Figure 14. A note head drawn using a filled polygontool. The figure shows also some proportionally spacedvertices.

4.3. An Code Based Note Head

As our last example we use the code view to define a Lisp-based note head. We implement a note head for a ’rattling’effect as described for example in [2]. The code is used tojoin a series of Bezier curves to form a complex, evolvingcurve that completely fills the space reserved for the notehead.

Let us examine the source code in more detail (seecode view in Figure 15). First, we define the outer con-tour of the shape and define a scaling factor for each of

the Bezier curves (y-shape, y-scaling ). We ba-sically draw the same curve repeatedly in a loop eachtime with a different transformation and scaling values(see the two OpenGL macroswith-GL-translateandwith-GL-scale in the bottom of the code view).Every other curve is also flipped vertically, using a nega-tive scaling value, to smoothly join the end of each curvewith the beginning of the next one. The whole shape isscaled inside the extent of the note by dividing the widthof the note by the number of individual curves in the notehead shape (the length of they-shape ).

The preview score in Figure 15 shows the final notehead shape.

Figure 15. A complex note head shape composed out ofseveral Bezier curves.

5. CONCLUSIONS

A new visual tool called ENP Note Head Designer was in-troduced along with several workable examples. The gen-eral note head scheme of ENP was also covered. The vi-sual interface described in this paper allows to define noteheads using either a text-based or graphically oriented ap-proach. This allows users with different backgrounds inprogramming to take advantage of this tool. Furthermore,the uniform representation of note heads in ENP allowsseveral advantages. First, it is possible to use the new noteheads in the current ENP session without any compilingor reloading phase. Second, it is also possible to use thenew note heads along with ENP-score-notation, the textbased format of ENP (see [5, 7]). There is no need forany additional coding; the note heads behave as any of the

build-in ones.Finally, there are some improvements planned in ND:

(1) Usually the note heads may have slightly differentparameters and offsets depending whether they areprinted stems up or stems down. The ND windowcould potentially be organized so that there are twodifferent detail views (one for up stem and another fordown stem) and one overview which would displayboth versions at once. Currently it is not possible todistinguish between up or down stem variants otherthan using the code based approach.

(2) There should also be a possibility to group the notehead entries in the menus, much in the same way asexpressions are currently grouped [4]. This wouldmake it more convenient to write dedicated librariesfor specialized cases such as modern notation orshape-notation.

(3) There are still some useful tools missing.(a) Grouping/Un-grouping. With this tool theuser could group several objects to form a unit thatcould then be moved simultaneously. (b) Rotation.All objects should be allowed to be rotated in a 2Dspace either interactively or additionally by providinga rotation angle.

(4) There should be a collection of more dedicated graph-ical tools such as triangles, squares, etc. This wouldallow more specialized behavior, thus, when zoomed,a square would retain its shape and so on. Now all thepoints forming a polygon can be moved individually.

(5) A grid with snap-to-grid option should probably beoffered. There should be the possibility to chooseindividual vertical and horizontal grid spacings (e.g.,proportions of the width horizontally, and fractions ofthe line spacing or staff height vertically).

(6) Finally,there should also be a possibility to save thecurrent ND session. This would allow to recall a notehead in an editable form and continue to work with it.Now, only the note head definitions can be saved in afile.

6. ACKNOWLEDGMENTS


7. REFERENCES

[1] MakeMusic! Inc.Finale User Manual.

[2] Erhard Karkoschka. Das Schriftbild der NeuenMusik. Hermann Moeck Verlag, Celle, 1966.


[4] Mika Kuuskankare and Mikael Laurson. ENP-Expressions, Score-BPF as a Case Study. InPro-ceedings of International Computer Music Confer-ence, pages 103–106, Singapore, 2003.

[5] Mika Kuuskankare and Mikael Laurson. Recent De-velopments in ENP-score-notation. InSound andMusic Computing ’04, October 2004.

[6] Mikael Laurson. Recent Developments in Patch-Work: PWConstraints - a Rule Based Approach toComplex Musical Problems. InSymposium on Sys-tems Research in the Arts, Baden-Baden, 1999.

[7] Mikael Laurson and Mika Kuuskankare. FromRTM-notation to ENP-score-notation. InJourneesd’Informatique Musicale, Montbeliard, France,2003.

[8] Sibelius Software Ltd.Sibelius3 User Guide.

[9] Han-Wen Nienhuys and Jan Nieuwenhuizen. Lily-Pond, a system for automated music engraving. InXIV Colloquium on Musical Informatics (XIV CIM2003), Firenze, Italy, May 2003.

[10] Howard A. Risatti.New Music Vocabulary. A Guideto Notational Signs for Contemporary Music. Univ.of Illinois Press, Urbana, 1973.

[11] Bill Schottstaedt. Common Music Notation. InBe-yond MIDI, The Handbook of Musical Codes. MITPress, Cambridge, Massachusetts, 1997.

[12] Mason Woo, Jackie Neider, Tom Davis, and DaveShreiner. OpenGL Programming Guide. AddisonWesley, Massachusetts, USA, 3rd edition, 1999.

Documents

T EXPRESSIVE NOTATION PACKAGE - Semantic …€¦ · and an interactive Tutorial both provided as a part of the ENP program. ... It uses an implementation of ANSI Common Lisp by LispWorks