Mathematical Theory Mathematical Theory of Gestures in Music Guerino Mazzola U Minnesota & Zürich [email protected] [email protected] [email protected]

Mathematical Theory Mathematical Theory of Gestures of Gestures

in Musicin Music

Guerino MazzolaGuerino MazzolaU Minnesota & ZürichU Minnesota & Zü[email protected] [email protected] [email protected] [email protected] www.encyclospace.org www.encyclospace.org

DANS CES MURS VOUÉS AUX MERVEILLES DANS CES MURS VOUÉS AUX MERVEILLES J’ACCUEILLE ET GARDE LES OUVRAGES J’ACCUEILLE ET GARDE LES OUVRAGES DE LA MAIN PRODIGIEUSE DE L’ARTISTE DE LA MAIN PRODIGIEUSE DE L’ARTISTE

ÉGALE ET RIVALE DE SA PENSÉE ÉGALE ET RIVALE DE SA PENSÉE L’UNE N’EST RIEN SANS L’AUTREL’UNE N’EST RIEN SANS L’AUTRE

(Paul Valéry, Palais Chaillot)(Paul Valéry, Palais Chaillot)

DANS CES MURS VOUÉS AUX MERVEILLES DANS CES MURS VOUÉS AUX MERVEILLES J’ACCUEILLE ET GARDE LES OUVRAGES J’ACCUEILLE ET GARDE LES OUVRAGES DE LA MAIN PRODIGIEUSE DE L’ARTISTE DE LA MAIN PRODIGIEUSE DE L’ARTISTE

ÉGALE ET RIVALE DE SA PENSÉE ÉGALE ET RIVALE DE SA PENSÉE L’UNE N’EST RIEN SANS L’AUTREL’UNE N’EST RIEN SANS L’AUTRE

(Paul Valéry, Palais Chaillot)(Paul Valéry, Palais Chaillot)

LA VÉRITÉDU BEAU

DANSLA MUSIQUE

Guerino Mazzola

• MotivationMotivation- performance - and music theory- performance - and music theory- gestural music and painting- gestural music and painting- French philosophy- French philosophy- Embodied AI- Embodied AI

• SpeculumSpeculum - the musical oniontology- the musical oniontology- classification of global compositions and networks- classification of global compositions and networks

• GesturesGestures- categories of gestures- categories of gestures- hypergestures- hypergestures- the Escher theorem and free jazz- the Escher theorem and free jazz

• SymbolsSymbols- homotopy- homotopy- gestoids- gestoids- finitely generated abelian groups and networks- finitely generated abelian groups and networks





Theodor W. AdornoTheodor W. Adorno(„Zu einer Theorie der musikalischen („Zu einer Theorie der musikalischen Reproduktion“ 1946):Reproduktion“ 1946):

Danach wäre die Aufgabe des Interpreten, Danach wäre die Aufgabe des Interpreten, NotenNoten so zu betrachen, bis sie dem so zu betrachen, bis sie dem insistenten Blick in Originalmanuskripte insistenten Blick in Originalmanuskripte sich verwandeln; nicht aber als sich verwandeln; nicht aber als Bilder der Seelenregung des Autors — Bilder der Seelenregung des Autors — sie sind auch dies, aber nur akzidentiell — sie sind auch dies, aber nur akzidentiell — sondern als die seismographischen Kurven, sondern als die seismographischen Kurven, die der Körper der Musik selber in seinen die der Körper der Musik selber in seinen gestischen Erschütterungengestischen Erschütterungen hinterlassen hat. hinterlassen hat.

Gestures in Performance TheoryGestures in Performance Theory

Gestures in Performance TheoryGestures in Performance Theory

Jürgen Uhde & Renate Wieland Jürgen Uhde & Renate Wieland („Forschendes Üben“ 2002):(„Forschendes Üben“ 2002):

Affekte waren ursprünglich ja Affekte waren ursprünglich ja Handlungen, bezogen auf ein Objekt Handlungen, bezogen auf ein Objekt draussen, im Prozess der Verinnerlichung draussen, im Prozess der Verinnerlichung haben sie sich von ihrem Gegenstand haben sie sich von ihrem Gegenstand gelöst, aber immer noch sind sie bestimmt gelöst, aber immer noch sind sie bestimmt von den Koordinaten des Raumes. (...) Es von den Koordinaten des Raumes. (...) Es gibt mithin etwas wie gibt mithin etwas wie gestische (Raum-)Koordinatengestische (Raum-)Koordinaten..

David LewinDavid Lewin(„Generalized Musical Intervals(„Generalized Musical Intervalsand Transformations“ 1987):and Transformations“ 1987):

If I am at If I am at ss and wish to get to and wish to get to tt, , what characteristic what characteristic gesturegesture should I should I performperform in order to in order to arrive there?arrive there?

Musical Transformational Theory Musical Transformational Theory

Robert S. HattenRobert S. Hatten(„Interpreting Musical Gestures, („Interpreting Musical Gestures, Topics, and Tropes“ 2004)Topics, and Tropes“ 2004)

Given the importance of gesture Given the importance of gesture to interpretation, why do we not to interpretation, why do we not have a comprehensive have a comprehensive theory theory of gesture in musicof gesture in music??

Music Theory Music Theory

Cecil TaylorCecil Taylor

The body is in no way supposed The body is in no way supposed to get involved in Western musicto get involved in Western music. I try to I try to imitate on the piano the imitate on the piano the leapsleaps in space a dancer makes. in space a dancer makes.

Jazz Jazz

ll

hh

ee

soundsoundeventsevents

scorescore

analysisanalysis

instrumental-instrumental-interfaceinterface

√√thawthaw

instrumentalizeinstrumentalizeinstrumentalizeinstrumentalize

positionposition

pitchpitch

timetime

gesturesgestures

Musical theory/notation

Musical theory/notation

Instruments/playing actionInstruments/

playing action Musical soundMusical sound

Tellef Kvifte: Instruments andTellef Kvifte: Instruments andthe electronic Age.the electronic Age.

Solum forlag, Oslo, 1988Solum forlag, Oslo, 1988

The marks are made, The marks are made, and you survey the and you survey the thing like you would a thing like you would a sort of sort of graphgraph. And you . And you see within this graph see within this graph the possibilities of all the possibilities of all types of types of fact being fact being plantedplanted....

David Sylvester: Interview with Francis David Sylvester: Interview with Francis Bacon: The Brutality of FactBacon: The Brutality of Fact. . Thames and Hudson, New York 1975Thames and Hudson, New York 1975

Francis BaconFrancis Bacon

Painting Painting

Charles Alunni (1951 -):Charles Alunni (1951 -):Ce n‘est pas la règle qui gouverne l‘action Ce n‘est pas la règle qui gouverne l‘action diagrammatique, mais l‘action qui fait émerger la règle.diagrammatique, mais l‘action qui fait émerger la règle.

Jean Cavaillès (1903 - 1944):Jean Cavaillès (1903 - 1944):Comprendre est attraper le geste et pouvoir continuer.Comprendre est attraper le geste et pouvoir continuer.

Gilles Deleuze (1925 - 1995):Gilles Deleuze (1925 - 1995): Francis Bacon. La logique de la sensation. Francis Bacon. La logique de la sensation. EEditions de la Différence, Paris 1981ditions de la Différence, Paris 1981

„graph“ „graph“ „diagramme“ „diagramme“ „geste“ „geste“

Zur Anzeige wird der QuickTime™ Dekompressor „YUV420 codec“

benötigt.

Stumpy: AI Lab, U ZurichStumpy: AI Lab, U Zurich

Embodied AI Embodied AI

„„Cheap design“

Cheap design“





The Oniontology of MusicThe Oniontology of Music

FactsFactssignssigns

realit

ies

realit

ies

communicationcommunication

ProcessesProcesses

GesturesGestures

Classify!

Classify!

The category The category GloComGloComAA of of global objectiveglobal objective A-addressed A-addressed

compositionscompositions has has

objectsobjects K KII, i.e., coverings of sets K by atlases I of local objective, i.e., coverings of sets K by atlases I of local objectiveA-addressed compositions with manifold gluing conditionsA-addressed compositions with manifold gluing conditions

and manifold and manifold morphisms morphisms ff: K: KII LLJJ, , includingincluding and and

compatible with atlas morphisms compatible with atlas morphisms : I : I J J

I

IV

II

VIV

IIIVII

factsfacts

Theorem (global addressed geometric classification)Theorem (global addressed geometric classification)

Let A = locally free of finite rank over commutative ring RLet A = locally free of finite rank over commutative ring R

Consider the Consider the objectiveobjective global compositions K global compositions KII at A with (*): at A with (*):

• the chart modules R.Kthe chart modules R.Ki i areare locally free of finite ranklocally free of finite rank • the function modules the function modules (K(Kii) are) are projective projective

There is a subscheme There is a subscheme JJn* n* of a projective R-scheme of finite type of a projective R-scheme of finite type

whose points whose points : Spec(S): Spec(S)JJn* n* parametrize the isomorphism parametrize the isomorphism

classesclasses of objective global compositions at address S of objective global compositions at address SRRA with (*).A with (*).

Theorem (global addressed geometric classification)Theorem (global addressed geometric classification)

Let A = locally free of finite rank over commutative ring RLet A = locally free of finite rank over commutative ring R

Consider the Consider the objectiveobjective global compositions K global compositions KII at A with (*): at A with (*):

• the chart modules R.Kthe chart modules R.Ki i areare locally free of finite ranklocally free of finite rank • the function modules the function modules (K(Kii) are) are projective projective

There is a subscheme There is a subscheme JJn* n* of a projective R-scheme of finite type of a projective R-scheme of finite type

whose points whose points : Spec(S): Spec(S)JJn* n* parametrize the isomorphism parametrize the isomorphism

classesclasses of objective global compositions at address S of objective global compositions at address SRRA with (*).A with (*).

KKI I can be can be reconstructedreconstructed from the from the coefficient system of retracted functions on free global compositions coefficient system of retracted functions on free global compositions

res*nres*n(K(KII) ) nn((AAn*n*) )

Fact: This construction is a special case of a Fact: This construction is a special case of a local networklocal network. .

Global compositions are classified by Global compositions are classified by limitslimits of powerset of powerset denotatorsdenotators..

KKI I can be can be reconstructedreconstructed from the from the coefficient system of retracted functions on free global compositions coefficient system of retracted functions on free global compositions

res*nres*n(K(KII) ) nn((AAn*n*) )

Fact: This construction is a special case of a Fact: This construction is a special case of a local networklocal network. .

Global compositions are classified by Global compositions are classified by limitslimits of powerset of powerset denotatorsdenotators..

ŸŸ1212

ŸŸ1212

ŸŸ1212

ŸŸ1212

TT44

TT22

TT55.-1.-1 TT1111.-1.-1DD

33 77

22 44

processesprocesses

A A global networkglobal network

Local and Global Limit Denotators and the Local and Global Limit Denotators and the Classification of Global Compositions.Classification of Global Compositions.COLLOQUIUM ON MATHEMATHICAL MUSIC THEORY COLLOQUIUM ON MATHEMATHICAL MUSIC THEORY H. Fripertinger, L. Reich (Eds.) H. Fripertinger, L. Reich (Eds.) Grazer Math. Ber., ISSN 1016–7692 Bericht Nr. 347 (2005), Grazer Math. Ber., ISSN 1016–7692 Bericht Nr. 347 (2005),

Global Networks in Computer Science?Global Networks in Computer Science?http://www.encyclospace.org/talks/networks.ppthttp://www.encyclospace.org/talks/networks.ppt

TheoremTheorem Given address A in Given address A in ModMod, we have a , we have a verification functorverification functor

|?|: |?|: AALfLfModModredred

AAGlobGlob

from the category from the category AALfLfModModredred of reduced, A-addressed locally flat of reduced, A-addressed locally flat

global networks to the category global networks to the category AAGlobGlob of A-addressed global of A-addressed global compositions.compositions.

TheoremTheorem Given address A in Given address A in ModMod, we have a , we have a verification functorverification functor

|?|: |?|: AALfLfModModredred

AAGlobGlob

from the category from the category AALfLfModModredred of reduced, A-addressed locally flat of reduced, A-addressed locally flat

global networks to the category global networks to the category AAGlobGlob of A-addressed global of A-addressed global compositions.compositions.

CorollaryCorollary There are non-interpretable global networks in There are non-interpretable global networks in AALfLfModMod

redred

CorollaryCorollary There are non-interpretable global networks in There are non-interpretable global networks in AALfLfModMod

redred

Gesture Theory in Computer Music Research:Gesture Theory in Computer Music Research:

• Frédéric Bevilacqua • Claude Cadoz Claude Cadoz • Antonio Camurri Antonio Camurri • Rolf Inge GodøyRolf Inge Godøy• Stefan Müller • Norbert Schnell• Koji Shibuya• McAgnus Todd• Marcelo WanderleyMarcelo Wanderley• etc.etc.

Gesture Theory in Computer Music Research:Gesture Theory in Computer Music Research:

• Frédéric Bevilacqua • Claude Cadoz Claude Cadoz • Antonio Camurri Antonio Camurri • Rolf Inge GodøyRolf Inge Godøy• Stefan Müller • Norbert Schnell• Koji Shibuya• McAgnus Todd• Marcelo WanderleyMarcelo Wanderley• etc.etc.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Koji Shibuya‘s RyukokuRyukokuviolin robotviolin robot

• factsfacts: complete classification of : complete classification of addressed global compositionsaddressed global compositions

• processesprocesses: relative classification : relative classification of global networks via a of global networks via a functor to global compositionsfunctor to global compositions

• gesturesgestures: no mathematical theory: no mathematical theory

Mathematical Music TheoryMathematical Music Theory





Zur Anzeige wird der QuickTime™ Dekompressor „Animation“

benötigt.

aa1111x+ax+a1212y+ay+a1313z = az = a

aa2121x+ax+a2222y+ay+a2323z = bz = b

aa3131x+ax+a3232y+ay+a3333z = cz = c

aa1111 a a1212 a a1313

aa2121 a a2222 a a2323

aa3131 a a3232 a a3333

xxyyzz

aabbcc

==

rotationrotation matrix equationmatrix equation

algebra compactifies gestures to symbolic formulasalgebra compactifies gestures to symbolic formulas

„„attempt at resuscitation“attempt at resuscitation“

Peter Gabriel: Symbolic formulas via digraphs = „quiver algebras“Peter Gabriel: Symbolic formulas via digraphs = „quiver algebras“

SS

PP

TT

QQ

KK

TTXX

RRKK

RRK K = R[X]= R[X]polynomial algebrapolynomial algebra

mathematics of transformational theorymathematics of transformational theory

Graphs are only the „skeleton“ of gestures, the „flesh“ is missing.Graphs are only the „skeleton“ of gestures, the „flesh“ is missing.

??

(Local) Gesture(Local) Gesture = = morphism g: morphism g: D D of digraphs with values in a of digraphs with values in a spatial digraphspatial digraph of a topological space X of a topological space X(= digraph of continuous curves in X)(= digraph of continuous curves in X)

XX

XX

DD

positionposition

pitchpitch

timetime

XXgg

bodybody

skeletonskeleton

A gesture A gesture morphism morphism u:u: gg h is a digraph morphism u, h is a digraph morphism u, such that there is a continuous map f: X such that there is a continuous map f: X Y which Y whichdefines a commutative diagram: defines a commutative diagram:

ff

DD

EE

XX

YY

gg

hh

uu

GG((gg, , hh))category category GG of (local) gestures of (local) gestures

Advantage: Digraphs have an inherent (intuitionistic) logic, Advantage: Digraphs have an inherent (intuitionistic) logic, because the category of digraphs is a topos.because the category of digraphs is a topos.

This is not only cheap, but free design!This is not only cheap, but free design!

Advantage: Digraphs have an inherent (intuitionistic) logic, Advantage: Digraphs have an inherent (intuitionistic) logic, because the category of digraphs is a topos.because the category of digraphs is a topos.

This is not only cheap, but free design!This is not only cheap, but free design!

A A global gestureglobal gesture(only bodies shown)(only bodies shown)

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

Z Y

Xxx

zz yy

11(t(t))

66(t(t))

22(t(t))

33(t(t))

44(t(t))

55(t(t))

One hand One hand product product = = 1122334455

66

of 6 gestural curves of 6 gestural curves in space-time in space-time (x,y,z;e) of piano(x,y,z;e) of piano

j = 1, 2, ... 5: j = 1, 2, ... 5: tips of fingerstips of fingers

j = 6: j = 6: the carpus the carpus

e = timee = time

11 = = ŸŸ

Zur Anzeige wird der QuickTime™ Dekompressor „H.264“

benötigt.

Stefan MüllerStefan Müller

DD

pp

real forms?real forms?tip spacetip space

positionposition

pitchpitch

onsetonset

Renate Wieland & Renate Wieland & Jürgen Uhde:Jürgen Uhde:

Forschendes ÜbenForschendes Üben

Die Klangberührung Die Klangberührung ist das Ziel der ist das Ziel der

zusammenfassenden zusammenfassenden Geste, der Anschlag ist Geste, der Anschlag ist

sozusagen sozusagen die Geste in der Gestedie Geste in der Geste..

circlecircle

knotknot

„„loop of loops “loop of loops “

Hypergestures! Hypergestures!

DigraphDigraph((FF, , ) = ) = topological space of (local) gestures oftopological space of (local) gestures ofof digraph of digraph FF with values in a with values in a spatial digraph . spatial digraph . Notation:Notation: F F @@XXXX

XX


benötigt.

spacespace

spacespace

timetime

ET-dance gesture ET-dance gesture

EE

gg

hh

hypergesture impossible!hypergesture impossible!

gg

hh

morphism exists!morphism exists!

Gestural mapsGestural maps are particular continuous maps are particular continuous maps

(u,v):(u,v): F F @@X X G G @@YY

canonically induced by a pair of maps canonically induced by a pair of maps u: u: GG FF (digraphs)(digraphs)v: X v: X Y Y (continuous)(continuous)

The The categorycategory HHGG == HHGG11 of of hypergestureshypergestures has has

1) hypergestures as objects and1) hypergestures as objects and2) gestural maps as continuous maps.2) gestural maps as continuous maps.

The The categorycategory HHGGn n of of n-foldn-fold hypergestureshypergestures has has

1) objects: n-fold hypergestures1) objects: n-fold hypergestures g: g: FFn n FFn-1n-1@ ... @ ... FF11@ @ X X

2) (n-1)-fold gestural maps as continuous maps. 2) (n-1)-fold gestural maps as continuous maps.

Have chain of successively refined gestural categoriesHave chain of successively refined gestural categories

G G HHGG == HHGG11 HHGG2 2 ...... HHGGn n HHGGn+1 n+1 ......

which represent the granularity of gestural relations, which represent the granularity of gestural relations, much as in differential geometry, where the categories ofmuch as in differential geometry, where the categories ofn-times differentiable manifolds do.n-times differentiable manifolds do.

E.g. gluing local gestures to global gestures E.g. gluing local gestures to global gestures in quasi-anatomic joints in quasi-anatomic joints

Proposition Proposition (Escher Theorem)(Escher Theorem)Given a topological space X, a sequence of digraphs Given a topological space X, a sequence of digraphs

FF1 1 , , FF22, ..., ... FFnn

and a permutation and a permutation of 1, 2,... n. of 1, 2,... n.

Then there is a homeomorphismThen there is a homeomorphism

FF11@ @ ...... FFnn@@X X FF(1)(1)@ @ ...... F F (n)(n)@@X X

gg

hh

kk

Comprendre est Comprendre est

attraper le geste et attraper le geste et

pouvoir continuer.pouvoir continuer.





homotopichomotopiccurvescurves

XX

Gestoids:Gestoids: From Gestures to Symbols From Gestures to Symbols

0 10 1

0 1

compositioncompositionof homotopicof homotopiccurvescurvesis associativeis associative

XX

AlgebraicAlgebraicTopology

Topology

The homotopy classes of curves of a gesture gThe homotopy classes of curves of a gesture gdefine the define the GestoidGestoid G Ggg of a gesture g. of a gesture g.

This consists of the This consists of the linear combinationslinear combinations

nn a annccnn

of homotopy classes cof homotopy classes cnn of curves between given points x, y of curves between given points x, y

of gesture g.of gesture g.

yy

xx

eei2i2tt

——

ii——

11

ii

X = SX = S11

GGg g ¬ ¬ 11(S(S11))

fundamental groupfundamental group

11(S(S11) ) ŸŸ

eei2i2ntnt ~ n ~ n

~ ~ Fourier formulaFourier formula f(t) = f(t) = nn a an n eei2i2nt nt nn a ann eei2i2ntnt

g:g:

11(X) (X) ŸŸn n ? ?

finitely generated abelian groups?finitely generated abelian groups?

11(X) (X) ŸŸn n ? ?

finitely generated abelian groups?finitely generated abelian groups?


benötigt.

FF

2F2F

3F3F

4F4F

Fourier balletFourier ballet

QEDQED

ZZnn

LLn,1n,1

SS33

11

action of action of ŸŸnn

11((LLn,1n,1) ) ŸŸnn

11((SS33) ) 00

gestures gestures = ?= ?

string theory of musicstring theory of music

gestures gestures = ?= ?

string theory of musicstring theory of music

Documents

Mathematical Theory Mathematical Theory of Gestures in Music Guerino Mazzola U Minnesota & Zürich [email protected] [email protected] [email protected]