PianoProtégé

TableofContentsProjectstatement……………...…………………………………………..…….3

Gameplay……………………………………………………………………………4

MIDIandSongCreation...…………………………………………………….6

DataEntities……………………………………………………………………….9

FSMs…………………………………………………………………………………11

Screenshots………………………………………………………………………14

TheProject

FollowingthepopularityofgamessuchasRockBandandGuitarHero,weproposeagamethatisofthesamevisionbutadifferentflavor.Thisprogramisagameusingbeatprecisionkeystrokeswiththedifferencebeingtheinstrumentbeingemulatedisthepiano.Theselectionofsongswillrangefrombeginninglevelusingonehandandthreekeystodifficultwheretheplayerwillusebothhandsand14keystoimitatethekeystrokesofmanyclassicalandcontemporarysongsplayedonpiano.ThiswillallbeplayedeitheronthekeyboardoronaMIDIkeyboard.

TheProblem

Themulti‐milliondollarideaofsimulatinginstrumentplayingdevelopedintovideogameshashitthenationbystorm,sowhycan’titbeextendedtootherpopularinstruments?Manypeoplehavehadtheurgetoplaythepianooratleastplaysongthatusethepiano,butdon’thavetheskillofactuallyplayingbecauseofmonetarylapsesorlackofconnectionstoaproperteacher.Thisprogramallowsforactualplayerandnon‐playersofthepianoaliketohavetheopportunitytoemulateplayingsongsonthepiano.

TheImpact

Thisprogramwillallowanyonetheopportunitytoemulateplayingsongswrittenforpianowithouthavingtohaveanyrealpianoplayingskills.Itwillovertimedeveloptheplayer’ssenseofrhythmandpossiblyinfluencethemtoactuallylearntoplaypianoforreal.Theprogramwillalsoopentheplayertoalibraryofsongsthatusepianointhemainmelody.

TheMotivation

ThisprogramwasthoughofoutofamutualenjoymentofplayingthegamesRockBandandGuitarHero.Wethoughtthattheimplementationofthisgamewouldbeagoodwaytousetheskillsweallhavewithmusicandprogrammingandfindawaytomergethetwotogether.WealsowantedtoextendourknowledgeofMIDIinterfacesthroughjava.Thiswillallowforustohaveexperiencedealingwithexternalinterfaceimplementationthoughajavaenvironment.

Gameplay

ThegameplayofPianoProtégéwillbeverysimilartothegameplaysuccessfullyimplementedingameslikeGuitarHeroandRockBand.Theuserwillchooseasongtoplayfromalist;thenthesongwillbegin.Whilethesongisplaying,symbolsrepresentingnoteswillscrolldownthescreenintimetothemusic.Thehorizontalpositionofthesesymbolsdetermineswhichnoteshouldbeplayed.Whenasymbolreachesthebottomofthescreen,theusershouldplaythecorrectnoteatthattime.Ifmorethanonenoteappearsatthesameheightonthescreen,achordisrepresented,andtheusershouldplayeachofthesenotesatthesametimetoreceivecredit.Everytimetheuserplaysacorrectnoteattherighttime,theuser’sscorewillincrease.Acomboiscreatedbyplayingconsecutivenotescorrectlywithoutplayinganyextrawrongnotes.Asthecomboincreases,theuser’sscorewillincreaseatahigherrate.Whenthesongends,theuserwillbeshownthetotalscore.

DifferencefromGuitarHero

Obviously,themaindifferencebetweenPianoProtégéandGuitarHeroistheparticularmusicalinstrumentsimulatedineachgame.GuitarHerousesacustomguitar‐likecontrollertosimulateplayingaguitar.PianoProtégéwillallowtheusertouseaMIDIkeyboard,whichprovidesforamuchclosersimulationofthepiano.Also,forthepiano,twosetsofnotesareneeded–oneforeachhand.Therefore,PianoProtégéwillhavetwosectionsofnotesscrollingdownthescreenatthesametime.

ThisscreenshotfromGuitarHeroissimilartowhatPianoProtégéwilllooklike(minusthebackgroundandfireworksgraphics).Thescreenshotshowstwosectionsofnotes,whichwillbeseeninPianoProtégé.However,inGuitarHero,thetwosectionsaremeanttobeplayedbytwodifferentpeople,whereasinPianoProtégé,theleftsectionwillbeplayedbytheuser’slefthand,andtherightsectionwillbeplayedbythesameuser’srighthand.

ExplanationoftheMIDIprotocol

MIDI,ortheMusicalInstrumentDigitalInterface,isanindustrystandardprotocoldesignedtoallowmusichardwareofalltypesandbrandstocommunicatewithoneanother.Originally,theprotocolwasonlyusedasahighbaudrate,serialprotocol(muchlikeRS‐232),buteventually,itmadeitswaytothecomputerandastandardfileformatevolved.Theprotocoldefinesmanydifferentmessagesthatrelatetomusicalcontrols.Themessagesthatwewillbefocusingonarethenote‐onandnote‐offevents.

Thestructureandpurposeofthesemessagetypesareverystraightforward.Thenote‐oneventtellstheeventlistenerwhenanoteonakeyboardispressedandhowharditwaspressed(knownasthenote'svelocity)andthenote‐offeventtellstheeventlistenerwhenthatnotewasreleased.Letusbrieflyexaminethestructure:

Note­onEvent

1‐0‐0‐1X‐X‐X‐XByte1

0‐X‐X‐X‐X‐X‐X‐XByte2

0‐X‐X‐X‐X‐X‐X‐XByte3

Thenote‐oneventconsistsofthreebytes.Themostsignificanthalfofthefirstbyteisthecode1‐0‐0‐1whichisthecodeforanote‐onevent.Theleastsignificanthalf,X‐X‐X‐Xhere,isthechannelnumber.ThisvaluecanbearbitrarilychosenbyuswhenwecreatetheMIDIfilesoanexplanationofitsuseisnotimportant.Theimportantpartsarebytes2and3.Byte2isthenotevalue.Sincewehave7bitstoworkwith,itisanumberbetween0and127.Thisvaluecorrespondstothenotesofakeyboardwith0beingthelowestpitchednoteand127beingthehighestpitchednote.Thevalue60usuallycorrespondstomiddleConthekeyboard.Byte3representsthevelocityofthenote.Asexplainedearlier,thevelocityisanumberbetween1and127thattellstheeventlistenerhowhardthenotewashit.Forourpurposes,weareonlyconcernedwiththisnumberbeingaboveacertainthreshold,forexample,onlyvelocity>60constitutesanoteplayedevent.Thenote‐offeventisalmostexactlythesameasthenote‐oneventexceptthecodeis1‐0‐0‐0andthe

velocityisalways0.Thepointofthenote‐offeventinourcaseistoletusknowhowlongtheplayerisholdingdownanote.

SongConstruction

NowthatwehaveexplainedwhatMIDIis,wewillshowhowthesongswillbeconstructed.TocreatetheMIDIfiles,wewilluseaMIDIeditor.Thelayoutoftheseprogramsareverystraightforward.TheyconsistofagridofsquareswheretheX‐axiscorrespondstoamusicallydefinedquantization(1/4ofabar,1/8thofabar,etc)andtheY‐axiscorrespondstotheascendingnotesofakeyboard(alsotheascendingnotevaluesintheMIDIspecification).Morespecifically,wewillbeusingaMIDIeditorinsideamulti‐trackstudio.Whatthiswillallowustodoisimportasongassoundfile(inwav,mp3,ogg,etc)andlineitupwiththeMIDInotesthatwewanttheusertoplay.Sobasically,asong(aswedefineit)willconsistofaMIDIfileandasoundfile.Sincetheyarebothsynced,wecanuseasingletimecodeforbothfiles(thesoundfileplayingandtheMIDIfilebeingvisualizedwithourcustomdisplay).WecancreatetheMIDInotesinanumberofways,wecanplaythesongfileandhavethemulti‐trackstudiorecordsomeoneplayingthekeyboard,wecanuseaprogramtoconvertsheetmusictoMIDI,orwecanjustputitinbyhandwithamouseandthemidieditorinterface.

Hereisapictureofsongconstructioninprogress,youcanseetheactualsoundfilerepresentedbythesoundwaveontop,andtheMIDIeditorrightunderit:

ERM

Player(PlayerID,Playername,HasScore)Song(SongId,Title,Composer,NumberofNotes,Length,Tempo,Difficulty)Note(NoteId,SongId,Keys,Duration,WaitTime)Score(PlayerID,SongId,DatePlayed,Score,Placement,HighestCombo)Calculation(SongId,Score,NumberOfCorrectNotes,HighestCombo)Playing(SongId,NoteId,boolCorrect)Starting(PlayerId,SongId,difficulty)0:1PlayertoStartinglink:Aplayermayatanyonetimehaveaminimumof0songsstartingandamaximumof1songbeingstarted0:1SongtoStartinglink:Aminimumof0songsmaybestartingatanyonetimetoamaximumof1.0:nSongtoPlayinglink:ASongmayatanyonetimebeplayingaminimumof0notesandamaximumofnnotes

1:1NotetoPlayinglink:Atanyonetimeanotewillbeplayingforonlyonesong1:1SongtoCalculatinglink:Asongmayatanyonetimehave1scorebeingcalculatedatatime1:1ScoretoCalculatinglink:AScoremayatanyonetimebecalculatedforonlyonesongatatime

EndStart Main Menu

Menus and Interfaces FSM

Choose Song Menu Instructions

View Scores

Play Song Pause Menu

End of Song

Choose Song Selected

Main Menu Selected

Program Run Program Exited

Main

Men

u Se

lect

ed Instructions Selected

Song Selected

Choo

se N

ew S

ong

Sele

cted

Retr

y Se

lect

ed

Song Complete

View Scores Selected

Main Menu Selecte

d

Song Paused

Start up song from Pause

Choose New Song

Back to Main M

enu

View Scores SelectedMain Menu Selected

End Song

Any More Notes?

NoYes

Note Expected

Wait for nextNote Event

Score Note

Note correctly scored

Note FSM

Play correct note

Play correct note

Sh

ow

note

Com

bo >

= 2

0 / In

cre

ase s

cor e

Stop song

Increase score

λ

λ

No note or wrong note played

Com

bo < 10 / Increase score

λ

Show

note

Show

note

Play correct note

No n

ote

or

wro

ng n

ote

pla

yed

Pla

y so

ng

Com

bo >

= 3

0 / Incre

ase s

cor e

No note or wrong note played

No no

te o

r wro

ng n

ote

playe

d

Show

note

Wrong note played

Com

bo <

20 / In

crease

sco

reλ

Wrong note played

Wrong note played

Play correct note

Com

bo >

= 1

0 /

Incr

eas

e s

core

λ

Wrong note played

Com

bo < 30 / Increase score

q0

Start Song

q1

No Note 1x

q2

Note 1x

q3

No Note 2x

q4

Note 2x

q5

No Note 3x

q6

Note 3x

q7

No note 4x

q8

Note 4x

q9

Correct Note Played q10

Correct Note Playedq11

Correct Note Played

q12

Correct Note Played

q13

Correct Note Not Played

q14

No More Notes

q15

End Song

Screenshots

Aaron Banks

Tyler Barker

Benjamin Eckel

Aaron Tureau