Syncing Audio, Video and Animations in Silverlight

Dan Wahlin

Session's Twitter Hash Tag

#MIX10Sync

Agenda

• What Problem Are We Solving?• Sync Options• Using Media Markers• Using Attached Properties• Creating Media Behaviors

What Problem Are We Solving?

Animation 1 Begins

Animation 2 Begins

Animation 3 Begins

….

Video 1 Plays

Video 1 StopsAudio 1 Plays

The End Goal

Sync Options

Empty Storyboard

Media Markers

Attached Properties

Media Behaviors

Sync Options

Empty Storyboard

Media Markers

Attached Properties

Media Behaviors

Media Markers

• Markers provide a way to raise events at specific times as audio or video plays

• Supported by Expression Encoder• MediaElement supports markers:–Markers property–MarkerReached event

• Useful for media-driven syncing of audio, video and animations

Creating Media Markers

1

3

4

2

Exporting Media Markers

5

Media Marker XML

• Marker XML file created by Expression Encoder:

<?xml version="1.0" encoding="utf-8"?><Markers> <Marker Time="00:00:02" Value="Media1" GenerateKeyFrame="True" GenerateThumbnail="False" /> <Marker Time="00:00:04" Value="Media2" GenerateKeyFrame="True" GenerateThumbnail="False" /></Markers>

Using Media Markers

• Adding to the Markers collection:void MainPage_Loaded(object sender, RoutedEventArgs e) { XDocument doc = XDocument.Load("Markers.xml"); var markers = from marker in doc.Descendants("Marker") select new TimelineMarker { Text = marker.Attribute("Value").Value, Time = TimeSpan.Parse(

marker.Attribute("Time").Value) }; foreach (var marker in markers) { this.MyMedia.Markers.Add(marker); } this.MyMedia.MarkerReached += MyMedia_MarkerReached; this.MyMedia.Play();}

Demo:Using Media Markers

Sync Options

Empty Storyboard

Media Markers

Attached Properties

Media Behaviors

Attached Properties

• Animations can target Attached Properties:

<DoubleAnimation Storyboard.TargetProperty="(Canvas.Top)" />

• Custom Attached Properties can be used to sync animations with audio/video

• As an Attached Property value changes (due to animation) an event can be raised

Creating an Attached Property

• Using DependencyProperty.RegisterAttached()

public class StoryboardTimer {

public static readonly DependencyProperty MillisecondsProperty = DependencyProperty.RegisterAttached(

"Milliseconds", typeof(double), typeof(StoryboardTimer), new PropertyMetadata(0.0, new

PropertyChangedCallback(OnMillisecondsChanged)) );

}

Animating an Attached Property

<StackPanel x:Name="LayoutRoot" sync:StoryboardTimer.Milliseconds="0">

<DoubleAnimation x:Name="MillisecondsAnimation" Storyboard.TargetName="LayoutRoot" Duration="00:00:15" From="0" To="15000" By="50" />

Storyboard.SetTargetProperty(MillisecondsAnimation, new PropertyPath(StoryboardTimer.MillisecondsProperty));

StoryboardTimer.MediaKeyFrameTriggered += MediaTimeline_MediaKeyFrameTriggered;

void MediaTimeline_MediaKeyFrameTriggered(...) { SyncMedia(e.Milliseconds); //Play or stop media}

Demo:Using Attached Properties

Sync Options

Empty Storyboard

Media Markers

Attached Properties

Media Behaviors

Media Behaviors

• Behaviors provide a flexible way to sync media with animations and perform special effects

• Supported in Visual Studio and Expression Blend

• Add new "behavior" to an existing element to extend its functionality

• Can be attached directly to a Storyboard when media needs to be synced with animations

The MediaTimelineBehavior

• MediaTimelineBehavior simplifies syncing media and animations:– Attach to Storyboard– Define media keyframes declaratively, in external

file or in code– Monitors Storyboard's current time– Automatically starts, stops or pauses media at

appropriate times based upon media keyframes

Creating the Behavior

• Creating the MediaTimelineBehavior:

public class MediaTimelineBehavior : Behavior<Storyboard> { protected override void OnAttached() { base.OnAttached(); _Storyboard = this.AssociatedObject; //Create DispatcherTimer object }

public void Begin() { //Start Storyboard and DispatcherTimer } protected override void OnDetaching() { base.OnDetaching(); //Unhook events and stop DispatcherTimer }}

Using the MediaTimelineBehavior

<Storyboard x:Name="SyncStoryboard"> <i:Interaction.Behaviors> <sync:MediaTimelineBehavior> <sync:MediaTimelineBehavior.MediaKeyFrames> <sync:MediaKeyFrame TargetName="Media1"

KeyTime="00:00:02" MediaAction="Play" /> <sync:MediaKeyFrame TargetName="Media1"

KeyTime="00:00:05" MediaAction="Stop" /> <sync:MediaKeyFrame TargetName="Media2"

KeyTime="00:00:03" MediaAction="Play" /> <sync:MediaKeyFrame TargetName="Media2"

KeyTime="00:00:06" MediaAction="Stop" /> </sync:MediaTimelineBehavior.MediaKeyFrames> </sync:MediaTimelineBehavior> </i:Interaction.Behaviors>

<!-- Storyboard Animations -->

</Storyboard>

The SlowMotionVideoBehavior

• SlowMotionVideoBehavior allows a video to be played in slow motion for a specific duration:

<MediaElement x:Name="Media1" AutoPlay="True" Volume="0" Source="Wildlife.wmv"> <ic:Interaction.Behaviors>

<sync:SlowMotionVideoBehavior x:Name="slowmotionBehavior" SlowMoStartTime="00:00:01" Duration ="00:00:30" /> </ic:Interaction.Behaviors></MediaElement>

Demo:Using Media Behaviors

Questions?

Contact Info

Blog and Codehttp://weblogs.asp.net/dwahlin

Twitter@DanWahlin

Email:dwahlin@theWahlinGroup.com