Version 2018.9 ReadMe - Avid Technologyresources.avid.com/.../MediaAnalytics_2018_9_ReadMe.pdf · detection, scene recognition, and speech-to-text conversion, by using third-party

Avid® | AI Media AnalyticsVersion 2018.9 ReadMe

Important Information

Avid® recommends that you read all the information in this ReadMe file thoroughly before setting up an Avid | AI Media Analytics environment. This ReadMe describes the required installation and configuration for running Avid | AI Media Analytics using Microsoft Azure Video Indexer. Additional information on MediaCentral | Cloud UX and MediaCentral | Asset Management is available online in guides. For more information about Azure Video Indexer, refer to Microsoft documentation.

Important: Search the Avid Knowledge Base at http://www.avid.com/learn-and-support for the most up-to-date ReadMe file, which contains the latest information that might have become available after the documentation was published.

Revision History

Contents

About Avid | AI Media Analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Installing the Media Analytics Packages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Assigning Media Analytics Rights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

(Optional) Enabling the Media Analytics Processing Trigger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

(Optional) Enabling Media Analytics Processing for Additional Asset Types. . . . . . . . . . . . . . . . . . . . 9

Creating Provider Configuration Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Triggering Media Analytics Processes by Using a Watch Folder Import. . . . . . . . . . . . . . . . . . . . . . . 16

Triggering Media Analytics Processes in MediaCentral | Cloud UX . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Monitoring Media Analytics Processes in the Process App . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Searching for Media Analytics Results in the Search App . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Displaying Media Analytics Results in the Markers Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Date Revised Changes Made

September 28, 2018 First publication

http://www.avid.com/learn-and-support#Avid-Knowledge-Base-Search

About Avid | AI Media Analytics

2


Avid | AI Media Analytics provides a framework that automates content indexing, such as facial detection, scene recognition, and speech-to-text conversion, by using third-party capabilities. The current version provides support for Microsoft Cognitive Services (Azure Video Indexer) and includes a Media Analytics API for additional providers.

Avid | AI Media Analytics v2018.9 enables MediaCentral customers to leverage Microsoft’s Cognitive Services to auto-index content using machine learning algorithms, creating a library of rich metadata that can be readily searched.

The following topics are covered in this chapter:

• Requirements

• Overview

Requirements

MediaCentral Version

The following MediaCentral versions support the Media Analytics feature:

• MediaCentral | Cloud UX v2018.9

• MediaCentral | Asset Management v2018.9

n To install and use the Avid AI Media Analytics features, MediaCentral Cloud UX and the Asset Management module must be set up and running.

For additional information, see the following guides:

• Avid MediaCentral | Cloud UX Installation and Configuration Guide

See the MediaCentral | Cloud UX v2018 documentation.

• Avid MediaCentral | Asset Management Installation Manual

See the MediaCentral | Asset Management v2018.6 and 2018.9 documentation.

Installation Packages

The Avid | AI Media Analytics installation packages are included in the MediaCentral Asset Management v2018.9 software. You do not need to download the packages separately but can directly start with the installation using the Asset Management Control Center. See “Installing the Media Analytics Packages” on page 4.

License

Using the Avid AI Media Analytics features requires the license “Asset Management Azure VideoIndexer.” Use the Avid Application Manager application to activate the license. See “Activating the License” on page 6.

http://avid.force.com/pkb/articles/en_US/readme/MediaCentral-Asset-Management-v2018-6-Documentation

http://avid.force.com/pkb/articles/en_US/ReadMe/MediaCentral-CloudUX-v2018-Documentation


3

Limitations

Note the following limitations of Avid AI Media Analytics version 2018.9:

• The current version comes with support for Microsoft Cognitive Services (Azure Video Indexer) but includes a Media Analytics API for additional providers.

• Media Analytics features can be applied to MediaCentral Asset Management assets of type “video.” You can enhance the Asset Management configuration to support additional video-based asset types. See “(Optional) Enabling Media Analytics Processing for Additional Asset Types” on page 9.

• Media Analytics does not currently support MediaCentral Production Management assets.

Microsoft Azure Dependencies

• To use the Azure Video Indexer features, you need a Microsoft Azure subscription and Video Indexer Account. Later in the installation process you will need to provide the Azure Video Indexer V2 subscription ID and account ID.

For more information, see https://docs.microsoft.com/en-us/azure/cognitive-services/video-indexer/connect-to-azure

• To use the face recognition feature, “training” of the Cognitive Services for your Video Indexer Account in the Video portal is required.

For more information, see https://www.videoindexer.ai

Included Components

After the installation of the required packages, the following is deployed:

• The service MediaAnalytics.AzureVideoIndexer is installed and added to the pool MAM_2000_Backend2

• The processes MCP_MEDIA_ANALYTICS and MCP_MEDIA_ANALYTICS_JOBS are available

• The data model includes all attributes and definitions for object classes, process classes, segment content classes, and strata that are required to use the Media Analytics features

• The scheduled task PerformAzureMediaAnalytics, which is configured to automatically trigger a MCP_MEDIA_ANALYTICS process for assets of type “video” can be enabled in Asset Management Scheduling Administrator

• The rights Processes/Create_MCP_MEDIA_ANALYTICS and CBA/MCP_MEDIA_ANALYTICS are available in the Asset Management User Management and can be assigned to user groups that are intended to use the Media Analytics features in User Manager

Overview

Setting up the Media Analytics features includes the following steps:

1. Installing the Media Analytics Packages

2. Activating the License

3. Assigning Media Analytics Rights

4. (Optional) Enabling the Media Analytics Processing Trigger

https://docs.microsoft.com/en-us/azure/cognitive-services/video-indexer/connect-to-azure

https://docs.microsoft.com/en-us/azure/cognitive-services/video-indexer/connect-to-azure

https://www.videoindexer.ai

Installing and Configuring the Avid | AI Media Analytics Features

4

5. (Optional) Enabling Media Analytics Processing for Additional Asset Types

6. Creating Provider Configuration Profiles

In MediaCentral Cloud UX, you can use the following Media Analytics features:

1. Triggering Media Analytics Processes in MediaCentral | Cloud UX

2. Monitoring Media Analytics Processes in the Process App

3. Searching for Media Analytics Results in the Search App

4. Displaying Media Analytics Results in the Markers Tab


The following topics provide information on how to install and configure the Media Analytics features:

• Installing the Media Analytics Packages

• Activating the License

• Assigning Media Analytics Rights

• (Optional) Enabling the Media Analytics Processing Trigger

• (Optional) Enabling Media Analytics Processing for Additional Asset Types

• Creating Provider Configuration Profiles

Installing the Media Analytics Packages

Use the following installation packages to install the Media Analytics features:

• Media Analytics Base (MediaAnalytics): Includes the Asset Management processes MCP_MEDIA_ANALYTICS and MCP_MEDIA_ANALYTICS_JOB that are controlled via configuration and handle dispatching, executing, and monitoring analytic jobs and writing the results to the target system.

• Media Analytics Azure VideoIndexer Integration (MediaAnalytics.AzureVideoIndexer): Includes the Asset Management WebService that handles the communication with the Azure Video Indexer API.

To install the Media Analytics packages:

1. Open the “Packages” folder on your Asset Management installation source.

2. Copy the installation packages MediaAnalytics_<version>.zip and MediaAnalytics.AzureVideoIndexer_<version>.zip to the package folder of the Control Center.

Tip: You can see the package directory by clicking the System Settings button on Control Center’s Dashboard view, selecting All from the Category selection list, and locating the system setting System.PackagesDirectory. The system setting value is the folder on the server that is running Control Center.


5

3. Open the Control Center:

a. Open a browser (Chrome or Internet Explorer) and enter the URL https://<hostname>:9911/ControlCenter

b. Provide the Control Center credentials and click Login.

4. Click Package Management on the Dashboard view or the Views side bar.

The Package Management view opens.

5. (Optional) In the Filters bar, select “Not Installed” from the State drop-down list.

6. Select “Install <version>” from the New drop-down list for the package MediaAnalytics and MediaAnalytics.AzureVideoIndexer.

7. Click Prepare.

8. In the System Settings tab, enter values for the following settings:

- MediaAnalytics.AzureVideoIndexer.AccountId: Provide your Azure Video Indexer V2 account ID. If not set, the first “Paid” account is used; if no “Paid” account of the subscription is available, the first “Trial” account of the subscription is used.

- MediaAnalytics.AzureVideoIndexer.SubscriptionKey: Provide your Azure Video Indexer V2 subscription ID.

- MediaAnalytics.AzureVideoIndexer.AzureStorageAccountKey: Provide the Azure storage account key for video upload to Video Indexer.

- MediaAnalytics.AzureVideoIndexer.AzureStorageAccountName: Provide the Azure storage account name for video upload to Video Indexer.

9. Click Start to start the installation.

The Jobs view opens and the selected packages are installed.


6

Activating the License

Use the Avid Application Manager application to activate your license. The application is installed at the same time you install Asset Management. After installation, open the Application Manager application and activate your license.

Avid supplies a license in two parts:

• System ID: A number that references the system on which the product is installed

• Activation ID: An alphanumeric string that references the product

Activating a license requires access to the Avid activation server through the Internet. If you activate the Asset Management product license, all mandatory license features are activated automatically. You must activate optional licenses, such as the Media Analytics license, separately.

To activate the Media Analytics license:

1. On the Asset Management server, click the Windows key, enter “Application Manager” in the Search box, and press Enter.

The Avid Application Manager opens.

2. Select the activation method “Online” for Asset Management Azure VideoIndexer.

The System ID and Activation ID fields are shown.

3. Enter the System ID and Activation ID that you received when you purchased the software. A “Paste” link lets you paste the number from the clipboard.

4. Click ACTIVATE to start the license activation process.

The Avid Application Manager confirms your system information. When the activation completes, a message confirms the successful activation. The ACTIVATE button changes to DEACTIVATE.


7

To view information about activated licenses, click the Details link.

5. Close the Application Manager.

Assigning Media Analytics Rights

To initiate Media Analytics processes in MediaCentral Cloud UX, users need to have assigned the corresponding user privilege:

• Processes/Create_MCP_MEDIA_ANALYTICS

To see the results of the Media Analytics process in the Markers tab in MediaCentral Cloud UX, users need to have assigned the corresponding user privileges:

• Strata/READ_MAN_ANNOTATIONS

• Strata/READ_MAN_AUDIOEFFECTS

• Strata/READ_MAN_CONTENTMODERATION

• Strata/READ_MAN_FACEDETECTION

• Strata/READ_MAN_FRAMEPATTERNS

• Strata/READ_MAN_KEYWORDS

• Strata/READ_MAN_OCR

• Strata/READ_MAN_SENTIMENTS

• Strata/READ_MAN_SHOTS

• Strata/READ_MAN_STT

If CBA rules are activated on the data model, you need to assign also the following right to user groups:

• CBA/MCP_MEDIA_ANALYTICS

Note that usage rights in Asset Management are normally handled on group level.

c Use the Asset Management User Manager to assign the above mentioned usage rights to the groups queried from the Platform User Management that are to use the Media Analytics features.


8

To assign Media Analytics rights:

1. Sign in to the Control Center.

2. Start Administrator from within Control Center.

3. Open the User Manager from within Administrator.

4. Open the Groups view.

5. Select the group to which you want to assign the Media Analytics rights in the Groups pane.

6. Click the Edit Group Rights button in the Group Rights pane.

The Edit Group Rights dialog opens.

7. Do one of the following:

t Click a right in the Available Rights box to assign the right to the group.

t Select the group whose user rights assignment is to be copied from the Sync To list.

t Click the Add All (right arrow) icon above the Available Rights box to assign all rights to the group at the same time.

8. Click OK.

The rights assignment is applied. The assigned rights are shown in the Group Rights pane.

(Optional) Enabling the Media Analytics Processing Trigger

When you install the MediaAnalytics package, the “PerformAzureMediaAnalytics” task is installed. The PerformAzureMediaAnalytics task is configured to trigger a MCP_MEDIA_ANALYTICS process if the attribute IS_PROXY_GENERATED for an asset of type Video is changed from “false” or “not set” to “true.” The task trigger is disabled by default.

Use the Asset Management Scheduling Administrator to enable the task trigger and modify its configuration, for example, if you want to add an additional video-based asset type — such as Episode, Feature, or Rushes — for which Media Analytics processes are to be triggered.

To enable the Media Analytics task trigger:



3. Open the Scheduling Administrator from within Administrator.

4. Select the task PerformAzureMediaAnalytics in the Tasks pane.

The Task Details pane shows the details of the task.

5. Select the Enabled check box.


9

6. (Optional) To change the change trigger configuration, do the following:

a. Double-click the trigger rule in the Change Triggers table.

The Trigger Rule dialog box opens. The following illustration shows the default configuration.

b. Change the trigger rule as needed and click OK.

7. Click Apply.

The task PerformAzureMediaAnalytics is shown as “Enabled” in the Tasks pane.

The task is executed according to the configured trigger.

(Optional) Enabling Media Analytics Processing for Additional Asset Types

The MediaAnalytics package provides a full configuration to run the Media Analytics processes for assets of type “video.” If required, you can enable the Media Analytics feature for other video-based asset types. By default these are:

• Archived Masterclip and Archived Sequence

• Commercial and Commercial Version

• Episode and Episode Version

• Feature and Feature Version

• Rushes

• Stock Footage

• Trailer


10

To add support for other asset types, you need to enhance the configuration using the following Asset Management administrative tools:

• Datamodel Administrator: Assign the Media Analytics-specific attributes and strata to the asset type

• System Administrator: In the BPMProcesses profile add the new asset type to the list of Allowed Object Classes

• Scheduling Administrator: Add the new video-based asset type to the task trigger configuration

To configure the asset type in Datamodel Administrator:

1. Start Asset Management Datamodel Administrator.

2. Select the asset type you want to use in the Object Classes group.

3. Assign the aspect MAM_ANALYZABLE:

a. Click the Edit button beneath the Aspects field.

b. In the Assign Aspects dialog box, select the MAM_ANALYZABLE check box and click OK.

4. Assign the attribute MEDIA_IS_ANALYZED:

a. Click the Assign button below the Attributes table.

b. In the Assign Attributes dialog box, select MEDIA_IS_ANALYZED in the left group box so that is moved to the “Attributes in <attribute name>” group box on the right and click OK.

5. Assign strata:

a. Click the Assign strata button beneath the “Strata in this object class” table.

b. In the Assign strata dialog box select the following check boxes:

- MAN_ANNOTATIONS

- MAN_AUDIOEFFECTS

- MAN_CONTENTMODERATION

- MAN_FACEDETECTION

- MAN_FRAMEPATTERNS

- MAN_KEYWORDS

- MAN_OCR

- MAN_SENTIMENTS

- MAN_SHOTS

- MAN_STT

c. Click OK.

6. Activate the data model.

To modify the Allowed Object Classes configuration:



3. Open the System Administrator from within Administrator.

4. In the Configuration tab, select the profile BPMProcesses.


11

5. Select the section MCP_MEDIA_ANALYTICS.

6. Double-click the Value field of the key AllowedObjectClasses, add the name of the new asset type separated by a comma to the list, and click outside the field.

To modify the Media Analytics task trigger:

1. Open the Scheduling Administrator from within Administrator.

2. Select the task PerformAzureMediaAnalytics in the Tasks pane.

3. Double-click the trigger rule in the Change Triggers table.

The Trigger Rule dialog box opens.

4. Click the Classes list and select the name of the new class that is to be supported. Only select an entry with prefix “Asset.”

5. Click OK to close the Trigger Rule dialog box.

6. Click Apply.

The task is executed according to the configured trigger.

Creating Provider Configuration Profiles

Before you can use media analytics feature in MediaCentral Cloud UX you need to create a provider configuration profile for each Media Analytics provider you want to integrate with MediaCentral Cloud UX. The Workflow settings app of the MediaCentral Cloud UX Administrator page lets you create and configure profiles for use with media analytics features in MediaCentral Cloud UX. Note the following:

• You can create one dedicated profile for each supported analytics provider.

• You can enable different analytics features for different providers at the same time.

• You cannot enable the same analytics feature for more than one provider at the same time.

To create a Provider Configuration profile:

1. Launch a supported web browser on a Windows or Mac client.

For more information on supported browsers, see the MediaCentral Compatibility Matrix.

2. To access the administrator apps, enter the following URL in the browser’s address bar.

https://<hostname>/admin-login

Where <hostname> is the Fully Qualified Domain Name (FQDN) of the MediaCentral Cloud UX server.

The Avid MediaCentral Cloud UX sign-in screen appears.

3. Enter your user name and password to access the MediaCentral Cloud UX user interface.

4. Click the Workflow Settings button on the toolbar.

5. Select Avid | AI Media Analytics in the Workflow Settings sidebar.

http://avid.force.com/pkb/articles/en_US/compatibility/Avid-Video-Compatibility-Charts


12

6. In the Avid | AI Media Analytics panel on the right, expand the Provider Configuration list, if required, and click the Add Provider (+) button to the right of the Provider Name column to open the settings panel.

The Video panel shows a table with supported analytics features. A column for the “New Provider” is created. All features are disabled.

The Provider Configuration panel displays a New Provider profile with controls to configure the analytics provider and supported features.


13

7. Enter the name of the providers’ service on the MediaCentral bus in the Provider Name field. For Video Indexer type: com.avid.mam.medianalytics.azurevideoindexer

8. Enter text in the Display Name field to add a custom name for this profile. While you type, the display name is updated in the Video area.

9. Enter the version of the providers’ service on the MediaCentral bus in the Version field. Default is “0.” For Video Indexer use: 0

10. Enter the realm of the providers’ service on the MediaCentral bus in the Realm field.For Video Indexer, type the ID of the Asset Management system realm (which is the same as the Asset Management System ID).

11. Specify which of the supported features should be selectable in the Video panel. All features are enabled by default.

- Annotations: annotation of the visual content

- Audio Effects: detects audio effects, such as silence

- Content Moderation: detects adult and racial content

- Face Recognition: detects faces in the video

- Frame Patterns: detects frame patterns, such as “black frames”

- Keywords: detects keywords for the audio or video content

- Metadata: provides asset metadata, such as the main language of the video

- Optical Character Recognition (OCR): detects and extracts text that is displayed in the video

- Sentiments: detects sentiment levels (positive, neutral, or negative)

- Scenes: detects scene changes

- Speech To Text (STT): creates transcripts of spoken words

If you disable a “Supported feature,” the associated option button cannot be selected in the Video panel, as shown in the following illustration.


14

12. Go to the Video panel and select the features that are to be used by your provider.

13. (Optional) In the provider profile, select or deselect the Enable check box to enable or disable the Provider profile. Each new Provider profile is enabled by default.

If you disable the Provider profile, its entry is hidden from the Video panel. In the profile configuration area, the edit fields are disabled.


t Click the Save button at the bottom of the list of profiles.

If you have created or edited multiple profiles, this button saves changes to all profiles. This button does not prompt you to confirm the action.

If you have not populated a mandatory field, you are prompted to provide the missing information.

t Click the Cancel button at the bottom of the list of profiles.

If you have created or edited multiple profiles, this button discards unsaved changes in all profiles. This button does not prompt you to confirm the action.

If you have unsaved changes and switch to another profile or section in the Workflow Settings app, you are prompted to save your work.


15

To edit a Provider Configuration profile:

1. In the Video panel enable or disable analytics features for the provider.

If you enable a feature that is already enabled for another Provider, the feature is disabled for the original provider. You can enable a feature only for one provider at the same time.

2. In the Provider Configuration area, click the provider profile menu to show its settings.

3. Edit either of the following:

t Enable or disable the provider profile.

t Change the Supported features settings.

t Edit the provider data.

4. Click Save.

The changed profile is saved.

To delete a Provider Configuration profile:

1. In the Provider Configuration area, click the trash button to the right of the Provider profile to be deleted.

The Delete Provider Configuration prompt opens.

2. Click Yes.

The Provider Configuration profile is deleted.

Working with the Avid | AI Media Analytics Features

16


The following topics provide information on how to use the Media Analytics features:

• Triggering Media Analytics Processes by Using a Watch Folder Import

• Triggering Media Analytics Processes in MediaCentral | Cloud UX

• Monitoring Media Analytics Processes in the Process App

• Searching for Media Analytics Results in the Search App

• Displaying Media Analytics Results in the Markers Tab

Triggering Media Analytics Processes by Using a Watch Folder Import

As an administrator, you can use an Asset Management watch folder to upload videos to the Asset Management database. If the PerformAzureMediaAnalytics triggered task is enabled, you start also media analytics jobs. See “(Optional) Enabling the Media Analytics Processing Trigger” on page 8.

Use the Asset Management Orchestration Monitor to monitor the progress of the video import and Media Analytics process.

To upload videos using a watch folder:

1. Open Windows Explorer.

2. Navigate to the Watch Folder directory \\<server>\MediaAssetManager\Workspace\WatchFolder\WatchServiceMamImport\ WatchFolder\VIDEO\VIDEO

3. Do one of the following, depending on the format of the video file you use:

t Paste an MP4 file in the sub-folder HIRESASPROXY

t Paste an MXF file in the sub-folder VIDEO


17

The watch folder import does the following:

- It creates an asset of type “Video” in the Asset Management database, creates a browse proxy, and starts the video analysis (shot detection) that creates the Default stratum, key frames, and extracts a thumbnail.

- If the PerformAzureMediaAnalytics triggered task is enabled, as soon as the browse proxy is created for the Video asset, the process MCP_MEDIA_ANALYTICS starts an analytics job.

Use Asset Management Orchestration Monitor to monitor the progress of the video import and Media Analytics process.

To monitor the watch folder import in Orchestration Monitor:

1. Open Internet Explorer and enter the URL https://<hostname>:9911/ControlCenter

2. Provide the Control Center credentials and click Login.

3. Click Administrator on the Control Center’s Dashboard view or Views sidebar.

4. On the Administrator login page, type your user name and password, and click Login.

5. Click the Orchestration Monitor link on the Overview page of Administrator.

Orchestration Monitor opens in a new tab.

6. On the Processes tab, use the Startup time control elements to search for processes that were started during a specific time interval.

7. Click Submit or press Enter.

The Hit list pane displays the result of your search. The following illustration shows the results of a search for processes with default settings.

- For the watch folder import, the processes MAM_IMPORT_VIDEO, MAM_CREATE_PROXY, and MAM_ANALYSE_VIDEO are triggered.


18

- As soon as the browse proxy is created (MAM_CREATE_PROXY is finished), the process MCP_MEDIA_ANALYTICS is started, which then triggers the process MCP_MEDIA_ANALYTICS_JOB.

To monitor Media Analytics in Orchestration Monitor:

1. On the Processes tab, deselect the Show only Processes with Workflows check box.

2. From the Process class list, select one of the following:

t MCP_MEDIA_ANALYTICS

t MCP_MEDIA_ANALYTICS_JOB

3. (Optional) Select a specific process life cycle from the Lifecycle list.

4. (Optional) Use the Startup time control elements to search for processes that were started during a specific time interval.

5. Click Submit or press Enter.

The Hit list pane displays the result of your search.

6. (Especially in error case) In the Hit List pane, click the toggle icon in front of a process.

The display expands and all workflow scripts that are started by script tasks of the selected process are shown.


t Click the Process class of a process to open its graphical visualization in the Process Visualizer. You can navigate through the process history in the Process Visualizer.

t Click a workflow script to open its details in the Details view.


19

Triggering Media Analytics Processes in MediaCentral | Cloud UX

By default assets are created by ingests, watch folder imports, or import processes in Asset Management. If the PerformAzureMediaAnalytics task is enabled, these actions automatically trigger the Media Analytics process. Independent from the activation status of the task, you can trigger Media Analytics processes manually for video assets in the MediaCentral Cloud UX Browse and Search app. Note the following:

• You need to be member of a user group that has the privileges to start the Media Analytics process.

• If the asset has already a browse proxy and therefore the Media Analytics process was already performed, the original analytics results will be overwritten by the new Media Analytics process.

• If the asset does not have a proxy yet, the Media Analytics process first triggers the proxy creation and then the analytics actions.

To trigger a Media Analytics process:

1. In the Browse or Search app’s Results area, select a video asset.


t Right-click and select Actions > Media Analytics.

t (Browse app only) Click the menu button and select Actions > Media Analytics.

The Media Analytics dialog box opens.

3. (Optional) Provide a main title for the process. This title is used as process name and helps you to identify the process in the Process app.

4. Click Ok.

The Media Analytics process is started. You can view the progress in the Process app.

If the process was successful, you can search for analytics results in the Search app and display time-based analytics metadata in the Markers tab.


20

Monitoring Media Analytics Processes in the Process App

Processes are displayed in the Process List, which is updated automatically upon process changes, matching the filter criteria. By default, processes of all process types currently being used are shown as a list in the Process app that is sorted by the Name column in alphabetical order. You can modify the Process List layout and process display to show only Media Analytics processes, as described in the following procedure.

Note that filters are cumulative. For example, if you apply a group-based filter and then life cycle-based filter, the life cycle-based filter is applied to the already filtered Processes List. Each filter is saved as a user setting and persists from one session to another.

To monitor Media Analytics processes:

1. Click the Process button on the Fast Bar.

2. To show only Media Analytics processes:

a. Click the Groups selector in the Quick Filter bar.

b. In the Process Groups window, expand the All Groups entry, click the Deselect All button, and select the Media Analytics and Media Analytics Job check boxes.

c. Click anywhere outside the window to apply the filter.

The Process List shows you only Media Analytics and Media Analytics Job processes.

3. To define the grouping behavior:

a. Click the Display Group button so it turns blue.

Processes are grouped by the process group property in the Process List.

b. Click the Expand/Collapse toggle button or the header of the Media Analytics and Media Analytics Job process group to show its processes.


21

4. (Optional) To apply filters:

a. To apply a creation-time filter, click the Creation Time list and select one of the time ranges “last hour,” “last 12 hours,” “today,” or “this week,” (the default value “any” disables creation time filtering).

b. To apply a life-cycle filter (Pending, Running, Finished, Error), do one of the following:

t To hide processes of a specific life cycle, click a blue life-cycle button so it turns gray.

t To show processes of a specific life cycle, click a gray life-cycle button so it turns blue.

5. Monitor the displayed processes, for example regarding progress and state changes. If an error occurs, a message is shown in the Message column.

6. (Optional) While you monitor the displayed processes, click the Refresh button to manually reload processes in the Process List.

Searching for Media Analytics Results in the Search App

Use the Search app to search for Media Analytics results and quickly access detail information. Starting with version 2018.9, you can use “time-based metadata” search options in the Search app. Note the following time-based metadata search features and limitations:

• The search engine returns results not only based on text-based metadata related to the entire asset (properties) but also related to the annotation of segments in strata (time-based metadata).

• Time-based metadata search is limited to text fields in strata and does not include time-based fields such as date and timecode.


22

• If you use more than one phrase and combine them with the “all phrases” matching option, an asset is only classified as a hit if all phrases are found in the same segment or all are found in theasset properties.

• After performing a search, the Results area in List view shows an additional Hits column, which displays the number segments in the asset. This number is related to all annotated segments and does not indicate the number of segments matching the search phrase(s).

• After running a search, the Asset Editor includes a custom tab called Hits, which provides detailed information on the matches returned through a phrase search. For time-based metadata results, a separate tab for each stratum that contains annotated segments is displayed under the Hits tab.

• For each stratum, all annotated segments with start and end timecode and text are listed. Within each stratum, you can add a filter to locate the desired segment. When you double-click a segment, the Asset Editor’s Media Viewer displays the segment’s start frame.

To search for Media Analytics results:

1. Click the Search button on the Fast Bar.

2. Enter a value in the Add Phrase field and press Enter (Windows) or Return (Mac).

A new pill is added to the Search header and the search is executed immediately. Additionally, the Results area is populated with the found assets. The Hits column displays the number of annotated segments in the asset.

3. (Optional) Enter an additional phrase in the new search pill as desired to narrow the search.

If entering more than one phrase, you can optionally customize the Matching options: Select “all phrases” or “any phrases”.


23

4. (Optional) Define a filter to create a more focused search:

a. Click the Add Filter field and select a filter from the menu.

After selecting a filter, the field is added to the Search header and a modifier appears to the right.

b. Select a qualifier from the menu.

c. Enter one or more values to the qualifier as required.

d. Press Enter (Windows) or Return (Mac) to complete the filter creation process.

The following table shows the specific Media Analytics filters and supported qualifiers:

To display and review Media Analytics results:

1. Double-click an asset in the Results area.

The asset loads into the Source Monitor of the Asset Editor. To show the time-based metadata results, for each stratum that contains annotated segments a separate tab is displayed under the Hits tab. Strata that do not contain annotated segments are not shown.

2. Click a tab to show the annotated segments of the stratum. Since the Asset Editor does not provide a horizontal scroll bar, you might need to enlarge the Asset Editor width to the left to show all available strata tabs.

3. As the segment list in a stratum can be lengthy, you can add a filter to expedite the process of locating the desired segment: Click the filter field and start typing text.

As you type, a “contains” search is conducted in the segments’ annotation. If you type more than word, a separate “contains” search for each word is conducted.

4. To access a specific position in the video, double-click a segment.

The position indicator is set to the segment’s start in the Media Viewer Timeline and the Media Viewer displays the segment’s start frame.

n Unlike when working in the Markers tab, the Media Viewer timeline does not show alternating colors for segments of a stratum selected in the Hits tab. To show segments on the Media Viewer timeline, go to the Markers tab and select the same stratum that is open in the Hits tab from the stratum selector.

Media Analytics Filter Qualifier

Analytics Completed

Contains Adult Content

Contains Racy Content

Media Analyzed

yes, no, exists, missing

Brands

Persons

Topics

exists, missing

Brand

Language

Language Code

Person

contains, begins with, end with, is, matches regex, exists, missing


24

Consider that if you have selected a stratum in the Markers tab and select another stratum in the Hits tab, the segmentation on the Media Viewer timeline refers to the Markers tab and does not reflect the segmentation of the stratum in the Hits tab.

Displaying Media Analytics Results in the Markers Tab

The results of the Media Analytics process are stored in the Asset Management database and can be displayed in the Markers tab. The results of each supported feature are shown on a separate stratum. Unlike the Hits tab, the Markers tab does not consider if a stratum is annotated but always provides access to all strata defined for the asset type.

To display Media Analytics results:

1. Double-click an asset in the Browse or Search app.

2. Click the Asset Editor’s Markers tab.

3. Click the stratum selector and select the stratum for which you want to show segments.

The Markers tab displays the segments for the selected stratum, as shown in the following example for the Speech To Text stratum.


25

Depending on the results of the analysis, segments on the stratum can be sequential, show gaps between them, or are “overlapping segments.”

- If the Media Analytics analysis results in sequential segments — the Out mark and In mark of adjacent segments are always adjacent frames — for the stratum, the Media Viewer timeline shows alternating orange and blue segments, as shown in the following example for the stratum Speech To Text.

- If the Media Analytics analysis results in segments with gaps, the Media Viewer timeline shows the base green timeline color between the alternating orange and blue segments, as shown in the following example for the stratum Face Detection.

- If the Media Analytics analysis results in overlapping segments — segments that share one or more frames in the stratum — the Media Viewer timeline shows alternating orange and blue sections that represent the individual segments. Overlapping segments are represented by a darker shade of the same color as shown in the following example for the stratum Annotations.

4. To navigate between the segments, do one of the following:

t Select a segment in the Markers tab.

The position indicator is set to the segment’s start in the Media Viewer Timeline and the Media Viewer displays the segment’s start frame.

t Select a segment in the Media Viewer Timeline.


26

The segment is also selected in the Markers tab. Note that this synchronization is not valid for overlapping segments.

t Play the video in the Media Viewer.

While the video is playing, the position indicator moves through the timeline and the corresponding segment is selected in the Markers tab. Note that this synchronization is not valid for overlapping segments.

Legal Notices

27

Legal NoticesProduct specifications are subject to change without notice and do not represent a commitment on the part of Avid Technology, Inc.

This product is subject to the terms and conditions of a software license agreement provided with the software. The product may only be used in accordance with the license agreement.

This product may be protected by one or more U.S. and non-U.S patents. Details are available at www.avid.com/patents.

This guide is protected by copyright. This guide is for your personal use and may not be reproduced or distributed, in whole or in part, without permission of Avid. Reasonable care has been taken in preparing this guide; however, it may contain omissions, technical inaccuracies, or typographical errors. Avid Technology, Inc. disclaims liability for all losses incurred through the use of this document. Product specifications are subject to change without notice.

Copyright © 2018 Avid Technology, Inc. and its licensors. All rights reserved.

The following disclaimer is required by Apple Computer, Inc.:APPLE COMPUTER, INC. MAKES NO WARRANTIES WHATSOEVER, EITHER EXPRESS OR IMPLIED, REGARDING THIS PRODUCT, INCLUDING WARRANTIES WITH RESPECT TO ITS MERCHANTABILITY OR ITS FITNESS FOR ANY PARTICULAR PURPOSE. THE EXCLUSION OF IMPLIED WARRANTIES IS NOT PERMITTED BY SOME STATES. THE ABOVE EXCLUSION MAY NOT APPLY TO YOU. THIS WARRANTY PROVIDES YOU WITH SPECIFIC LEGAL RIGHTS. THERE MAY BE OTHER RIGHTS THAT YOU MAY HAVE WHICH VARY FROM STATE TO STATE.

The following disclaimer is required by Sam Leffler and Silicon Graphics, Inc. for the use of their TIFF library:Copyright © 1988–1997 Sam Leffler Copyright © 1991–1997 Silicon Graphics, Inc.

Permission to use, copy, modify, distribute, and sell this software [i.e., the TIFF library] and its documentation for any purpose is hereby granted without fee, provided that (i) the above copyright notices and this permission notice appear in all copies of the software and related documentation, and (ii) the names of Sam Leffler and Silicon Graphics may not be used in any advertising or publicity relating to the software without the specific, prior written permission of Sam Leffler and Silicon Graphics.

THE SOFTWARE IS PROVIDED “AS-IS” AND WITHOUT WARRANTY OF ANY KIND, EXPRESS, IMPLIED OR OTHERWISE, INCLUDING WITHOUT LIMITATION, ANY WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

IN NO EVENT SHALL SAM LEFFLER OR SILICON GRAPHICS BE LIABLE FOR ANY SPECIAL, INCIDENTAL, INDIRECT OR CONSEQUENTIAL DAMAGES OF ANY KIND, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER OR NOT ADVISED OF THE POSSIBILITY OF DAMAGE, AND ON ANY THEORY OF LIABILITY, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

The following disclaimer is required by the Independent JPEG Group:This software is based in part on the work of the Independent JPEG Group.

This Software may contain components licensed under the following conditions:Copyright (c) 1989 The Regents of the University of California. All rights reserved.

Redistribution and use in source and binary forms are permitted provided that the above copyright notice and this paragraph are duplicated in all such forms and that any documentation, advertising materials, and other materials related to such distribution and use acknowledge that the software was developed by the University of California, Berkeley. The name of the University may not be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.

Copyright (C) 1989, 1991 by Jef Poskanzer.

Permission to use, copy, modify, and distribute this software and its documentation for any purpose and without fee is hereby granted, provided that the above copyright notice appear in all copies and that both that copyright notice and this permission notice appear in supporting documentation. This software is provided "as is" without express or implied warranty.

Copyright 1995, Trinity College Computing Center. Written by David Chappell.

Permission to use, copy, modify, and distribute this software and its documentation for any purpose and without fee is hereby granted, provided that the above copyright notice appear in all copies and that both that copyright notice and this permission notice appear in supporting documentation. This software is provided "as is" without express or implied warranty.

Copyright 1996 Daniel Dardailler.

Permission to use, copy, modify, distribute, and sell this software for any purpose is hereby granted without fee, provided that the above copyright notice appear in all copies and that both that copyright notice and this permission notice appear in supporting documentation, and that the name of Daniel Dardailler not be used in advertising or publicity pertaining to distribution of the software without specific, written prior permission. Daniel Dardailler makes no representations about the suitability of this software for any purpose. It is provided "as is" without express or implied warranty.

Modifications Copyright 1999 Matt Koss, under the same license as above.

Copyright (c) 1991 by AT&T.

Permission to use, copy, modify, and distribute this software for any purpose without fee is hereby granted, provided that this entire notice is included in all copies of any software which is or includes a copy or modification of this software and in all copies of the supporting documentation for such software.

http://www.avid.com/US/about-avid/legal-notices/patent-marking

Legal Notices

28

THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED WARRANTY. IN PARTICULAR, NEITHER THE AUTHOR NOR AT&T MAKES ANY REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.

This product includes software developed by the University of California, Berkeley and its contributors.

The following disclaimer is required by Paradigm Matrix:Portions of this software licensed from Paradigm Matrix.

The following disclaimer is required by Ray Sauers Associates, Inc.:“Install-It” is licensed from Ray Sauers Associates, Inc. End-User is prohibited from taking any action to derive a source code equivalent of “Install-It,” including by reverse assembly or reverse compilation, Ray Sauers Associates, Inc. shall in no event be liable for any damages resulting from reseller’s failure to perform reseller’s obligation; or any damages arising from use or operation of reseller’s products or the software; or any other damages, including but not limited to, incidental, direct, indirect, special or consequential Damages including lost profits, or damages resulting from loss of use or inability to use reseller’s products or the software for any reason including copyright or patent infringement, or lost data, even if Ray Sauers Associates has been advised, knew or should have known of the possibility of such damages.

The following disclaimer is required by Videomedia, Inc.:“Videomedia, Inc. makes no warranties whatsoever, either express or implied, regarding this product, including warranties with respect to its merchantability or its fitness for any particular purpose.”

“This software contains V-LAN ver. 3.0 Command Protocols which communicate with V-LAN ver. 3.0 products developed by Videomedia, Inc. and V-LAN ver. 3.0 compatible products developed by third parties under license from Videomedia, Inc. Use of this software will allow “frame accurate” editing control of applicable videotape recorder decks, videodisc recorders/players and the like.”

The following disclaimer is required by Altura Software, Inc. for the use of its Mac2Win software and Sample Source Code:©1993–1998 Altura Software, Inc.

The following disclaimer is required by 3Prong.com Inc.:Certain waveform and vector monitoring capabilities are provided under a license from 3Prong.com Inc.

The following disclaimer is required by Interplay Entertainment Corp.:The “Interplay” name is used with the permission of Interplay Entertainment Corp., which bears no responsibility for Avid products.

This product includes portions of the Alloy Look & Feel software from Incors GmbH.

This product includes software developed by the Apache Software Foundation (http://www.apache.org/).

© DevelopMentor

This product may include the JCifs library, for which the following notice applies:JCifs © Copyright 2004, The JCIFS Project, is licensed under LGPL (http://jcifs.samba.org/). See the LGPL.txt file in the Third Party Software directory on the installation CD.

Avid Interplay contains components licensed from LavanTech. These components may only be used as part of and in connection with Avid Interplay.

Portions © Copyright 2003-2007 of MOG Solutions.

Attn. Government User(s). Restricted Rights LegendU.S. GOVERNMENT RESTRICTED RIGHTS. This Software and its documentation are “commercial computer software” or “commercial computer software documentation.” In the event that such Software or documentation is acquired by or on behalf of a unit or agency of the U.S. Government, all rights with respect to this Software and documentation are subject to the terms of the License Agreement, pursuant to FAR §12.212(a) and/or DFARS §227.7202-1(a), as applicable.

TrademarksAvid, the Avid Logo, Avid Everywhere, Avid DNXHD, Avid DNXHR, Avid Nexis, AirSpeed, Eleven, EUCON, Interplay, iNEWS, ISIS, Mbox, MediaCentral, Media Composer, NewsCutter, Pro Tools, ProSet and RealSet, Maestro, PlayMaker, Sibelius, Symphony, and all related product names and logos, are registered or unregistered trademarks of Avid Technology, Inc. in the United States and/or other countries. The Interplay name is used with the permission of the Interplay Entertainment Corp. which bears no responsibility for Avid products. All other trademarks are the property of their respective owners. For a full list of Avid trademarks, see: http://www.avid.com/US/about-avid/legal-notices/trademarks.

Avid | AI Media Analytics ReadMe • Revised September 27, 2018 • This document is distributed by Avid in online (electronic) form only, and is not available for purchase in printed form.

http://www.apache.org/

http://jcifs.samba.org/

http://www.avid.com/US/about-avid/legal-notices/trademarks

http://www.avid.com/US/about-avid/legal-notices/trademarks

Documents

Version 2018.9 ReadMe - Avid Technologyresources.avid.com/.../MediaAnalytics_2018_9_ReadMe.pdf · detection, scene recognition, and speech-to-text conversion, by using third-party