Suzanne Little - Cloud LSVA · multimedia data – a computer scientist’s view • Big data in...

Suzanne Little Dublin City University

Insight Centre for Data Analytics

Cloud Large Scale Video Analysis

H2020-ICT-2015 Cloud-LSVA

Big Data - research

Understanding high-volume, high-speed

multimedia data – a computer scientist’s view

• Big data in ADAS means manual annotation is

infeasible

• Data that’s not annotated is not usable

• Annotation via Machine Learning

• Deep learning: a step change in computer vision

• Semantics: formal meaning

~15-20 TB/day

~300 hr (2014)

Lossy content

Large amount of files

Worldwide upload points

~10-50 TB/day/vehicle

~8 hr collection window

Lossless content

Reduced amount of files

Limited upload points

ADAS Context

Open-road Acquisition

Big Data

Volume

Velocity

Variety

Storage Processing

Annotating video

Road scene Ground-truth Camera setup

Scene level

Static Objects

Dynamic Objects

Background

Actors

Vehicles

Classes

CamSeq dataset

• 101 original frames

• 101 ground truth frames

Teaching Computers to “see”

• Supervised Machine Learning uses labelled examples to train

computer systems

• Large numbers of examples are required for every concept

Annotation 4564 654654 06465 46546 54604640

4892 1894 24087 5469876 5868765

4840 6847684 68406 8484 65

41847 21098 98065 4 98 406546984

6046 84065 484 0 54 8406 4868 46

5180 2 3210684 05418 940 6541

08 405 4198 4 0 541 98 40 65 498

40 65 41 98 40 654 9840 65403

216984 06 54 98406 54 9840 6 541

98 405418 40 6 549 804 6 54 098

40 654 984

Campo dei Miracoli

La Torre di Pisa

Field of Miracles, Pisa, Italy

Leaning Tower

Pisa, Italy

Sunny day in Pisa

My holidays

magic

What about Deep Learning?

A deep convolutional network

Conv1

Input

Conv2 Conv3

Max

pooling

Max

pooling

Max

pooling

Max

pooling

Conv4 Conv5

Output

Deep Learning

• Rebranding of an old idea?

• Now possible because of powerful hardware (GPUs)

• Requires large quantities of unlabelled input

• Can require a lot of working memory

• Configuring, optimising complexity and performance is a

bit of an art at present

Semantics

• The meaning and context of annotations

• An ontology formally specifies the terms and their

relationships

• Can be used for:

– Query expansion (car is a vehicle)

– Inference (cars have doors)

– Vocabulary (sidewalk == footpath)

Object

Child

Child Child

Child

Child

Semantics

Annotation for ADAS is more than labels

Volume

Velocity Variety Veracity Value

Cloud-LSVA

Move from manual to (semi-) automatic video annotation - Scale

- Accuracy

• Aiding human annotators

• Supporting data growth

• Adapting computer vision models

• Operating on small (vehicle) and large (cloud) scale

Value

Acknowledgments

• Cloud-LSVA is funded by the EU H2020 framework under grant

number 688099.

• Specific contributions to this talk were made by Dr Kevin

McGuinness (DCU), Dr Houssem Chatbri (DCU) and Manuel Reis-

Monteiro (Valeo), Dr Marcos Nieto (Vicomtech)

• For more information on Cloud-LSVA see http://cloud-lsva.eu

• Contact: suzanne.little@dcu.ie