TOKYO UNIVERSITY OF SCIENCE University Research Administration Center

1-3, Kagurazaka, Shinjuku-ku, Tokyo, 162-8601, Japan E-MAIL: ura@admin.tus.ac.jp

Communicationand information

processing

2019.03

Object segmentation from videos captured by freely moving cameraObject segmentation from videos captured by freely moving camera

Takayuki HAMAMOTO (Professor, Department of Electrical Engineering, Faculty of Engineering, Tokyo University of Science)

Purpose of Research

Moving object extraction is one of the most important techniques because a wide range of applications can be expected, such as video surveillance system, etc. Previous methods have assumed that a video was captured with a stationary camera. In contrast, the proposed system allows us to extract moving objects from a video captured with a freely-moving camera. The proposed system will be applicable to a wide range of applications such as automatic digest generation of a video, etc.

By analyzing motion information (motion flow fields) acquired from a video, the proposed system separates such motion flow fields into the motions of the moving objects and those of background regions. In particular, clustering of the motion flows is performed by using a histogram of oriented gradients that is computed using the motion flow fields. Unlike the previous methods that require an analysis of long-term motion information of a video, the proposed system can discriminate the motions of the moving objects by only using the two consecutive frames. Therefore, the proposed system is able to capture the small motions of objects. Based on the results obtained by this motion field analysis, the proposed system enables us to extract regions of the moving objects using an algorithm called “graph cut,” which is a well-known method for image segmentation.

Summary of Research

Future Developments

• Explore an effective approach when the camera moves toward the forward-and-backward direction or rotates• Verify the effectiveness of the proposed system using various videos • Real time processing implementation

A novel method for extracting moving objects from a video captured with a freely-moving camera is proposed Only the two consecutive frames are used for an analysis for discriminating the motions of the moving

objects from those of background regions Small motions of objects can be differentiated

Points

Original image

Extracted moving objects

Motion vector field

Motion detection result(green: foreground; blue: background)