Artifacts Detection by Extracting Edge Features and Error Block Analysis from Broadcasted Videos

Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos

Supervised by Prof. Oksam Chae

Md. Mehedi Hasan, 2010315443Image Processing Lab,

Department of Computer EngineeringKyung Hee University, Korea

2012.05.08

Presentation Outline

2

• Objectives • Challenges

Introduction

Contributions

Related Works

• The Proposed Video Artifacts measure and Error Frame Detection• The Proposed Spatial Error Block Analysis System

Proposed Artifact Detection and Error Pattern Analysis

Experimental Results

Conclusion and Future Work


3

Introduction

Objective• To gain a system that detect video artifacts happened not only in compression (Block

based) but also occurred during transmission or broadcasting.• To reduce the time complexity of the conventional pixel based detection methods which

requires high memory and too much computation time.• Selection of light weighted human vision measurement system and Choosing a detection

mechanism to detect the distorted frames in real-time.• Introduce a error block classification and analysis method that can be used in video

restoration, error concealment , video retrieval and many other commercial applications .


Noise and error model for broadcasting and surveillance systems

Introduction

4

Video artifacts detection and distorted pattern analysis is difficult Videos are distorted with compression , wireless transmission based and broadcasting based artifacts In image and video communication , original image and video is not accessible which is called No-reference approach, is a

challenging research issue. Compression based artifacts are sustained in a block based manner (typically , 8 by 8) but wireless transmission and

broadcasting related artifacts are not always sustained in a block based manner. A real time application that not only show the quality measure but also detect the distorted frames from videos. Classify and analyze the error patterns from defected frames that can be used in video restoration, error concealment and

retrieval.


Challenges

Introduction

Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos 5

Sample Videos and Im-ages

Courtesy: Samples provided by KBS

Proposed Method

6

• Contains Edge magnitude and direction • Less Sensitive to illumination Variation and Noise• Extract frames from videos and analyze• Incorporate Kirsch Mask to detect edge pixels.• Can detect candidate frame with high disruption in sequence of frames(Temporal Information).

• More gradient direction is analyzed for complex environment • Block classification is done in three steps.• Edge Block and Texture Content Block is analyzed .• Error Block Analysis is incorporated for better accuracy that can be used is Error concealment and restoration.

Video Artifacts Measure and Error Frame Detection

Spatial Error Block Analysis (SEBA)

Statistical Background Modeling and Multiple Motion Analysis for the Parametric Gesture Representation

7

Related Works


Artifact Measure and Error Frame Detection


Generate Distortion Metric


Further the locations of the compression block boundaries may be detected by observing where the maximum correlation value occurs.

The resulting correlation results are proposed to generate a picture quality rating for the image which represents the amount of human-perceivable block degradation that has been introduced into the proposed video signal.

Combining the results in a simple way yields a metric that shows a promising performance with respect to practical reliability, prediction accuracy, and computational efficiency.

Distortion Metric for Error Frames

Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos10

Error Frame Detection To compute the distortion measure of every frame we compare deviation with

the previous frame. If the value is within a certain threshold value then it is considered as successful undistorted frame. Otherwise it is consider as distorted frame and forwarded to next report results module.

Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos11 0

N

r msrn

F B n

Criteria Function

Deviation of

Frames

Calculate Mean of frames

Spatial Error Block Analysis


Proposed System

Flowchart of Block Analysis


Edge Direction Classification

Error Block Classification

Block Shape and Rotation

Formulation

Forward parameters for error

concealment

Detected Error Frame

Sobel Mask in 60Gradient direction

Magnitude and Histogram

Accum.

Convolution Mask and shift

matching

Restoration and Retrieval

Spatial Error Block Analysis

Edge Direction & Error Block Classification


Edge Direction Classification

Uniform block: the gray level of EB may be constant or nearly so. I.e., there is no obvious edge in the block.Edge block: there are few edges passing through the block and the direction of each edge, in general, is with no or little change.Texture block: both gray level and edge direction varies significantly in the block, so the edge magnitudes of many directions are very strong.

Error Block Classification- 1

Histogram Accumulation

Error Block Classification- 2

Error Block Classification(2)Bin Reduction:

The bin reduction of histogram of gradients is used for classifying the edge blocks and texture blocks.

It also can be used for improving the speed and performance of our algorithm.

Bin: 59, 0,1, 14, 15, 16, 29, 30, 31, 44, 45 and 46 are most contributing for texture blocks


Block Rotation and Shape Formulation


Histogram Characteristics

Convolution Mask

Phase Offset Calculation

Block Matching andShifting

Experimental Results(1)


Algorithm Pearson Correlation

Spearman Correlation

Block_msr -.721 .685MGBIM [9] -.597 .584S[63] .614 .570

Algorithm Pearson Correlation

Spearman Correlation

Block_msr -.843 .838

MGBIM [9] -.727 .925S[63] .944 .937

Approaches Pearson Corr. Spearman Corr. RMSE

Wu and Yuen’s [9] .6344 .7365 7.1869Vlachos’ [65] .5378 .7930 7.0183Pan et al.’s [66] .6231 .6684 8.4497Perra et al.’s [67] .6916 .6531 8.4357Pan et al.’s [68] .5008 .6718 8.1979Muijs & Kirenko’s [69]

.7875 .6939 7.9394

Proposed Method .8627 .7104 7.0236

Pearson Correlation and Spearman for FUB database Pearson Correlation and Spearman for LIVE database

Test result using different approaches on the MPEG-2 video dataset



DATASET Wu et al.'s Pan et al.'s Mujis et al.'s ProposedSequence Recall Preci-

sionRecall Preci-

sionRecall Preci-

sionRecall Preci-

sion

LIV E : BlueSky 87.01 87.02 88.31 98.27 88.31 98.27 86.35 95.40LIV E : Pedes-trian

88.88 88.03 83.34 93.31 67.29 77.24 76.74 96.52

LIV E : RiverBed

76.58 86.50 87.57 97.57 64.28 74.89 75.54 92.26

LIV E : Rush-Hour

77.64 87.54 86.83 96.83 68.80 78.02 77.63 90.60

LIV E : ParkRun

78.08 82.05 77.35 97.32 66.20 76.23 85.47 95.49

OCN : One 69.44 89.28 77.77 93.33 63.89 79.31 83.33 96.77OCN : Mr:Big 70.23 88.67 79.41 90.94 68.56 75.42 85.58 98.11OCN : Swim 66.87 85.72 84.56 95.24 65.55 78.56 88.23 95.46

Comparison of different algorithms showing the detection rate of distorted frames



Pattern orientation calculation considering histogram bin Bin Reduction : Selection of significant histogram bins

Rotation Formulation and Bin Reduction

Histogram Accumulation of Match and Shifting



High Priority bins to take the decision: 32, (88, 90, 92) and 128 [For Matched Case, High accumulation]. Second high priority bins to take decision: (14, 15, 16) and (44, 45, 46)[Accumulation from High (full unmatched) to zero (partially matched)].

Discussion and Decision

21

We have proposed an efficient Video artifact measurement and error frame detection method- that does not restrict itself only compression based artifacts.

Major Contribution-1

Our Error block analysis algorithm is less sensitive to illumination variation and noise. Moreover, it can deal with not only traditional artifacts but also wireless transmission and broadcasting related artifacts.


Our analysis method can formulate the distortion pattern rotation and shape- in later part which can be used in video restoration, concealment and retrieval.


Feature WorkWe will use the analytical parameters for video error concealment. How we incorporate these

information for next step is a challenging research issue.

Conclusion and Future Work


Publication ListSCI/SCIE Indexed Journals1. Md. Mehedi Hasan, Kiok Ahn, Mahbub Murshed, Oksam Chae; “Hawkeye: A Cloud Architecture for Automated Video Error Detection in Real-

time”, INFORMATION Journal) (Accepted: 12th April, 2012) (SCIE) [ISSN: 1343-4500, E-ISSN: 1344-8994].2. Md. Mehedi Hasan, Kiok Ahn, JeongHeon Lee, SM Zahid Ishraque, Oksam Chae; “Fast and Reliable Structure-Oriented Distortion Measure for

Video Processing”, Advanced Science Letters (Accepted: 6th December, 2011) (SCIE, IF: 1.253) [ISSN: 1936-6612, E-ISSN: 1936-7317].

International Journals3. Md. Mehedi Hasan, Kiok Ahn, Oksam Chae; “Faster Detection of Independent Lossy Compressed Block Errors in Images and Videos”,

International Journal of Signal Processing, Image Processing and Pattern Recognition, vol. 5, no. 1,pp. 151-164, March, 2012)[ISSN: 2005-4254].

4. Md. Mehedi Hasan, Kiok Ahn, Oksam Chae; “Measuring Blockiness of Videos using Edge Enhancement Filtering ”, SIP, Communications in Computer and Information Science (CCIS), vol. 260, pp. 10-19, January, 2012) (Springer- Verlag, Berlin-Heidelberg)[ ISSN: 1865-0929, ISBN: 978-3-642-27182-3].

International Conference Papers5. Md. Mehedi Hasan, Kiok Ahn, Md. Shariful Haque, Oksam Chae; “Blocking Artifact Detection by Analyzing the Distortions of Local Properties

in Images ”,ICCIT 2011, 14th International Conference on Computer and Information Technology, IEEE Xplore, pp. 475-480, Dec. 22-24, 2012) [ISBN: 978-1- 61284-907-2].

6. Md. Mehedi Hasan, Kiok Ahn, SM Zahid Ishraque, Oksam Chae; “Hawkeye: Real-time Video Error Detection Using Cloud Computing Platform ”,AIM 2012, Proceedings of the FTRA International Conference on Advanced IT, engineering and Management), pp. 121-122, Seoul, Korea, Feb. 6-8, 2012).

7. Md. Mehedi Hasan, Kiok Ahn, Oksam Chae; “Measuring Artifacts of Broadcasted Videos by Accumulating Edge Gradient Magnitude ”,YSEC 2012, Proceedings of the 37th KIPS Spring Conference), Korea, April 26-28, 2012).

8. Md. Mehedi Hasan, Kiok Ahn, Mohammad Shoyaib, Oksam Chae; “Content- Based Error Detection and Concealment for Video Transmission over WLANS ”,AIM Summer 2012, Proceedings of the FTRA International Conference on Advanced IT, engineering and Management), Jeju, Korea, July 10-12, 2012) [Accepted].

9. Mahbub Murshed, SM Zahid Ishraque, Md. Mehedi Hasan, Oksam Chae; “Cloud Architecture for Lossless Image Compression by Efficient Bit-Plane Similarity Coding ”, AIM 2012, Proceedings of the FTRA International Conference on Advanced IT, engineering and Management), pp. 123-124, Seoul, Korea, Feb. 6-8, 2012).

10. Minsun Park, Md. Mehedi Hasan, Jaemyun Kim, Oksam Chae; “Hand Detection and Tracking Using Depth and Color Information ”,IPCV 2012, The 2012 International Conference on Image Processing, Computer Vision, and Pattern Recognition), Las Vegas, USA, July 16-19, 2012).

22Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos

Questions and Comments

23Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos

Documents

Artifacts Detection by Extracting Edge Features and Error Block Analysis from Broadcasted Videos