Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos
Supervised by Prof. Oksam Chae
Md. Mehedi Hasan, 2010315443Image Processing Lab,
Department of Computer EngineeringKyung Hee University, Korea
2012.05.08
Presentation Outline
2
• Objectives • Challenges
Introduction
Contributions
Related Works
• The Proposed Video Artifacts measure and Error Frame Detection• The Proposed Spatial Error Block Analysis System
Proposed Artifact Detection and Error Pattern Analysis
Experimental Results
Conclusion and Future Work
Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos
3
Introduction
Objective• To gain a system that detect video artifacts happened not only in compression (Block
based) but also occurred during transmission or broadcasting.• To reduce the time complexity of the conventional pixel based detection methods which
requires high memory and too much computation time.• Selection of light weighted human vision measurement system and Choosing a detection
mechanism to detect the distorted frames in real-time.• Introduce a error block classification and analysis method that can be used in video
restoration, error concealment , video retrieval and many other commercial applications .
Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos
Noise and error model for broadcasting and surveillance systems
Introduction
4
Video artifacts detection and distorted pattern analysis is difficult Videos are distorted with compression , wireless transmission based and broadcasting based artifacts In image and video communication , original image and video is not accessible which is called No-reference approach, is a
challenging research issue. Compression based artifacts are sustained in a block based manner (typically , 8 by 8) but wireless transmission and
broadcasting related artifacts are not always sustained in a block based manner. A real time application that not only show the quality measure but also detect the distorted frames from videos. Classify and analyze the error patterns from defected frames that can be used in video restoration, error concealment and
retrieval.
Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos
Challenges
Introduction
Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos 5
Sample Videos and Im-ages
Courtesy: Samples provided by KBS
Proposed Method
6
• Contains Edge magnitude and direction • Less Sensitive to illumination Variation and Noise• Extract frames from videos and analyze• Incorporate Kirsch Mask to detect edge pixels.• Can detect candidate frame with high disruption in sequence of frames(Temporal Information).
• More gradient direction is analyzed for complex environment • Block classification is done in three steps.• Edge Block and Texture Content Block is analyzed .• Error Block Analysis is incorporated for better accuracy that can be used is Error concealment and restoration.
Video Artifacts Measure and Error Frame Detection
Spatial Error Block Analysis (SEBA)
Statistical Background Modeling and Multiple Motion Analysis for the Parametric Gesture Representation
7
Related Works
Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos
Artifact Measure and Error Frame Detection
Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos 8
Generate Distortion Metric
Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos 9
Further the locations of the compression block boundaries may be detected by observing where the maximum correlation value occurs.
The resulting correlation results are proposed to generate a picture quality rating for the image which represents the amount of human-perceivable block degradation that has been introduced into the proposed video signal.
Combining the results in a simple way yields a metric that shows a promising performance with respect to practical reliability, prediction accuracy, and computational efficiency.
Distortion Metric for Error Frames
Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos10
Error Frame Detection To compute the distortion measure of every frame we compare deviation with
the previous frame. If the value is within a certain threshold value then it is considered as successful undistorted frame. Otherwise it is consider as distorted frame and forwarded to next report results module.
Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos11 0
N
r msrn
F B n
Criteria Function
Deviation of
Frames
Calculate Mean of frames
Spatial Error Block Analysis
Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos12
Proposed System
Flowchart of Block Analysis
Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos13
Edge Direction Classification
Error Block Classification
Block Shape and Rotation
Formulation
Forward parameters for error
concealment
Detected Error Frame
Sobel Mask in 60Gradient direction
Magnitude and Histogram
Accum.
Convolution Mask and shift
matching
Restoration and Retrieval
Spatial Error Block Analysis
Edge Direction & Error Block Classification
Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos14
Edge Direction Classification
Uniform block: the gray level of EB may be constant or nearly so. I.e., there is no obvious edge in the block.Edge block: there are few edges passing through the block and the direction of each edge, in general, is with no or little change.Texture block: both gray level and edge direction varies significantly in the block, so the edge magnitudes of many directions are very strong.
Error Block Classification- 1
Histogram Accumulation
Error Block Classification- 2
Error Block Classification(2)Bin Reduction:
The bin reduction of histogram of gradients is used for classifying the edge blocks and texture blocks.
It also can be used for improving the speed and performance of our algorithm.
Bin: 59, 0,1, 14, 15, 16, 29, 30, 31, 44, 45 and 46 are most contributing for texture blocks
Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos15
Block Rotation and Shape Formulation
Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos16
Histogram Characteristics
Convolution Mask
Phase Offset Calculation
Block Matching andShifting
Experimental Results(1)
Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos 17
Algorithm Pearson Correlation
Spearman Correlation
Block_msr -.721 .685MGBIM [9] -.597 .584S[63] .614 .570
Algorithm Pearson Correlation
Spearman Correlation
Block_msr -.843 .838
MGBIM [9] -.727 .925S[63] .944 .937
Approaches Pearson Corr. Spearman Corr. RMSE
Wu and Yuen’s [9] .6344 .7365 7.1869Vlachos’ [65] .5378 .7930 7.0183Pan et al.’s [66] .6231 .6684 8.4497Perra et al.’s [67] .6916 .6531 8.4357Pan et al.’s [68] .5008 .6718 8.1979Muijs & Kirenko’s [69]
.7875 .6939 7.9394
Proposed Method .8627 .7104 7.0236
Pearson Correlation and Spearman for FUB database Pearson Correlation and Spearman for LIVE database
Test result using different approaches on the MPEG-2 video dataset
Experimental Results(2)
Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos 18
DATASET Wu et al.'s Pan et al.'s Mujis et al.'s ProposedSequence Recall Preci-
sionRecall Preci-
sionRecall Preci-
sionRecall Preci-
sion
LIV E : BlueSky 87.01 87.02 88.31 98.27 88.31 98.27 86.35 95.40LIV E : Pedes-trian
88.88 88.03 83.34 93.31 67.29 77.24 76.74 96.52
LIV E : RiverBed
76.58 86.50 87.57 97.57 64.28 74.89 75.54 92.26
LIV E : Rush-Hour
77.64 87.54 86.83 96.83 68.80 78.02 77.63 90.60
LIV E : ParkRun
78.08 82.05 77.35 97.32 66.20 76.23 85.47 95.49
OCN : One 69.44 89.28 77.77 93.33 63.89 79.31 83.33 96.77OCN : Mr:Big 70.23 88.67 79.41 90.94 68.56 75.42 85.58 98.11OCN : Swim 66.87 85.72 84.56 95.24 65.55 78.56 88.23 95.46
Comparison of different algorithms showing the detection rate of distorted frames
Experimental Results(3)
Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos 19
Pattern orientation calculation considering histogram bin Bin Reduction : Selection of significant histogram bins
Rotation Formulation and Bin Reduction
Histogram Accumulation of Match and Shifting
Experimental Results(4)
Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos 20
High Priority bins to take the decision: 32, (88, 90, 92) and 128 [For Matched Case, High accumulation]. Second high priority bins to take decision: (14, 15, 16) and (44, 45, 46)[Accumulation from High (full unmatched) to zero (partially matched)].
Discussion and Decision
21
We have proposed an efficient Video artifact measurement and error frame detection method- that does not restrict itself only compression based artifacts.
Major Contribution-1
Our Error block analysis algorithm is less sensitive to illumination variation and noise. Moreover, it can deal with not only traditional artifacts but also wireless transmission and broadcasting related artifacts.
Major Contribution-2
Our analysis method can formulate the distortion pattern rotation and shape- in later part which can be used in video restoration, concealment and retrieval.
Major Contribution-3
Feature WorkWe will use the analytical parameters for video error concealment. How we incorporate these
information for next step is a challenging research issue.
Conclusion and Future Work
Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos
Publication ListSCI/SCIE Indexed Journals1. Md. Mehedi Hasan, Kiok Ahn, Mahbub Murshed, Oksam Chae; “Hawkeye: A Cloud Architecture for Automated Video Error Detection in Real-
time”, INFORMATION Journal) (Accepted: 12th April, 2012) (SCIE) [ISSN: 1343-4500, E-ISSN: 1344-8994].2. Md. Mehedi Hasan, Kiok Ahn, JeongHeon Lee, SM Zahid Ishraque, Oksam Chae; “Fast and Reliable Structure-Oriented Distortion Measure for
Video Processing”, Advanced Science Letters (Accepted: 6th December, 2011) (SCIE, IF: 1.253) [ISSN: 1936-6612, E-ISSN: 1936-7317].
International Journals3. Md. Mehedi Hasan, Kiok Ahn, Oksam Chae; “Faster Detection of Independent Lossy Compressed Block Errors in Images and Videos”,
International Journal of Signal Processing, Image Processing and Pattern Recognition, vol. 5, no. 1,pp. 151-164, March, 2012)[ISSN: 2005-4254].
4. Md. Mehedi Hasan, Kiok Ahn, Oksam Chae; “Measuring Blockiness of Videos using Edge Enhancement Filtering ”, SIP, Communications in Computer and Information Science (CCIS), vol. 260, pp. 10-19, January, 2012) (Springer- Verlag, Berlin-Heidelberg)[ ISSN: 1865-0929, ISBN: 978-3-642-27182-3].
International Conference Papers5. Md. Mehedi Hasan, Kiok Ahn, Md. Shariful Haque, Oksam Chae; “Blocking Artifact Detection by Analyzing the Distortions of Local Properties
in Images ”,ICCIT 2011, 14th International Conference on Computer and Information Technology, IEEE Xplore, pp. 475-480, Dec. 22-24, 2012) [ISBN: 978-1- 61284-907-2].
6. Md. Mehedi Hasan, Kiok Ahn, SM Zahid Ishraque, Oksam Chae; “Hawkeye: Real-time Video Error Detection Using Cloud Computing Platform ”,AIM 2012, Proceedings of the FTRA International Conference on Advanced IT, engineering and Management), pp. 121-122, Seoul, Korea, Feb. 6-8, 2012).
7. Md. Mehedi Hasan, Kiok Ahn, Oksam Chae; “Measuring Artifacts of Broadcasted Videos by Accumulating Edge Gradient Magnitude ”,YSEC 2012, Proceedings of the 37th KIPS Spring Conference), Korea, April 26-28, 2012).
8. Md. Mehedi Hasan, Kiok Ahn, Mohammad Shoyaib, Oksam Chae; “Content- Based Error Detection and Concealment for Video Transmission over WLANS ”,AIM Summer 2012, Proceedings of the FTRA International Conference on Advanced IT, engineering and Management), Jeju, Korea, July 10-12, 2012) [Accepted].
9. Mahbub Murshed, SM Zahid Ishraque, Md. Mehedi Hasan, Oksam Chae; “Cloud Architecture for Lossless Image Compression by Efficient Bit-Plane Similarity Coding ”, AIM 2012, Proceedings of the FTRA International Conference on Advanced IT, engineering and Management), pp. 123-124, Seoul, Korea, Feb. 6-8, 2012).
10. Minsun Park, Md. Mehedi Hasan, Jaemyun Kim, Oksam Chae; “Hand Detection and Tracking Using Depth and Color Information ”,IPCV 2012, The 2012 International Conference on Image Processing, Computer Vision, and Pattern Recognition), Las Vegas, USA, July 16-19, 2012).
22Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos
Questions and Comments
23Edge-Based Feature Extraction for Artifacts Detection and Error Pattern Analysis from Broadcasted Videos