978-1-4673-0174-9/12/$31.00 ©2012 IEEE 155 ICALIP2012

Sensor Pattern Noise in JPEG Compressed Images

Chenyang Shi, Yuting Su, Jing Zhang, Junyu Xu The School of Electronic Information Engineering

Tianjin University Tianjin, P. R. China

E-mail:{shichenyang,ytsu,zhangjing,xujunyu}@tju.edu.cn

Abstract

Sensor pattern noise has been used as a unique

fingerprint for various forensic tasks, including source camera identification. However, quantization noise introduced by JPEG compression may impact sensor pattern noise; the robustness of sensor pattern noise is too weak to identify compressed images. This paper proposes a new method to identify the source of JPEG compressed images based on sensor pattern noise. By introducing quantization noise model, a modified model of sensor pattern noise is made to weaken the effects of quantization noise. Experimental results demonstrate the effectiveness of this approach. 1. Introduction

In recently years, due to popularity of digital cameras and the advances of network technologies, sophisticated image editing software and wide adoptions of digital multimedia coding standards, digital multimedia applications have become increasingly popular in our daily life and digital images have been an increasingly larger share of information transmission on the Internet. However, the digital nature of the media files can now be easily manipulated, synthesized and tampered in numerous ways without leaving visible clues. As a result, the integrity of image content can no longer be taken for granted and a number of forensic-related issues arise. Verifying the content of digital images or identifying the source camera that took the image would be obviously useful for instance in the court of law, when digital pictures are presented as evidence.

To deal with digital image forensic, several methods have been proposed to identify the source of images. One of them is sensor pattern noise (SPN/ PRNU) that is unique stochastic characteristic of image sensors. Jessica Fridrich, et al. firstly proposed the method that identifies source cameras by SPN [1].

Furthermore, it is improved in [2] by combining with the consistency of demosaicing artifacts. However, the SPN noise extracted by wavelet-based denoising filter [3] is strongly influenced by the pixel values around the edges in the image. For solving this problem, [4] proposed a method to select pixels used for identification according to the texture complexity. The limitation of the current method of extracting SPN is that the SPN can be severely contaminated by scene details, therefore, [5] proposed a method to attenuate the influence of scene details on SPN by assigning weighting factor inversely proportional to the magnitude of the SPN components. [6] Cluster the pixels according to the levels of their PNU noises. The robust features of the PNU noises are obtained by calculating the average of the noise residuals in each cluster, reducing the random noise and scene content. However, the robustness of methods in [1-6] is weak for identifying compressed images. Nowadays, in order to save storage space, a large share of images is compressed. JPEG is the most commonly used graphics formats on the Internet. In addition, JPEG compression is a key post-processing operation in cameras. Therefore, the previous methods of source cameras identification are not sufficient. And it is still a hot issue that a JPEG compressed image is matched to a specific camera.

In this paper, a new method was proposed to improve source camera identification accuracy after JPEG compressed. By introducing quantization noise model and analyzing how quantization noise influences SPN, a modified model for SPN was made with the Laplacian model of quantization noise [8]. The rest of this paper is organized as follows: Section 2 describes the source camera identification algorithm. In Section 3, we discuss SPN in JPEG compressed images and introduce the modified model of sensor pattern noise in JPEG compressed images. We analyze experiment results in Section 4. Finally, conclusion and future work are given in Section 5.

156

2. Source Camera Identification Method Based on Sensor Pattern Noise

Because of the noise-like character of SPN, it is natural to detect its presence in an image using correlation [1]. To verify that a specific image P was taken with the camera C, we first determine the camera reference pattern noise Pc. The presence of the reference pattern in P will be established using correlation. 2.1. Sensor Pattern Noise

Before the digital image is produced, various noise sources degrade the image, because the real-world scene is processed by a pipeline of various camera components, such as CCD/COMS, colour filter array (CFA), demosaicing, white-balancing, automatic gain control (AGC), Gamma correction, post-processing, and JPEG compression. Some of these noise sources are temporal, some of these are spatial and others are a combination of these [7].Temporal noise consists of shot noise and flicker noise. The two main components of the pattern noise are fixed pattern noise (FPN) and photo-response non-uniformity (PRNU). Due to dark current, some of the variations are somewhat systematic: the spatial pattern of these variations remains constant. The spatial pattern forms FPN. Some middle-to high-end consumer cameras can suppress FPN automatically by subtracting a dark frame .The other pattern noise source is PRNU that is caused primarily by pixel non- uniformity. In natural images, the PRNU is the dominant part of the pattern noise, which consists of low-frequency defects, and pixel non-uniformity (PNU) noise. Low-frequency defects, due to light refraction on dust particles, optical surfaces and zoom settings are not a characteristic of the sensor. The PNU noise is unlikely even sensors coming from the same wafer [2], which is an intrinsic characteristic of the sensor.

2.2. Extracting Reference Pattern Noise

The noise residual of one image can be easily

extracted by subtracting the original image from its noise-free version.

( ) ( ) ( )( )kkk F ppn −= (1)

Where F represents a de-noising filter in [3], and

P(k) is the kth noisy image, k=1,2,…, Np . Noise residual contains not only pattern noise but also much random

noise. It can be obtained an approximate reference pattern noise by averaging multiple images. This process can suppress random noise and the scene contents.

2.3. Identification by Correlation

To determine whether a specific image P was taken

by camera C, we calculate the correlation ρc between the noise residual n of P and the camera reference pattern Pc is calculated as (2)

( ) ( ) ( ) ( )cc

cccc corr

ppnnppnnpnp

−−−•−

== ,ρ (2)

where the bar above a symbol denotes the mean value. We can now experimentally determine the set of

ρc(q) for other images q taken by C and the set of ρc(q') for images q' that were not taken by C. The distribution of two detection statistics are used to obtain the receiver operating characteristic (ROC) curves in terms of false positive rate (FPR) and true positive rate (TPR) values. In our works, the area under the ROC curve (AUC) was used to evaluate the performance of diagnostic and predictive proposed method.

3. Sensor Pattern Noise in Compressed Images

In order to save storage space, currently most cameras saved output images as JPEG format that is one of compression format. Quantization is the mainly process used in lossy image compression schemes, which will introduce quantization error that coexist with pattern noise in the noise residual of the compressed image. In this section, we describe sensor pattern noise in compressed images and in particular, focus on the modified model for pattern noise in compressed images. Our algorithm diagram flow is shown in Figure 1.

Figure 1. Camera identification flow

Obtain the camera reference pattern

noise (RPN)

Extract noise residual of compressed image

Modify noise Residual (MNR)

Calculate the correlation between RPN and MNR

Identification

157

3.1. Sensor Pattern Noise in JPEG Compressed Images

JPEG compression involving quantization of the

discrete cosine transform (DCT) coefficients will introduce quantization error to interfere with the pattern noise. At high compression ratios, quantization error produces visually undesirable compression artifacts that can dramatically lower the perceived quality of a particular image; meanwhile quantization noise may impact sensor pattern noise to decrease the accuracy of source camera identification. A simple experiment result using the method based on the pattern noise proposed by Jessica Fridrich et al in [3] is shown in Figure 2.

Figure 2. The correlations between RPN and the NR as a function of the JPEG quality factor for Nikon_D70

As can be seen from Figure 2， the correlations between RPN (reference pattern noise) and NR (the noise residual) of the compressed image decreases with the quality factor. That is why the source camera identification accuracy would become lower with the decrease of the quality factor, and the method of Jessica Fridrich proposed would fail with low quality factor. In this paper, a novel algorithm employing the quantization noise model is proposed to identify the source camera. 3.2. Modified Model of Sensor Pattern Noise in the JPEG Compressed Image

Considering the fact that the compression ratio is higher than a certain value, quantization noise may heavily impact sensor pattern noise. As demonstrated in Figure 2, the hypothesis underlying our modified model for pattern noise is that the higher a compression

ratio, the more likely that pattern noise associated with strong quantization noise, and thus the less trustworthy the component should be. In this section, we proposed a modified model for pattern noise to improve the reliability of sensor pattern noise in JPEG compression.

The first step in the JPEG compression process is to subdivide the input image into nonoverlapping pixel blocks of size 8×8, which are transformed by the DCT. And the DCT coefficients are then quantized with the standard JPEG quantization tables, which introduces quantization error. This error would interfere with the sensor pattern noise, which decreases the source camera identification accuracy. Thus, we can improve the source camera identification accuracy by eliminating the quantization noise with a Laplacian model [8] that is a spatial domain quantization noise model based on a statistical noise analysis of the error introduced by quantizing the DCT coefficients.

The modified model for sensor pattern noise can be obtained by assigning a weighting factor to attenuate the interference of quantization noise. If quantization noise is larger than sensor pattern noise, the weighting factor will be decreased as a function of JPEG quality factor; otherwise the weighting factor is one. The modified model of sensor pattern noise is formulated as:

( ) ( ) ( ) ( )( ) ( ) ( ) ( )⎩⎨⎧

<•+≥

=jikjiPjinbaQjikjiPjin

jinqc

qc

,,,,,,

,σσ (3)

where n(i,j) is the (i,j)th component of n; σq(i,j) denotes (i,j)th standard deviation of quantization error. Pc(i,j) is the (i,j)th component of Pc.

Let nq denotes quantization noise and nq(i,j) is the (i,j)th component of nq. Pc(i,j) coexist with nq(i,j) in n(i,j). When |nq(i,j)|>|Pc(i,j)|, nq(i,j) will impact Pc(i,j). And we believe that the pattern noise is not completely reliability. In order to diminish the influence of quantization noise on sensor pattern noise, we assign an attenuate factor (aQ+b) to n(i,j). nq is Gaussian distributed with zero-mean and σ q - standard deviation.

( ) ( ){ }

( ) ( ) ( ){ }( ) ( ){ }

( ) ( ) ( ){ }( ) ( )[ ]

( )[ ] α

σσ

σ

σ

=Φ−=−Φ−Φ−=

<<−−=

>=

>>≥

>

kkk

jikjinjikP

jikjinP

jiPjikjinP

jiPjinP

qqq

qq

cqq

cq

121

,,,1

,,

,,,

,,

(4)

158

Because of P(|nq(i,j)|>|Pc(i,j)|)≥α, if we let α=99.2%, k≤0.01.Verified by experiments, in this paper, we choose k=0.001.The values of a and b in (aQ+b) are decided by camera brands, and Q is JPEG compression quality factor. 4. Experiments and Results

In this section, we chose 600 photos from Dresden image database [9] including 4 cameras labeled as Cc(c=1,2,3,4). The camera brands are Nikon_D70, Nikon_D200, 5MP-9Y2 and Coach 6M. Images taken in the Nikon NEF raw format were converted into TIFF format. All images from 5MP-9Y2 and Coach 6M were collected in the JPEG format. We randomly selected 100 images from each camera and calculate the reference Pc for cameras by averaging the noise residual which was extracted through formula (1). We took another set of randomly selected 50 images, and then JPEG compressed them with different quality factors from 50 to 95 by step length that is 5 and calculated correlations with each reference Pc. Unless noted otherwise, when correlating a noise residual with a camera reference pattern of different dimensions, we cropped the larger of the two to fit the dimensions of the smaller one, and all experiments are in the gray-scale.

Figure 3. Correlation between RPN and RN /MRN as a function of the JPEG quality factor for Nikon_D70

(a) Fridrich's method

(b) Proposed method

Figure 4. Correlation between RPN from Nikon_D70 and NR/MNR of 200 JPEG compressed images (QF=70) from four cameras

Figure 3 shows the curves which describe that the correlation coefficients vary with JPEG quality factors. It is found that our proposed algorithm is more robust than Fridrich's.

In this experiment, we only chose 200 images from four cameras after JPEG compression with the quality factor 70, and then obtained correlation coefficients between the reference Pc from Nikon_D70 with noise residual of each image, the scatter chart of which were shown in Figure 4. Figure 4 (a) and (b) correspond to Fridrich's method and ours, respectively. From Figure 4, we can know that our method obtained better result than Fridrich's.

Only considering the JEPG quality factors 50, 70 and 90, we compare our method with Fridrich's in [3] based on JPEG compressed images. The ROC curves and AUC are shown as follows.

159

Figure 5. ROC curves for the comparison of two methods QF=50 Table 1. The AUC under different quality factors

Camera Brands

JPEG QF=90 Fridrich's

method Proposed method

AUC AUCNikon_D70 0.9161 0.9617

Nikon_D200 0.8801 0.9908 5MP-9Y2 0.9992 0.9997 Coach 6M 0.9974 0.9974

JPEG QF=70 Nikon_D70 0.7883 0.9435

Nikon_D200 0.8808 0.9645 5MP-9Y2 0.9871 0.9985 Coach 6M 0.9864 0.9998

JPEG QF=50 Nikon_D70 0.7115 0.9389

Nikon_D200 0.8784 0.9686 5MP-9Y2 0.9785 0.9997 Coach 6M 0.9854 0.9991

Table 1 shows that the AUC obtained with

proposed approach and Fridrich's method under JPEG quality factor 50, 70 and 90. The experimental results show that both two methods can obtain the satisfactory results, when JPEG quality factor is 90. If the testing images after JPEG compressed heavily with the quality factor of 50, the AUC decreases rapidly computed by Fridrich's method and the results are shown in Table 1. The comparison results suggest that proposed method achieves large improvement on AUC especially for the heavily JPEG compression.

5. Conclusion and Future Work

In this paper, we present an improved source camera identification method for JPEG compressed images based on sensor pattern noise. The presence of

sensor pattern noise is established using correlation in the detection of spread-spectrum watermarks. We investigated the reliability of source camera identification from images after JPEG compression. Experimental results were evaluated using the ROC curves and AUC. Preliminary results show that the proposed method can reduce false positive rate and increase the value of AUC. Our future work will focus on modified model for sensor pattern noise in compressed images and generalization of this results over a large set of digital cameras.

6. Acknowledgment

This work was supported in part by the National Natural Science Foundation of China (Grant No. 61170239), in part by the Natural Science Foundation of Tianjin (No.09JCYBJC00900). References [1] J. Lukáš, J. Fridrich, and M.Goljan, “Determining Digital Image Origin Using Sensor Imperfections,” in Proc. of the SPIE International Conference on Image and Video Communications and Processing, 2005, pp. 249–260. [2] Y. Sutcu, S. Bayram, H.T. Sencar, and N. Memon, “Improvements on Sensor Noise Based Source Camera Identification,” in Proc. of IEEE International Conference on Multimedia and Expo, 2007, pp. 24–27. [3] J.Lukáš,J.Fridrich,and M.Goljan, “Digital Camera Identification from Sensor Pattern Noise,” IEEE Trans. on Information Forensics and Security, vol. 1, 2006, pp. 205–214. [4] K. Matsushita and H. Kitazawa, "An Improved Camera Identification Method Based on the Texture Complexity and the Image Restoration," International Conference on Convergence and Hybrid Information Technology, 2009, pp. 171–175. [5] Chang-Tsun Li, “Source Camera Identification Using Enhanced Sensor Pattern Noise,” International Conference on Convergence and Hybrid Information Technology, 2010, pp. 280–287. [6] Y.Tomioka, H.Kitazawa, “Digital Camera Identification Based on the Clustered Pattern Noise of Image Sensors,” IEEE International Conference on Multimedia and Expo(ICME), 2011, pp. 1-4. [7] Wiger van Houten, Zeno Geradts, “Source video camera identification for multiply compressed videos originating from You Tube,” Digital Investigation Volume 6, Issues 1-2, September 2009, pp.48-60. [8] Mark A. Robertson, Robert L.Stevenson, “DCT Quantization Noise in Compressed Images,” IEEE Transactions on Circuits and Systems for Video Technology, VOL.15, NO.1, Jan.2005, pp.27-38. [9] http://forensics.inf.tu-dresden.de/ddimgdb/news