Data Hiding in Encrypted H.264AVC Video

596 IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, VOL. 9, NO. 4, APRIL 2014

Data Hiding in Encrypted H.264/AVC VideoStreams by Codeword Substitution

Dawen Xu, Rangding Wang, and Yun Q. Shi, Fellow, IEEE

Abstract Digital video sometimes needs to be stored andprocessed in an encrypted format to maintain security andprivacy. For the purpose of content notation and/or tamper-ing detection, it is necessary to perform data hiding in theseencrypted videos. In this way, data hiding in encrypted domainwithout decryption preserves the confidentiality of the content.In addition, it is more efficient without decryption followed bydata hiding and re-encryption. In this paper, a novel schemeof data hiding directly in the encrypted version of H.264/AVCvideo stream is proposed, which includes the following threeparts, i.e., H.264/AVC video encryption, data embedding, anddata extraction. By analyzing the property of H.264/AVC codec,the codewords of intraprediction modes, the codewords of motionvector differences, and the codewords of residual coefficients areencrypted with stream ciphers. Then, a data hider may embedadditional data in the encrypted domain by using codewordsubstitution technique, without knowing the original video con-tent. In order to adapt to different application scenarios, dataextraction can be done either in the encrypted domain or in thedecrypted domain. Furthermore, video file size is strictly pre-served even after encryption and data embedding. Experimentalresults have demonstrated the feasibility and efficiency of theproposed scheme.

Index Terms Data hiding, encrypted domain, H.264/AVC,codeword substituting.

I. INTRODUCTION

CLOUD computing has become an important technologytrend, which can provide highly efficient computationand large-scale storage solution for video data. Given thatcloud services may attract more attacks and are vulnerableto untrustworthy system administrators, it is desired that thevideo content is accessible in encrypted form. The capabilityof performing data hiding directly in encrypted H.264/AVCvideo streams would avoid the leakage of video content, which

Manuscript received November 12, 2013; accepted January 21, 2014. Dateof publication January 27, 2014; date of current version March 4, 2014.This work was supported in part by the Natural Science Foundation ofChina under Grants 61301247 and 61170137, in part by Zhejiang ProvincialNatural Science Foundation of China under Grant LY13F020013, and inpart by Ningbo Natural Science Foundation under Grant 2013A610059. Theassociate editor coordinating the review of this manuscript and approving itfor publication was Dr. Adnan M. Alattar.

D. Xu is with the School of Electronics and Information Engineer-ing, Ningbo University of Technology, Ningbo 315016, China (e-mail:[email protected]).

R. Wang is with the CKC Software Laboratory, Ningbo University, Ningbo315211, China (e-mail: [email protected]).

Y. Q. Shi is with the Department of Electrical and Computer Engineering,New Jersey Institute of Technology, Newark, NJ 07102-1982 USA (e-mail:[email protected]).

Color versions of one or more of the figures in this paper are availableonline at http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TIFS.2014.2302899

can help address the security and privacy concerns with cloudcomputing [1]. For example, a cloud server can embed theadditional information (e.g., video notation, or authenticationdata) into an encrypted version of an H.264/AVC video byusing data hiding technique. With the hidden information, theserver can manage the video or verify its integrity withoutknowing the original content, and thus the security and pri-vacy can be protected. In addition to cloud computing, thistechnology can also be applied to other prominent applicationscenarios. For example, when medical videos or surveillancevideos have been encrypted for protecting the privacy of thepeople, a database manager may embed the personal informa-tion into the corresponding encrypted videos to provide thedata management capabilities in the encrypted domain.

Till now, few successful data hiding schemes in theencrypted domain have been reported in the open literature.In [2], a watermarking scheme in the encrypted domainusing Paillier cryptosystem is proposed based on the secu-rity requirements of buyer-seller watermarking protocols.A Walsh-Hadamard transform based image watermarkingalgorithm in the encrypted domain using Paillier cryptosystemis presented in [3]. However, due to the constraints of the Pail-lier cryptosystem, the encryption of an original image resultsin a high overhead in storage and computation. Note that,several investigations on reversible data hiding in encryptedimages are reported in [4][8] recently. The encryption is per-formed by using bit-XOR (exclusive-OR) operation. In thesemethods, however, the host image is in an uncompressedformat. In [9], a robust watermarking algorithm is proposed toembed watermark into compressed and encrypted JPEG2000images.

As said the above mentioned works have been focusedon image. With the increasing demands of providing videodata security and privacy protection, data hiding in encryptedH.264/AVC videos will undoubtedly become popular in thenear future. Obviously, due to the constraint of the underlyingencryption, it is very difficult and sometimes impossible totransplant the existing data hiding algorithms to the encrypteddomain. To the best of our knowledge, there has been noreport on the implementation of data hiding in encryptedH.264/AVC video streams. Only few joint data-hiding andencryption approaches that focus on video have been proposed.For example, in [10], during H.264/AVC compression, theintra-prediction mode (IPM), motion vector difference (MVD)and DCT coefficients signs are encrypted, while DCT coeffi-cients amplitudes are watermarked adaptively. In [11], a com-bined scheme of encryption and watermarking is presented,

1556-6013 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

XU et al.: DATA HIDING IN ENCRYPTED H.264/AVC VIDEO STREAMS 597

which can provide the access right as well as the authenticationof video content simultaneously. The IPMs of 4 4 luminanceblock, the sign bits of texture, and the sign bits of MVDsare encrypted, while IPM is used for watermarking. However,the watermarked bitstream is not fully format-compliant as aresult a standard decoder may crash since it cannot parse awatermarked bitstream. Concretely, the value 2 of IPMdoes not exist in the actual standard. In summary, in theexisting related technologies [10][11], encryption and dataembedding are implemented almost simultaneously duringH.264/AVC compression process. However, to meet the afore-mentioned application requirements, its necessary to performdata hiding directly in the encrypted domain. In addition, theapproaches in [10] and [11] do not operate on the compressedbitstream. That is, encryption and watermark embedding areaccomplished in the encoding process, while decryption andwatermark detection are completed in the decoding processThe compression/decompression cycle is time-consuming andhampers real-time implementation. Besides, encryption andwatermark embedding would lead to increasing the bit-rateof H.264/AVC bitstream.

Therefore, it becomes highly desirable to develop datahiding algorithms that work entirely on encoded bitstreamin the encrypted domain However, there are some signifi-cant challenges for data hiding directly in compressed andencrypted bitstream. The first challenge is to determine whereand how the bitstream can be modified so that the encryptedbitstream with hidden data is still a compliant compressedbitstream. The second challenge is to insure that decryptedvideos containing hidden data can still appear to be of highvisual fidelity. The third challenge is to maintain the filesize after encryption and data hiding, which requires that theimpact on compression gain is minimal. The fourth challengeis that the hidden data can be extracted either from theencrypted video stream or from the decrypted video stream,which is much more applicable in practical applications.

Based on the analysis given above, we propose a novelscheme to embed secret data directly in compressed andthen encrypted H.264/AVC bitstream. Firstly, by analyzingthe property of H.264/AVC codec, the codewords of IPMs,the codewords of MVDs, and the codewords of residualcoefficients are encrypted with a stream cipher. The encryptionalgorithm is combined with the Exp-Golomb entropy codingand Context-adaptive variable-length coding (CAVLC) [12],which keeps the codeword length unchanged. Then, datahiding in the encrypted domain is performed based on anovel codeword substituting scheme. In contrast to the existingtechnologies [10][11] discussed above, the proposed schemecan achieve excellent performance in the following threedifferent prospects.

The data hiding is performed directly in encryptedH.264/AVC video bitstream.

The scheme can ensure both the format compliance andthe strict file size preservation.

The scheme can be applied to two different applicationscenarios by extracting the hidden data either from theencrypted video stream or from the decrypted videostream.

The remainder of the paper is organized as follows. InSection II, we describe the proposed scheme, which includesthree parts, i.e., H.264/AVC video encryption, data embeddingand data extraction. Experimental results are presented inSection III. Discussion is shown in Section IV. Finally inSection V, conclusion is drawn.

II. PROPOSED SCHEME

In this section, a novel scheme of data hiding in theencrypted version of H.264/AVC videos is presented, whichincludes three parts, i.e., H.264/AVC video encryption, dataembedding and data extraction. The content owner encryptsthe original H.264/AVC video stream using standard streamciphers with encryption keys to produce an encrypted videostream. Then, the data-hider (e.g., a cloud server) can embedthe additional data into the encrypted video stream by usingcodeword substituting method, without knowing the originalvideo content. At the receiver end, the hidden data extractioncan be accomplished either in encrypted or in decryptedversion. The diagram of the proposed framework is shown inFig. 1, where the encryption and data embedding are depictedin part (a), and the data extraction and video decryption areshown in part (b).

A. Encryption of H.264/AVC Video Stream

Video encryption often requires that the scheme be timeefficient to meet the requirement of real time and formatcompliance. It is not practical to encrypt the whole compressedvideo bitstream like what the traditional ciphers do becauseof the following two constraints, i.e., format compliance andcomputational cost. Alternatively, only a fraction of video datais encrypted to improve the efficiency while still achievingadequate security. The key issue is then how to select thesensitive data to encrypt. According to the analysis givenin [13], it is reasonable to encrypt both spatial information(IPM and residual data) and motion information (MVD) duringH.264/AVC encoding.

In this paper, an H.264/AVC video encryption schemewith good performance including security, efficiency, andformat compliance is proposed. By analyzing the prop-erty of H.264/AVC codec, three sensitive parts (i.e., IPMs,MVDs, and residual coefficients) are encrypted with streamciphers. Compared with [13], the proposed encryption algo-rithm is performed not during H.264/AVC encoding but inthe H.264/AVC compressed domain. In this case, the bit-stream will be modified directly. Selective encryption in theH.264/AVC compressed domain has been already presentedon context-adaptive variable length coding (CAVLC) [14] andcontext-adaptive binary arithmetic coding (CABAC) [15]. Inthis paper, we have improved and enhanced the previousproposed approach by encrypting more syntax elements. Weencrypt the codewords of IPMs, the codewords of MVDs,and the codewords of residual coefficients. The encryptedbitstream is still H.264/AVC compliant and can be decoded byany standard-compliant H.264/AVC decoder, but the encryptedvideo data is treated completely different compared to plain-text video data. In fact, performing the format-compliant


Fig. 1. Diagram of proposed scheme. (a) Video encryption and data embedding at the sender end. (b) Data extraction and video display at the receiver endin two scenarios.

encryption directly on the compressed bitstream is extremelycomplicated as the internal states of the encoder have to bepreserved [16], otherwise the remaining data is interpretedfalsely which may easily lead to format violations.

1) Intra-Prediction Mode (IPM) Encryption: According toH.264/AVC standard, the following four types of intra codingare supported, which are denoted as Intra_4 4, Intra_1616,Intra_chroma, and I_PCM [12]. Here, IPMs in the Intra_44and Intra_16 16 blocks are chosen to encrypt.

Four intra prediction modes (IPMs) are available in theIntra_16 16. The IPM for Intra_16 16 block is spec-ified in the mb_type (macroblock type) field which alsospecifies other parameters about this block such as codedblock pattern (CBP). Table I is the list of mb_type valueswith their meanings which are taken from the standard [17].In H.264/AVC baseline profile, the mb_type is encoded withthe Exp-Golomb code. To maintain standard-compliance ofthe encrypted bitstream, we can encrypt the codeword ofan IPM without modifying the CBP. In addition, to keepthe codewords length unchanged, the encrypted codewordshould have the same size as the original codeword. It can

be observed that the combination of CBP is the same in everyfour lines, and the codewords have the same length in everytwo consecutive lines, as shown in Table I. For example,the codewords corresponding to 3 and 4 are 00100and 00101, respectively, which have the same length andthe same value of CBP. Thus for Intra_16 16 block, theIPM encryption is performed by applying a bitwise XORoperation between the last bit of the codewords and a bit ofthe pseudorandom sequence to keep the value of CBP andthe length of codeword unchanged [18]. The pseudorandomsequence is generated via a standard secure cipher (e.g., RC4)determined by an encryption key E_Key1.

In H.264/AVC, each Intra_4 4 luminance block is pre-dicted from its spatially neighboring samples. Specifically,H.264/AVC offers nine prediction modes (0-8) for Intra_44luminance blocks. The choice of prediction mode for eachIntra_4 4 luminance block must be signalled to the decoderand this could potentially require a large number of bits.To efficiently compress the prediction-mode data, the predic-tive coding technique is applied to signal prediction modes.For each currently considered block E , the most probable


TABLE I

MACROBLOCK TYPES FOR I SLICES AND VARIABLE LENGTH OF

CODEWORD IN H.264/AVC [17]

mode (M P ME ) is defined as the smaller of the predictionmodes of the spatially adjacent upper block A and leftblock B [19]. If either of these adjacent blocks is not available,the corresponding value is set to 2, indicating DC predictionmode.

The prediction mode of the currently considered block E isdenoted as ModeE . If ModeE is equal toM P ME , only onebit is needed to signal the prediction mode. When ModeE andM P ME are different, the codeword is composed of one flagbit 0and three bits fixed-length code [19].

For Intra_4 4 block, if ModeE is equal to M P ME ,the codeword is kept unchanged. Otherwise, three bits fixed-length code (denoted as X) [13] in each codeword is encryptedwith a pseudorandom sequence which is generated via astandard secure cipher (e.g., RC4) determined by an encryptionkey E_Key2. Bitwise XOR operation is still utilized as theencryption scheme.

From what described above, it is obvious that the length ofthe encrypted codeword is the same as the original one. Forthe format compliance in the decoding process, the encryptedIPMs of blocks in the first row and/or in the first columnshould have the decodable value, since not all modes areavailable along the top and the left borders of each framedue to the lack of neighbors. In our scheme, if the IPMafter encryption is not available for a border block, then theIPM encryption of this block will be skipped. This further

TABLE II

MVDs AND CORRESPONDING EXP-GOLOMB CODEWORDS

indicates that IPM encryption is not secure enough in somespecific locations and should be used in combination withother encrypting method. In summary, IPM encryption implieschanging the actual mode to another one without violating thesemantics and bitstream compliance.

2) Motion Vector Difference (MVD) Encryption: In order toprotect both texture information and motion information, notonly the IPMs but also the motion vectors should be encrypted.In H.264/AVC, motion vector prediction is further performedon the motion vectors, which yields MVD. In H.264/AVCbaseline profile, Exp-Golomb entropy coding [19] is used toencode MVD. The codeword of Exp-Golomb is constructedas[M zeros] [1] [I N FO ], where I N FO is an M-bit fieldcarrying information.

Table II shows the values of MVDs and correspondingExp-Golomb codewords. The last bit of the codeword isencrypted by applying the bitwise XOR operation with astandard stream cipher, which is determined by an encryptionkey E_Key3. According to Table II, the last bit encryptionmay change the sign of MVD, but does not affect the lengthof the codeword and satisfies the format compliance [10]. Thatis, the resulting ciphertexts are still valid Exp-Golomb codes.For example, the codewords corresponding to 2 and 2are 00100 and 00101, respectively, which have the samelength. It should be noted that when the value of MVD isequal to 0, its corresponding codeword 1 keeps unchangedduring the encryption process.

3) Residual Data Encryption: In order to keep high security,another type of sensitive data, i.e., the residual data in bothI-frames and P-frames should be encrypted. In this section, anovel method for encrypting the residual data based on thecharacteristics of codeword is presented in detail.


In H.264/AVC baseline profile, CAVLC entropy codingis used to encode the quantized coefficients of a residualblock [19]. Each CAVLC codeword can be expressed as thefollowing format:

{Coe f f token, Sign of T railingOnes,Level, T otal zeros, Run bef ore}

The specific function of each syntax element is describedin [19]. To keep the bitstream compliant, not all syntaxelements can be modified during encryption process. Forexample,Coef f token, T otal zeros, and Run bef ore shouldremain unchanged [14]. Therefore, residual data encryp-tion can be accomplished by modifying the codewords ofSign_of_TrailingOnes and Level.

The Sign_of_TrailingOnes is encoded with a single bit. Bit0 is assigned for +1 and bit 1is assigned for 1. Thecodeword of Sign_of_TrailingOnes is encrypted by applyingthe bitwise XOR operation with a standard stream cipher,which is determined by an encryption key E_Key4.

The codeword for each Level is made up of a prefix(level_prefix) and a suffix (level_suffix) as

Level codeword = [level pre f i x], [level su f f i x]Table III shows Levels with different suffixLength and corre-

sponding codewords. The last bit of the codeword is encryptedby applying the bitwise XOR operation with a standard streamcipher, which is determined by an encryption key E_Key5.According to Table III, the last bit encryption may changethe sign of Levels, but does not affect the length of thecodeword and satisfies the format compliance. For example,when suffixLength is equal to 1, the codewords correspondingto 2 and -2 are 010 and 011, respectively, which havethe same length. It should be noted that when suffixLength isequal to 0, the codewords should keep unchanged during theencryption process.

B. Data Embedding

Although few methods have been proposed to embed datainto H.264/AVC bitstream directly [20][21], however, thesemethods cannot be implemented in the encrypted domain.In the encrypted bitstream of H.264/AVC, the proposed dataembedding is accomplished by substituting eligible codewordsof Levels in Table III. Since the sign of Levels are encrypted,data hiding should not affect the sign of Levels. Besides,the codewords substitution should satisfy the following threelimitations. First, the bitstream after codeword substitutingmust remain syntax compliance so that it can be decoded bystandard decoder. Second, to keep the bit-rate unchanged, thesubstituted codeword should have the same size as the originalcodeword. Third, data hiding does cause visual degradationbut the impact should be kept to minimum. That is, theembedded data after video decryption has to be invisible toa human observer. So the value of Level corresponding tothe substituted codeword should keep close to the value ofLevel corresponding to the original codeword. In addition,the codewords of Levels within P-frames are used for datahiding, while the codewords of Levels in I-frames are remained

TABLE III

LEVELS AND CORRESPONDING CODEWORDS

unchanged. Because I-frame is the first frame in a groupof pictures (GOPs), the error occurred in I-frame will bepropagated to subsequent P-frames.

According to the analysis given above, we can see that thereare no corresponding substituted codewords when suffixLengthis equal to 0 or 1, as shown in Table III. When suffixLengthis equal to 0, we cannot find a pair of codewords with thesame size. When suffixLength is equal to 1, one codeword alsocannot be substituted by another codeword with the same size,since this substitution would change the sign of Level. Thenthe codewords of Levels which suffixLength is 2 or 3 would be


Fig. 2. CAVLC codeword mapping. (a) su f f i x Length = 2& Level > 0.(b) su f f i x Length = 2& Level > 0. (c) su f f i x Length = 3& Level > 0.(d) su f f i x Length = 3& Level < 0.

divided into two opposite codespaces denoted as C0and C1as shown in Fig. 2. The codewords assigned in C0and C1 areassociated with binary hidden information 0 and 1.

Suppose the additional data that we want to embed is abinary sequence denoted as B = {b(i)|i = 1, 2, , L, b(i) {0, 1}}. Data hiding is performed directly in encrypted bit-stream through the following steps.

Step1. In order to enhance the security, the additionaldata is encrypted with the chaotic pseudo-random sequenceP = {p(i)|i = 1, 2, , L, p(i) {0, 1}}[22] to generate theto-be-embedded sequence W = {w(i)|i = 1, 2, , L, w(i) {0, 1}}. The sequence P is generated by using logistic mapwith an initial value [22], i.e., the data hiding key. It is verydifficult for anyone who does not retain the data hiding keyto recover the additional data.

Step2. The codewords of Levels are obtained by parsing theencrypted H.264/AVC bitstream.

Step3. If current codeword belongs to codespaces C0orC1,the to-be-embedded data bit can be embedded by codewordsubstituting. Otherwise, the codeword is left unchanged. Thedetailed procedure of codeword substituting for data hiding isshown in Fig. 3. For example, when Level is positive 1 and itssufflxLength is 3, then its corresponding codeword is 1000which belongs to C0 as shown in Fig. 2(c). If the data bit1 needs to be embedded, the codeword 1000 should bereplaced with 1010. Otherwise, if the data bit 0 needs toembedded, the codeword 1000 will keep unchanged.

Step4. Choose the next codeword and then go to Step3 fordata hiding. If there are no more data bits to be embedded,the embedding process is stopped.

Suppose the to-be-embedded data is 1001, the CAVLCcodeword of Level parsing from H.264/AVC bitstream is

Fig. 3. The procedure of codeword mapping.

01 010 00100 00100 0001011 0000100 and the encryptionstream is 10111, an example of data embedding based oncodeword mapping is shown in Fig. 4(a).

C. Data Extraction

In this scheme, the hidden data can be extracted either inencrypted or decrypted domain, as shown in Fig. 1(b). Dataextraction process is fast and simple. We will first discuss theextraction in encrypted domain followed by decrypted domain.

1) Scheme I: Encrypted Domain Extraction. To protectprivacy, a database manager (e.g., cloud server) may only getaccess to the data hiding key and have to manipulate datain encrypted domain. Data extraction in encrypted domainguarantees the feasibility of our scheme in this case.

In encrypted domain, as shown in Fig. 1(b), encrypted videowith hidden data is directly sent to the data extraction module,and the extraction process is given as follows.

Step1: The codewords of Levels are firstly identified byparsing the encrypted bitstream.

Step2: If the codeword belongs to codespace C0, theextracted data bit is 0. If the codeword belongs to codespaceC1, the extracted data bit is 1.

Step3: According to the data hiding key, the same chaoticpseudo-random sequence P that was used in the embeddingprocess can be generated. Then the extracted bit sequencecould be decrypted by using P to get the original additionalinformation. Since the whole process is entirely operated inencrypted domain, it effectively avoids the leakage of originalvideo content.

An example of data extraction in encrypted domain is shownin Fig. 4(b).

2) Scheme II: Decrypted Domain Extraction. In scheme I,both embedding and extraction of the data are performedin encrypted domain. However, in some cases, users wantto decrypt the video first and extract the hidden data fromthe decrypted video. For example, an authorized user, whichowned the encryption key, received the encrypted video withhidden data. The received video can be decrypted usingthe encryption key. That is, the decrypted video still includesthe hidden data, which can be used to trace the source of thedata. Data extraction in decrypted domain is suitable for this


Fig. 4. An example of data embedding and extraction. (a) Data embedding.(b) Data extraction in encrypted domain. (c) Data extraction in decrypteddomain.

case. As shown in Fig. 1(b), the received encrypted video withhidden data is first pass through the decryption module. Thewhole process of decryption and data extraction is given asfollows.

Step1: Generate encryption streams with the encryption keysas given in encryption process.

Step2: The codewords of IPMs, MVDs,Sign_of_TrailingOnes and Levels are identified by parsing theencrypted bitstream.

Step3: The decryption process is identical to the encryptionprocess, since XOR operation is symmetric. The encrypted

codewords can be decrypted by performing XOR operationwith generated encryption streams, and then two XOR opera-tions cancel each other out, which renders the original plain-text. Since the encryption streams depend on the encryptionkeys, the decryption is possible only for the authorized users.After generating the decrypted codewords with hidden data,the content owner can further extract the hidden information.

Step4: According to Table III, the last bit encryption maychange the sign of Level. However, as shown in Fig. 2, theencrypted codeword and the original codeword are still in thesame codespaces. If the decrypted codeword of Level belongsto codespace C0, the extracted data bit is 0. If the decryptedcodeword of Level belongs to codespace C1, the extracted databit is 1.

Step5: Generate the same pseudo-random sequence P thatwas used in embedding process according to the data hidingkey. The extracted bit sequence should be decrypted to get theoriginal additional information.

An example of data extraction in decrypted domain is shownin Fig. 4(c).

III. EXPERIMENTAL RESULTS

The proposed data hiding scheme has been implementedin the H.264/AVC reference software version JM-12.2. Sixwell-known standard video sequences (i.e., Stefan, Table,Tempete, Mobile, Hall, and News) in QCIF format (176 144)at the frame rate 30 frames/s are used for simulation. The first100 frames in each video sequence are used in the experiments.The GOP (Group of Pictures) structure is IPPPP: one I framefollowed four P frames.

A. Security of Encryption Algorithm

For the proposed video encryption scheme, the securityincludes both cryptographic security and perceptual security.Cryptographic security denotes the security against crypto-graphic attacks, which depends on the ciphers adopted by thescheme. In the proposed scheme, the secure stream cipher(e.g., RC4) is used to encrypt the bitstream, and chaoticpseudo-random sequence generated by logistic map is usedto encrypt the additional data. They have been proved tobe secure against cryptographic attacks. Perceptual securityrefers to whether the encrypted video is unintelligible or not.Generally, it depends on the encryption schemes properties.For example, encrypting only IPM cannot keep secure enough,since the encrypted video is intelligible [10]. The proposedscheme encrypts IPM, MVD and residual coefficients, whichkeeps perceptual security of the encrypted video. The demon-stration is shown in Figs. 5 and 6. An original frame from eachvideo is depicted in Fig. 5, and their corresponding encryptedresults are depicted in Fig. 6. Other frames have a similareffect of encryption. Due to space limitations, we do not listthe results of all frames. It should be mentioned that not everyvideo can be degraded to the same extent. The perceptual qual-ity of high-motion videos with a complex textured backgroundbecomes much more scrambled after encryption than that oflow-motion videos with a static background. The reason is thatthere are less residual coefficients and MVDs in low-motion


Fig. 5. Original video frames.

Fig. 6. Encrypted video frames.

videos that are available for encryption. In general, scramblingperformance of the described encryption system is more thanadequate.

B. Visual Quality of Stego Video

The encrypted video containing hidden data provided by theserver should be decrypted by the authorized user. Therefore,the visual quality of the decrypted video containing hiddendata is expected to be equivalent or very close to that ofthe original video. By modifying the compressed bitstreamto embed additional data, the most important challenge is tomaintain perceptual transparency, which refers to the modifi-cation of bitstream should not degrade the perceived contentquality. In this paper, only the codewords of Levels withinP-frames are modified for data hiding. Simulation results havedemonstrated that we can embed the additional data witha large capacity into P-frames while preserving high visualquality. The encrypted and decrypted video frames with hiddendata are shown in Figs 7 and 8 respectively. In the experiments,no visible artifacts have been observed in all of the decryptedvideo frames with hidden data.

Besides subjective observation, PSNR (Peak Signal to NoiseRatio), SSIM (Structural Similarity Index), and VQM (VideoQuality Measurement) have been adopted to evaluate theperceptual quality [22]. PSNR is widely used objective videoquality metric. However, it does not perfectly correlate with a

Fig. 7. Encrypted video frames with hidden data.

Fig. 8. Decrypted video frames with hidden data.

perceived visual quality due to nonlinear behavior of humanvisual system. The SSIM index lies in the range between0 and 1, where 1 indicates the reference image is identicalthan the target image. Since H.264/AVC is lossy compression,in order to better illustrate the data hiding on the video quality,the visual quality of non-stego video stream should be tested.The video sequence obtained by decompressing non-stegovideo stream is used as the target sequence, while the originaluncompressed video sequence is used as the reference videosequence. Similarly, in order to test the visual quality of stegovideo stream, the video sequence obtained by encrypting, datahiding, decrypting, and decompressing process is used as thetarget sequence. That is, in this case, the target video containshidden data. The VQM is another approach to measure videoquality that correlates more with the human visual system.In general, the lower VQM value indicates higher perceptualvideo quality, and zero indicates excellent quality. The experi-mental results are shown in Table IV. As can be seen, a higherQP (quantization parameter) will result in lower video quality.The visual quality degradation of decrypted video containinghidden data is very low even for large payloads, i.e., it isgenerally hard to detect the degradation in video quality causedby data hiding.

C. Embedding Capacity

Data hiding payload can be assessed in kilobits per second(kbits/s) [23]. The maximum payload capacity in each video


TABLE IV

EMBEDDING CAPACITY, PSNR, SSIM, AND VQM IN DIRECTLY DECRYPTED VIDEOS (CAVLC CODEWORD MAPPING)

encoded with different QP values is given in Table IV. Payloadof the proposed scheme depends on type of video content andthe QP values. The reason for this is that each video streamhas a different number of qualified codewords. Embeddingcapacity is determined by the number of qualified codewords.Obviously, high motion sequence (e.g., Stefan, Table) hasmuch larger number of qualified codewords. This is be causedby high motion sequence has more qualified Levels, sincedata hiding is only performed within P-frames. Specifically,payload decreases with increase in QP value. When QP valueincreases, the number of residual coefficients decreases, andthen the qualified Levels will be less.

D. Bit Rate Variation

To further evaluate the performance of the proposed scheme,bit rate variation BR_var caused by encryption and data hidingis also introduced.

B R var = B R em B R origB R orig

100%

where BR_em is the bit rate generated by encryption and dataembedding encoder, and BR_orig is the bit rate generated bythe original encoder. The bit rate of the encrypted and stegovideo remains unchanged. This is because the encryption anddata hiding are all performed by replacing a suitable codewordto another codeword with the same length, as described inSection II (A) and (B). In summary, the encryption processand data hiding process do not affect compression efficiencyof encoders, since compression efficiency is typically definedby the bit rate.

Fig. 9. Exp-Golomb codeword mapping. (a) MV D > 0. (b) MV D < 0.

IV. DISCUSSION

In addition to the codewords of Levels, the Exp-Golombcodewords of MVDs can also be used for data hiding. FromTable II, we can see that there are no corresponding substitutedcodewords when MVD is equal to 0, because we cannotfind a pair of codeword with the same size. In addition, thecodewords 010 and 011should not be modified during datahiding, since they are encrypted codeword pair. Then the restedcodewords would be divided into two opposite codespacesdenoted as C0and C1as shown in Fig. 9. The codewordsassigned in C0and C1 are associated with binary hiddeninformation 0 and 1.

Data hiding and extraction procedure are the same as thepreviously described in Section II (B) and (C), respectively.The experiment results are shown in Table V. According toTable V, for high motion sequences (such as Stefan, Table)and high texture sequences (such as Tempete and Mobile), thedegradation in video quality caused by MVDs Exp-Golombcodeword substituting is more serious than the previous Levels


TABLE V

EMBEDDING CAPACITY, PSNR, SSIM, AND VQM IN DIRECTLY DECRYPTED VIDEOS (EXP-GOLOMB CODEWORD MAPPING)

TABLE VI

TEST RESULTS BASED ON THE COMBINATION OF CAVLC CODEWORD MAPPING AND EXP-GOLOMB CODEWORD MAPPING

codeword substituting method. For this type of video, it isappropriate to embed data using the codewords of Levels.According to Table IV, for low motion sequence (such as Hall,News), the embedding capacity is low if only the codewordsof Levels are used for data hiding. In this case, both theCAVLC codewords of Levels and the Exp-Golomb codewordsof MVDs can be used for data hiding. As depicted in Table V,for this type of video, the degradation in video quality causedby data hiding is quite small. So the combination is entirelyfeasible. The test results based on the combination of theCAVLC codewords of Levels and the Exp-Golomb codewordsof MVDs are also given in Table VI. Compared with Table IV,the embedding capacity is improved obviously, but the videoquality degradation is also negligible. Based on the aboveanalysis, we can make a flexible choice of embedding carrieraccording to the situation.

To the best of our knowledge, till now, there is no algorithmto embed additional data directly in encrypted H.264/AVCvideo stream. Therefore, no detailed experimental comparisons

are given in the paper. As described in Section I, in the existingrelated technologies [10][11], encryption and data hidingare accomplished almost simultaneously during H.264/AVCencoding process. In addition, encryption and data embeddingwould lead to increasing the bit-rate of H.264/AVC bitstream.On the contrary, our proposed scheme can encrypt H.264/AVCvideo stream directly and then embeds data into encryptedH.264/AVC video stream to meet the privacy-preservingrequirements. The bit-rate of the encrypted H.264/AVC videostream containing hidden data is exactly the same as theoriginal H.264/AVC video stream.

V. CONCLUSION

Data hiding in encrypted media is a new topic that hasstarted to draw attention because of the privacy-preservingrequirements from cloud data management. In this paper, analgorithm to embed additional data in encrypted H.264/AVCbitstream is presented, which consists of video encryption,data embedding and data extraction phases. The algorithm


can preserve the bit-rate exactly even after encryption anddata embedding, and is simple to implement as it is directlyperformed in the compressed and encrypted domain, i.e.,it does not require decrypting or partial decompression ofthe video stream thus making it ideal for real-time videoapplications. The data-hider can embed additional data into theencrypted bitstream using codeword substituting, even thoughhe does not know the original video content. Since data hidingis completed entirely in the encrypted domain, our method canpreserve the confidentiality of the content completely. Withan encrypted video containing hidden data, data extractioncan be carried out either in encrypted or decrypted domain,which provides two different practical applications. Anotheradvantage is that it is fully compliant with the H.264/AVCsyntax. Experimental results have shown that the proposedencryption and data embedding scheme can preserve file-size,whereas the degradation in video quality caused by data hidingis quite small.

REFERENCES

[1] W. J. Lu, A. Varna, and M. Wu, Secure video processing: Problems andchallenges, in Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing,Prague, Czech Republic, May 2011, pp. 58565859.

[2] B. Zhao, W. D. Kou, and H. Li, Effective watermarking scheme inthe encrypted domain for buyer-seller watermarking protocol, Inf. Sci.,vol. 180, no. 23, pp. 46724684, 2010.

[3] P. J. Zheng and J. W. Huang, Walsh-Hadamard transform in the homo-morphic encrypted domain and its application in image watermarking,in Proc. 14th Inf. Hiding Conf., Berkeley, CA, USA, 2012, pp. 115.

[4] W. Puech, M. Chaumont, and O. Strauss, A reversible datahiding method for encrypted images, Proc. SPIE, vol. 6819,pp. 68191E-168191E-9, Jan. 2008.

[5] X. P. Zhang, Reversible data hiding in encrypted image, IEEE SignalProcess. Lett., vol. 18, no. 4, pp. 255258, Apr. 2011.

[6] W. Hong, T. S. Chen, and H. Y. Wu, An improved reversible datahiding in encrypted images using side match, IEEE Signal Process.Lett., vol. 19, no. 4, pp. 199202, Apr. 2012.

[7] X. P. Zhang, Separable reversible data hiding in encryptedimage, IEEE Trans. Inf. Forensics Security, vol. 7, no. 2,pp. 826832, Apr. 2012.

[8] K. D. Ma, W. M. Zhang, X. F. Zhao, N. Yu, and F. Li, Reversible datahiding in encrypted images by reserving room before encryption, IEEETrans. Inf. Forensics Security, vol. 8, no. 3, pp. 553562, Mar. 2013.

[9] A. V. Subramanyam, S. Emmanuel, and M. S. Kankanhalli, Robustwatermarking of compressed and encrypted JPEG2000 images, IEEETrans. Multimedia, vol. 14, no. 3, pp. 703716, Jun. 2012.

[10] S. G. Lian, Z. X. Liu, and Z. Ren, Commutative encryption andwatermarking in video compression, IEEE Trans. Circuits Syst. VideoTechnol., vol. 17, no. 6, pp. 774778, Jun. 2007.

[11] S. W. Park and S. U. Shin, Combined scheme of encryption andwatermarking in H.264/scalable video coding (SVC), New DirectionsIntell. Interact. Multimedia, vol. 142, no. 1, pp. 351361, 2008.

[12] T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra, Overviewof the H.264/AVC video coding standard, IEEE Trans. Circuits Syst.Video Technol., vol. 13, no. 7, pp. 560576, Jul. 2003.

[13] S. G. Lian, Z. X. Liu, Z. Ren, and H. L. Wang, Secure advancedvideo coding based on selective encryption algorithms, IEEE Trans.Consumer Electron., vol. 52, no. 2, pp. 621629, May 2006.

[14] Z. Shahid, M. Chaumont, and W. Puech, Fast protection of H.264/AVCby selective encryption of CAVLC and CABAC for I and P frames,IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 5, pp. 565576,May 2011.

[15] M. N. Asghar and M. Ghanbari, An efficient security system forCABAC bin-strings of H.264/SVC, IEEE Trans. Circuits Syst. VideoTechnol., vol. 23, no. 3, pp. 425437, Mar. 2013.

[16] T. Stutz and A. Uhl, A survey of H.264 AVC/SVC encryp-tion, IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 3,pp. 325339, Mar. 2012.

[17] Advanced Video Coding for Generic Audiovisual Services, ITU, Geneva,Switzerland, Mar. 2005.

[18] J. G. Jiang, Y. Liu, Z. P. Su, G. Zhang, and S. Xing, An improvedselective encryption for H.264 video based on intra prediction modescrambling, J. Multimedia, vol. 5, no. 5, pp. 464472, 2010.

[19] I. E. G. Richardson, H.264 and MPEG-4 Video Compression: VideoCoding for Next Generation Multimedia. Hoboken, NJ, USA: Wiley,2003.

[20] D. K. Zou and J. A. Bloom, H.264 stream replacement watermarkingwith CABAC encoding, in Proc. IEEE ICME, Singapore, Jul. 2010,pp. 117121.

[21] D. W. Xu and R. D. Wang, Watermarking in H.264/AVC compresseddomain using Exp-Golomb code words mapping, Opt. Eng., vol. 50,no. 9, p. 097402, 2011.

[22] D. W. Xu, R. D. Wang, and J. C. Wang, Prediction mode modulateddata-hiding algorithm for H.264/AVC, J. Real-Time Image Process.,vol. 7, no. 4, pp. 205214, 2012.

[23] T. Shanableh, Data hiding in MPEG video files using multivariateregression and flexible macroblock ordering, IEEE Trans. Inf. ForensicsSecurity, vol. 7, no. 2, pp. 455464, Apr. 2012.

Dawen Xu received the M.S. degree in communica-tion and information system from Ningbo University,Ningbo, China, and the Ph.D. degree in computerapplied technology from Tongji University, Shang-hai, China, in 2005 and 2011, respectively. He is anAssociate Professor with the School of Electronicsand Information Engineering, Ningbo University ofTechnology, Ningbo. His current research interestsinclude digital watermarking and information hiding,and signal processing in the encrypted domain. Hehas been a Technical Paper Reviewer for IEEE

conferences, journals, and magazines.

Rangding Wang received the M.S. degree fromNorthwestern Polytechnical University, Xian,China, in 1987, and the Ph.D. degree from TongjiUniversity, Shanghai, China, in 2004. He is aProfessor with the Faculty of Information Scienceand Engineering, Ningbo University, Ningbo,China. His current research interests includemultimedia information security, digital speechprocessing, digital watermarking, steganography,and steganalysis. He is the author or coauthor ofmore than 120 research papers and two books. He

holds more than 20 patents.

Yun Q. Shi (M90SM93F05) received the B.S.and M.S. degrees from Shanghai Jiao Tong Uni-versity, Shanghai, China, and the M.S. and Ph.D.degrees from the University of Pittsburgh, Pitts-burgh, PA, USA. He has been a Professor with theDepartment of Electrical and Computer Engineer-ing, New Jersey Institute of Technology, Newark,NJ, USA, since 1987. His current research inter-ests include digital data hiding, steganalysis, foren-sics and information assurance, and visual signalprocessing and communications. He is the author or

coauthor of more than 300 research papers, one book, and five book chapters,and an Editor of more than ten books. He holds 28 U.S. patents.

Dr. Shi received the Innovators Award 2010 from New Jersey Inventors Hallof Fame for Innovations in Digital Forensics and Security. His U.S. patent7 457 341 titled System and Method for Robust Reversible Data Hiding andData Recovery in the Spatial Domain won the 2010 Thomas Alva EdisonPatent Award from the Research and Development Council of New Jersey.He served as an Associate Editor for two IEEE TRANSACTIONS and a fewother journals.

/ColorImageDict > /JPEG2000ColorACSImageDict > /JPEG2000ColorImageDict > /AntiAliasGrayImages false /CropGrayImages true /GrayImageMinResolution 150 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 600 /GrayImageDepth -1 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages false /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict > /GrayImageDict > /JPEG2000GrayACSImageDict > /JPEG2000GrayImageDict > /AntiAliasMonoImages false /CropMonoImages true /MonoImageMinResolution 400 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict > /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile (None) /PDFXOutputConditionIdentifier () /PDFXOutputCondition () /PDFXRegistryName () /PDFXTrapped /False

/Description >>> setdistillerparams> setpagedevice

Documents

Data Hiding in Encrypted H.264AVC Video