An Efficient Algorithm For The Inverse ST Problem Ge Nong Sen Zhang Computer Sci. Dept. Dept. of Math., Computer Sci. and Stat. Sun Yat-Sen University, PRC SUNY College at Oneonta, USA Email: [email protected] Email: [email protected] The Schindler Transform (ST) [1] can speed up the block sorting phase of the Burrows- Wheeler Transform (BWT) by limiting the context sorting to the first k (k ∈ [0,N ], where N is the length of the text) positions only. Under the ST’s partial sorting scheme, if two rows share the exactly same k-order context, they may not be ordered alpha- betically; instead, the relative order between them in the original matrix is preserved in the transformed matrix. A major tradeoff for the ST to achieve the speedup gain over the BWT is that the inverse ST appears to be more complicated than the inverse BWT. To deal with the existence of identical k-order contexts, Schindler suggested a hash based approach in which the text retrieval has to rely on the hash table based context lookup, which in turn has to rely on the complete retrieval of all the k-order contexts. An improved solution proposed by Yokoo [2] uses no hash table; however, it still needs to restore all the k-order contexts, which clearly requires O(kN ) for both the time and the space complexities. Recently, Nong and Zhang [3] had proposed an auxiliary vector based framework, which is different from any possible k-order context retrieval based approaches formerly suggested by others, but similar to that used for the inverse BWT. This framework relies on two size-N vectors T k and C k (details are omitted due to space limit) to correctly retrieve the true immediate preceding character for a given character, which allows the original text to be recovered directly from the transformed text without statically restoring the complete k-order contexts. As a consequence, this framework requires only O(N ) space. However, its running time remains to be O(kN ), for it has to visit each column of the k-order context matrix to obtain T k and C k . Since T k and C k can be deduced from the context switch vector D (the data structure indicating whether each pair of two neighbor rows in the context matrix are the same or not) in linear time, the more efficient the D can be calculated, the faster the ST can be inverted. If two k-order contexts are different, either their first halves are different already, or their second halves are different; furthermore, the second half of the k-order context matrix can be deduced from its first half due to the rotating scheme in ST. Based on this observation, we proposed a dynamic programming approach to quickly calculate D by doubling the steps to reach the kth column in comparing the k-order contexts. This “doubling technique” based algorithm requires only O(N log k) time to calculate D, thus resulting in an O(N log k) time complexity algorithm to invert ST. The space complexity of the algorithm remains to be O(N ). This new algorithm can be used to build efficient compression solutions based on the ST. REFERENCES [1] M. Schindler, “A fast block-sorting algorithm for lossless data compression,” in Proceedings DCC ’97 Data Compression Conference, Snowbird, UT, USA, Mar. 1997, p. 469. [2] H. Yokoo, “Notes on block-sorting data compression,” Electronics and Communications in Japan (Part III: Fundamental Electronic Science), vol. 82, no. 6, pp. 18–25, 1999. [3] G. Nong and S. Zhang, “Unifying the Burrows-Wheeler and the Schindler transforms,” in Proceedings DCC ’06 Data Compression Conference, Snowbird, UT, USA, Mar. 2006, p. 464. 2007 Data Compression Conference (DCC'07) 0-7695-2791-4/07 $20.00 © 2007

[IEEE 2007 Data Compression Conference (DCC'07) - Snowbird, UT, USA (2007.03.27-2007.03.29)] 2007 Data Compression Conference (DCC'07) - An Efficient Algorithm For The Inverse ST Problem

Download PDF Report

Upload
sen
View
213
Download
1

Embed Size (px)

Citation preview

Page 1: [IEEE 2007 Data Compression Conference (DCC'07) - Snowbird, UT, USA (2007.03.27-2007.03.29)] 2007 Data Compression Conference (DCC'07) - An Efficient Algorithm For The Inverse ST Problem

An Efficient Algorithm For The Inverse ST Problem

Ge Nong Sen ZhangComputer Sci. Dept. Dept. of Math., Computer Sci. and Stat.

Sun Yat-Sen University, PRC SUNY College at Oneonta, USAEmail: [email protected] Email: [email protected]

The Schindler Transform (ST) [1] can speed up the block sorting phase of the Burrows-Wheeler Transform (BWT) by limiting the context sorting to the first k (k ∈ [0, N ],where N is the length of the text) positions only. Under the ST’s partial sorting scheme,if two rows share the exactly same k-order context, they may not be ordered alpha-betically; instead, the relative order between them in the original matrix is preservedin the transformed matrix. A major tradeoff for the ST to achieve the speedup gainover the BWT is that the inverse ST appears to be more complicated than the inverseBWT. To deal with the existence of identical k-order contexts, Schindler suggested ahash based approach in which the text retrieval has to rely on the hash table basedcontext lookup, which in turn has to rely on the complete retrieval of all the k-ordercontexts. An improved solution proposed by Yokoo [2] uses no hash table; however, itstill needs to restore all the k-order contexts, which clearly requires O(kN) for both thetime and the space complexities. Recently, Nong and Zhang [3] had proposed an auxiliaryvector based framework, which is different from any possible k-order context retrievalbased approaches formerly suggested by others, but similar to that used for the inverseBWT. This framework relies on two size-N vectors Tk and Ck (details are omitted dueto space limit) to correctly retrieve the true immediate preceding character for a givencharacter, which allows the original text to be recovered directly from the transformedtext without statically restoring the complete k-order contexts. As a consequence, thisframework requires only O(N) space. However, its running time remains to be O(kN),for it has to visit each column of the k-order context matrix to obtain Tk and Ck. SinceTk and Ck can be deduced from the context switch vector D (the data structure indicatingwhether each pair of two neighbor rows in the context matrix are the same or not) inlinear time, the more efficient the D can be calculated, the faster the ST can be inverted.If two k-order contexts are different, either their first halves are different already, ortheir second halves are different; furthermore, the second half of the k-order contextmatrix can be deduced from its first half due to the rotating scheme in ST. Based onthis observation, we proposed a dynamic programming approach to quickly calculate Dby doubling the steps to reach the kth column in comparing the k-order contexts. This“doubling technique” based algorithm requires only O(N log k) time to calculate D, thusresulting in an O(N log k) time complexity algorithm to invert ST. The space complexityof the algorithm remains to be O(N). This new algorithm can be used to build efficientcompression solutions based on the ST.

REFERENCES

[1] M. Schindler, “A fast block-sorting algorithm for lossless data compression,” in Proceedings DCC ’97 DataCompression Conference, Snowbird, UT, USA, Mar. 1997, p. 469.

[2] H. Yokoo, “Notes on block-sorting data compression,” Electronics and Communications in Japan (Part III:Fundamental Electronic Science), vol. 82, no. 6, pp. 18–25, 1999.

[3] G. Nong and S. Zhang, “Unifying the Burrows-Wheeler and the Schindler transforms,” in Proceedings DCC ’06Data Compression Conference, Snowbird, UT, USA, Mar. 2006, p. 464.