Cluster based mixed coding schemes for inverted file index compression - Chen, J.; Zhong, P.; Cook, T. (Journal of Digital Information Management, 2008)
The cluster property of document collections in today's search engines provides valuable information for index compression. By clustering d-gaps of an inverted list based on a threshold, and then encoding clustered and non-clustered d-gaps using different methods, we can tailor to the specific properties of different d-gaps and achieve better compression ratio. Based on this idea, in this paper we propose a cluster based approach and presents two new codes for inverted file index compression: mixed gamma/flat binary code and mixed delta/flat binary code, Experiment results show that the two new codes achieve better or equal performance in terms of compression ratio comparing to interpolative code which is considered as the most efficient bitwise code at present Besides, the two new codes have much lower complexity comparing to interpolative code and therefore enable faster encoding and decoding. By adjusting the parameters for the mixed codes, even better results may be achieved. Experiments show promising results with our approaches.
Keywords: d-gap; Index compression; Inverted file; Inverted list

Web information extraction using web-specific features - Chen, J.; Zhong, P. (Journal of Digital Information Management, 2008)
Several problems exist with traditional HMM based approaches for Web information extraction (IE) due to the lack of consideration on Web-specific features. To address this Issue we present a Generalized Hidden Markov Model (GHMM) that extends HMMs by making use of Web-specific Information for Web IE. In GHMMbased approach, Web content blocks instead of terms are used as basic extraction unit. Besides, instead of using the traditional sequential state transition order, GHMM decides the state transition order based on layout structure of the corresponding web page. Furthermore, GHMM uses multiple emission features derived from Web information instead of single emission feature. Experimental study shows that GHMM based approach can effectively improve Web IE comparing to traditional HMM based approaches.
Keywords: Hidden; Information extraction; Markov model