Compression of Textual Column-Oriented Data

Authors

  • Vinicius Fulber Garcia Department of Languages and Computer Systems, Universidade Federal de Santa Maria, 1000, Santa Maria
  • Sergio Luis Sardi Mergen Department of Languages and Computer Systems, Universidade Federal de Santa Maria, 1000, Santa Maria

Keywords:

Compression, column-oriented databases, LZ, PPM, BWT, entropy encoding, DSM, NSM, PAX

Abstract

Column-oriented data are well suited for compression. Since values of the same column are stored contiguously on disk, the information entropy is lower if compared to the physical data organization of conventional databases. There are many useful light-weight compression techniques targeted at specific data types and domains, like integers and small lists of distinct values, respectively. However, compression of textual values formed by skewed and high-cardinality words is usually restricted to variations of the LZ compression algorithm. So far there are no empirical evaluations that verify how other sophisticated compression methods address columnar data that store text. In this paper we shed a light on this subject by revisiting concepts of those algorithms. We also analyse how they behave in terms of compression and speed when dealing with textual columns where values appear in adjacent positions.

Downloads

Download data is not yet available.

Downloads

Published

2018-07-03

How to Cite

Garcia, V. F., & Mergen, S. L. S. (2018). Compression of Textual Column-Oriented Data. Computing and Informatics, 37(2), 405–423. Retrieved from http://147.213.75.17/ojs/index.php/cai/article/view/2018_2_405