lzip: Introduction
1 Introduction
**************
Lzip is a lossless data compressor with a user interface similar to the one
of gzip or bzip2. Lzip uses a simplified form of the 'Lempel-Ziv-Markov
chain-Algorithm' (LZMA) stream format and provides a 3 factor integrity
checking to maximize interoperability and optimize safety. Lzip can compress
about as fast as gzip (lzip -0) or compress most files more than bzip2
(lzip -9). Decompression speed is intermediate between gzip and bzip2. Lzip
is better than gzip and bzip2 from a data recovery perspective. Lzip has
been designed, written, and tested with great care to replace gzip and
bzip2 as the standard general-purpose compressed format for unix-like
systems.
For compressing/decompressing large files on multiprocessor machines
plzip can be much faster than lzip at the cost of a slightly reduced
compression ratio. ⇒plzip manual (plzip)Top.
For creation and manipulation of compressed tar archives tarlz can be
more efficient than using tar and plzip because tarlz is able to keep the
alignment between tar members and lzip members. ⇒tarlz manual
(tarlz)Top.
The lzip file format is designed for data sharing and long-term
archiving, taking into account both data integrity and decoder availability:
* The lzip format provides very safe integrity checking and some data
recovery means. The program lziprecover can repair bit flip errors
(one of the most common forms of data corruption) in lzip files, and
provides data recovery capabilities, including error-checked merging
of damaged copies of a file. ⇒Data safety (lziprecover)Data
safety.
* The lzip format is as simple as possible (but not simpler). The lzip
manual provides the source code of a simple decompressor along with a
detailed explanation of how it works, so that with the only help of the
lzip manual it would be possible for a digital archaeologist to extract
the data from a lzip file long after quantum computers eventually
render LZMA obsolete.
* Additionally the lzip reference implementation is copylefted, which
guarantees that it will remain free forever.
A nice feature of the lzip format is that a corrupt byte is easier to
repair the nearer it is from the beginning of the file. Therefore, with the
help of lziprecover, losing an entire archive just because of a corrupt
byte near the beginning is a thing of the past.
The member trailer stores the 32-bit CRC of the original data, the size
of the original data, and the size of the member. These values, together
with the "End Of Stream" marker, provide a 3 factor integrity checking
which guarantees that the decompressed version of the data is identical to
the original. This guards against corruption of the compressed data, and
against undetected bugs in lzip (hopefully very unlikely). The chances of
data corruption going undetected are microscopic. Be aware, though, that
the check occurs upon decompression, so it can only tell you that something
is wrong. It can't help you recover the original uncompressed data.
Lzip uses the same well-defined exit status values used by bzip2, which
makes it safer than compressors returning ambiguous warning values (like
gzip) when it is used as a back end for other programs like tar or zutils.
Lzip will automatically use for each file the largest dictionary size
that does not exceed neither the file size nor the limit given. Keep in
mind that the decompression memory requirement is affected at compression
time by the choice of dictionary size limit.
The amount of memory required for compression is about 1 or 2 times the
dictionary size limit (1 if input file size is less than dictionary size
limit, else 2) plus 9 times the dictionary size really used. The option
'-0' is special and only requires about 1.5 MiB at most. The amount of
memory required for decompression is about 46 kB larger than the dictionary
size really used.
When compressing, lzip replaces every file given in the command line
with a compressed version of itself, with the name "original_name.lz". When
decompressing, lzip attempts to guess the name for the decompressed file
from that of the compressed file as follows:
filename.lz becomes filename
filename.tlz becomes filename.tar
anyothername becomes anyothername.out
(De)compressing a file is much like copying or moving it. Therefore lzip
preserves the access and modification dates, permissions, and, when
possible, ownership of the file just as 'cp -p' does. (If the user ID or
the group ID can't be duplicated, the file permission bits S_ISUID and
S_ISGID are cleared).
Lzip is able to read from some types of non-regular files if either the
option '-c' or the option '-o' is specified.
Lzip will refuse to read compressed data from a terminal or write
compressed data to a terminal, as this would be entirely incomprehensible
and might leave the terminal in an abnormal state.
Lzip will correctly decompress a file which is the concatenation of two
or more compressed files. The result is the concatenation of the
corresponding decompressed files. Integrity testing of concatenated
compressed files is also supported.
Lzip can produce multimember files, and lziprecover can safely recover
the undamaged members in case of file damage. Lzip can also split the
compressed output in volumes of a given size, even when reading from
standard input. This allows the direct creation of multivolume compressed
tar archives.
Lzip is able to compress and decompress streams of unlimited size by
automatically creating multimember output. The members so created are large,
about 2 PiB each.
Info Catalog
lzip: Top
lzip: Top
lzip: Output