Variable-length encoding

In coding theory, variable-length encoding is a symbol encoding scheme in which codes of differing lengths are used to encode symbols for representation through a communication channel or in a storage medium. The equivalent concept in computer science is bit string. Variable-length codes can allow sources to be compressed and decompressed with zero error (lossless data compression) and still be read back symbol by symbol. An independent and identically-distributed source may be compressed almost arbitrarily close to its entropy. This is in contrast to fixed-length coding methods, for which data compression is only possible for large blocks of data, and any compression beyond the logarithm of the total number of possibilities comes with a finite (though perhaps arbitrarily small) probability of failure. For these reasons, they were sometimes used to pack English text into fewer bytes in adventure games for early microcomputers. However, disks, increases in computer memory, and general purpose compression algorithms have rendered such methods obsolete. Multibyte encodings are usually the result of a need to increase the number of characters which can be encoded without breaking backward compatibility with an existing constraint.…

In coding theory, variable-length encoding is a symbol encoding scheme in which codes of differing lengths are used to encode symbols for representation through a communication channel or in a storage medium. The equivalent concept in computer science is bit string. Variable-length codes can allow sources to be compressed and decompressed with zero error (lossless data compression) and still be read back symbol by symbol. An independent and identically-distributed source may be compressed almost arbitrarily close to its entropy. This is in contrast to fixed-length coding methods, for which data compression is only possible for large blocks of data, and any compression beyond the logarithm of the total number of possibilities comes with a finite (though perhaps arbitrarily small) probability of failure. For these reasons, they were sometimes used to pack English text into fewer bytes in adventure games for early microcomputers. However, disks, increases in computer memory, and general purpose compression algorithms have rendered such methods obsolete. Multibyte encodings are usually the result of a need to increase the number of characters which can be encoded without breaking backward compatibility with an existing constraint. For example, with one byte (8 bits) per character, one can encode 256 possible characters; in order to encode more than 256 characters, the obvious choice would be to use two or more bytes per encoding unit, two bytes (16 bits) would allow 65,536 possible characters, but such a change would break compatibility with existing systems and therefore might not be feasible at all. Unlikely source symbols can be assigned longer codewords while likely source symbols can be assigned shorter codewords, thus giving a low expected codeword length. Some examples of well-known variable-length coding strategies are Huffman coding, Lempel–Ziv coding, arithmetic coding, and context-adaptive variable-length coding. For example, the mapping M_1 = \{\, \texttt{a}\mapsto \texttt{0}, \texttt{b}\mapsto \texttt{0}, \texttt{c}\mapsto \texttt{1}\,\} is not non-singular because both and map to the same bit string ; any extension of this mapping will generate a lossy (non-lossless) coding. Such singular coding may still be useful when some loss of information is acceptable (for example, when such code is used in audio or video compression, where a lossy coding becomes equivalent to source quantization). However, the mapping M_2 = \{\, \texttt{a} \mapsto \texttt{1}, \texttt{b} \mapsto \texttt{011}, \texttt{c}\mapsto \texttt{01110}, \texttt{d}\mapsto \texttt{1110}, \texttt{e}\mapsto \texttt{10011}, \texttt{f}\mapsto\texttt{0}\} is non-singular; its extension will generate a lossless coding, which will be useful for general data transmission (but this feature is not always required). It is not necessary for the non-singular code to be more compact than the source (and in many applications, a larger code is useful, for example as a way to detect or recover from encoding or transmission errors, or in security applications to protect a source from undetectable tampering). The mapping M_3 = \{\, \texttt{a}\mapsto \texttt{0}, \texttt{b}\mapsto \texttt{01}, \texttt{c}\mapsto \texttt{011}\,\} is uniquely decodable (this can be demonstrated by looking at the follow-set after each target bit string in the map, because each bitstring is terminated as soon as we see a \texttt{0} bit which cannot follow any existing code to create a longer valid code in the map, but unambiguously starts a new code). Consider again the code M_2 from the previous section. This code is not uniquely decodable, since the string can be interpreted as the sequence of codewords , but also as the sequence of codewords . Two possible decodings of this encoded string are thus given by and . However, such a code is useful when the set of all possible source symbols is completely known and finite, or when there are restrictions (such as a formal syntax) that determine if source elements of this extension are acceptable. Such restrictions permit the decoding of the original message by checking which of the possible source symbols mapped to the same symbol are valid under those restrictions. 1\times\frac{1}{2}+2\times\frac{1}{4}+3\times\frac{1}{8}+3\times\frac{1}{8}=\frac{7}{4} . As the entropy of this source is 1.75 bits per symbol, this code compresses the source as much as possible so that the source can be recovered with zero error.

Text derived from https://en.wikipedia.org/w/index.php?oldid=1365342874 under CC BY-SA 4.0; modifications have been made.