IEEE 754
IEEE Standard for Floating-Point Arithmetic
≈ 18 min read
Names in other languages
- EnglishIEEE 754
- FrançaisIEEE 754
- 한국어IEEE 754
- PolskiIEEE 754
- PortuguêsIEEE 754
- РусскийIEEE 754
- 简体中文IEEE 754
- 繁體中文IEEE 754
- 中文IEEE 754
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
The IEEE Standard for Floating-Point Arithmetic (IEEE 754) is a technical standard for floating-point arithmetic originally established in 19851 by the Institute of Electrical and Electronics Engineers (IEEE). The standard addressed many problems found in the diverse floating-point implementations that made them difficult to use reliably and portably.
Many hardware floating-point units use the IEEE 754 standard. The standard defines: * arithmetic formats: sets of binary and decimal floating-point data, which consist of finite numbers (including signed zeros and subnormal numbers), infinities, and special "not a number" values (NaNs) * interchange formats: encodings (bit strings) that may be used to exchange floating-point data in an efficient and compact form * rounding rules: properties to be satisfied when rounding numbers during arithmetic and conversions * operations: arithmetic and other operations (such as trigonometric functions) on arithmetic formats * exception handling: indications of exceptional conditions (such as division by zero, overflow, etc.) IEEE 754-20081, published in August 2008, includes nearly all of the original IEEE 754-1985 standard, plus the IEEE 854-1987 (Radix-Independent Floating-Point Arithmetic) standard.
The current version, IEEE 754-20191, was published in July 2019.…
The IEEE Standard for Floating-Point Arithmetic (IEEE 754) is a technical standard for floating-point arithmetic originally established in 1985 by the Institute of Electrical and Electronics Engineers (IEEE). The standard addressed many problems found in the diverse floating-point implementations that made them difficult to use reliably and portably.
Many hardware floating-point units use the IEEE 754 standard. The standard defines: arithmetic formats: sets of binary and decimal floating-point data, which consist of finite numbers (including signed zeros and subnormal numbers), infinities, and special "not a number" values (NaNs) interchange formats: encodings (bit strings) that may be used to exchange floating-point data in an efficient and compact form rounding rules: properties to be satisfied when rounding numbers during arithmetic and conversions operations: arithmetic and other operations (such as trigonometric functions) on arithmetic formats exception handling: indications of exceptional conditions (such as division by zero, overflow, etc.) IEEE 754-2008, published in August 2008, includes nearly all of the original IEEE 754-1985 standard, plus the IEEE 854-1987 (Radix-Independent Floating-Point Arithmetic) standard.
The current version, IEEE 754-2019, was published in July 2019. It is a minor revision of the previous version, incorporating mainly clarifications, defect fixes and new recommended operations.
History
The need for a floating-point standard arose from chaos in the business and scientific computing industry in the 1960s and 1970s. IBM used a hexadecimal floating-point format with seven bits always used for the exponent regardless of precision. CDC and Cray computers used ones' complement representation, which admits a value of +0 and −0.
CDC 60-bit computers did not have full 60-bit adders, so integer arithmetic was limited to 48 bits of precision from the floating-point unit. Exception processing from divide-by-zero was different on different computers. Moving data between systems and even repeating the same calculations on different systems was often difficult.
The first IEEE standard for floating-point arithmetic, IEEE 754-1985, was published in 1985. It covered only binary floating-point arithmetic. A new version, IEEE 754-2008, was published in August 2008, following a seven-year revision process, chaired by Dan Zuras and edited by Mike Cowlishaw.
It replaced both IEEE 754-1985 (Binary Floating-Point Arithmetic) and IEEE 854-1987 (Radix-Independent Floating-Point Arithmetic) standards. The binary formats in the original standard are included in this new standard along with three new basic formats, one binary and two decimal.
To conform to the current standard, an implementation must implement at least one of the basic formats as both an arithmetic format and an interchange format. The international standard ISO/IEC/IEEE 60559:2011 (with content identical to IEEE 754-2008) has been approved for adoption through ISO/IEC JTC 1/SC 25 under the ISO/IEEE PSDO Agreement and published.
The current version, IEEE 754-2019 published in July 2019, is derived from and replaces IEEE 754-2008, following a revision process started in September 2015, chaired by David G. Hough and edited by Mike Cowlishaw. It incorporates mainly clarifications (e.g. totalOrder) and defect fixes (e.g. minNum), but also includes some new recommended operations (e.g. augmentedAddition).
The international standard ISO/IEC 60559:2020 (with content identical to IEEE 754-2019) has been approved for adoption through ISO/IEC JTC 1/SC 25 and published. The next projected revision of the standard is in 2029. an exponent range from emin to emax, with emin = 1 − emax, or equivalently emin = − (emax − 1), for all IEEE 754 formats.
A format comprises Finite numbers, which can be described by three integers: s = a sign (zero or one), c = a significand (also called a coefficient or mantissa) having no more than p digits when written in base b (i.e., an integer in the range through 0 to b p − 1), and q = an exponent such that emin ≤ q + p − 1 ≤ emax. The numerical value of such a finite number is .
Moreover, there are two zero values, called signed zeros: the sign bit specifies whether a zero is +0 (positive zero) or −0 (negative zero). Two infinities: +∞ and −∞. Two kinds of NaN (not-a-number): a quiet NaN (qNaN) and a signaling NaN (sNaN).
For example, if b = 10, p = 7, and emax = 96, then emin = −95, the significand satisfies 0 ≤ c ≤ , and the exponent satisfies . Consequently, the smallest non-zero positive number that can be represented is 1×10 −101 , and the largest is 9999999×10 90 (9.999999×10 96 ), so the full range of numbers is −9.999999×10 96 through 9.999999×10 96 .
The numbers −b 1−emax and b 1−emax (here, −1×10 −95 and 1×10 −95 ) are the smallest (in magnitude) normal numbers; non-zero numbers between these smallest numbers are called subnormal numbers. Some numbers may have several possible floating-point representations. For instance, if b = 10, and p = 7, then −12.345 can be represented by −12345×10 −3 , −123450×10 −4 , and −1234500×10 −5 .
However, for most operations, such as arithmetic operations, the result (value) does not depend on the representation of the inputs. For the decimal formats, any representation is valid, and the set of these representations is called a cohort. When a result can have several representations, the standard specifies which member of the cohort is chosen.
For the binary formats, the representation is made unique. Most floating point numbers, those with absolute value larger than 2 , are called normal.
They have a significand which is between 1 (included) and 2 (excluded), which thus always has a leading 1 bit; this bit is not explicitly stored in the representation, but is left implicit; this rule is called leading bit convention, implicit bit convention, or hidden bit convention, and it allows the format to have an extra bit of precision. A normal number's exponent has a "bias" added to it, and the resulting biased exponent is always a positive integer.
Numbers smaller than 2 are called subnormal numbers. These numbers are represented by the biased exponent 0, which stands for emin. For subnormal numbers, there is no longer an implicit leading 1 bit, and the significand lies between 0 and 1.
Due to the possibility of multiple encodings (at least in formats called interchange formats), a NaN may carry other information: a sign bit (which has no meaning, but may be used by some operations) and a payload, which is intended for diagnostic information indicating the source of the NaN (but the payload may have other uses, such as NaN-boxing ). In the table above, integer values are exact, whereas values in decimal notation (e.g. 1.0) are rounded values.
The minimum exponents listed are for normal numbers; the special subnormal number representation allows even smaller (in magnitude) numbers to be represented with some loss of precision. For example, the smallest positive number that can be represented in binary64 is 2 −1074 ; contributions to the −1074 figure include the emin value −1022 and all but one of the 53 significand bits (2 −1022 − (53 − 1) = 2 −1074 ).
Decimal digits is the precision of the format expressed in terms of an equivalent number of decimal digits. It is computed as digits × log 10 base. E.g. binary128 has approximately the same precision as a 34 digit decimal number. log 10 MAXVAL is a measure of the range of the encoding.
Its integer part is the largest exponent shown on the output of a value in scientific notation with one leading digit in the significand before the decimal point (e.g. 1.698 is near the largest value in binary32, 9.999999 is the largest value in decimal32). The binary32 (single) and binary64 (double) formats are two of the most common formats used today.
The figure below shows the absolute precision for both formats over a range of values. This figure can be used to select an appropriate format given the expected value of a number and the required precision. An example of a layout for 32-bit floating point is and the 64 bit layout is similar.
The original IEEE 754-1985 standard also had the concept of extended formats, but without any mandatory relation between emin and emax. For example, the Motorola 68881 80-bit format, where emin = − emax, was a conforming extended format, but it became non-conforming in the 2008 revision.
Binary
For the exchange of binary floating-point numbers, interchange formats of length 16 bits, 32 bits, 64 bits, and any multiple of 32 bits ≥ 128 are defined. The 16-bit format is intended for the exchange or storage of small numbers (e.g., for graphics).
The encoding scheme for these binary interchange formats is the same as that of IEEE 754-1985: a sign bit, followed by w exponent bits that describe the exponent offset by a bias, and p − 1 bits that describe the significand. The width of the exponent field for a k-bit format is computed as w = round(4 log 2 (k)) − 13.
The existing 64- and 128-bit formats follow this rule, but the 16- and 32-bit formats have more exponent bits (5 and 8 respectively) than this formula would provide (3 and 7 respectively). As with IEEE 754-1985, the biased-exponent field is filled with all 1 bits to indicate either infinity (trailing significand field = 0) or a NaN (trailing significand field ≠ 0).
For NaNs, quiet NaNs and signaling NaNs are distinguished by using the most significant bit of the trailing significand field exclusively, and the payload is carried in the remaining bits.
Decimal
For the exchange of decimal floating-point numbers, interchange formats of any multiple of 32 bits are defined. As with binary interchange, the encoding scheme for the decimal interchange formats encodes the sign, exponent, and significand.
Two different bit-level encodings are defined, and interchange is complicated by the fact that some external indicator of the encoding in use may be required. The two options allow the significand to be encoded as a compressed sequence of decimal digits using densely packed decimal or, alternatively, as a binary integer.
The former is more convenient for direct hardware implementation of the standard, while the latter is more suited to software emulation on a binary computer. In either case, the set of numbers (combinations of sign, significand, and exponent) that may be encoded is identical, and special values (±zero with the minimum exponent, ±infinity, quiet NaNs, and signaling NaNs) have identical encodings.
At the extremes, a value with a magnitude strictly less than k=b^{\text{emax}}\left(b-\tfrac{1}{2}b^{1-p}\right) will be rounded to the minimum or maximum finite number (depending on the value's sign). Any numbers with exactly this magnitude are considered ties; this choice of tie may be conceptualized as the midpoint between \pm b^{\text{emax}}(b-b^{1-p}) and \pm b^{\text{emax}+1} , which, were the exponent not limited, would be the next representable floating-point numbers larger in magnitude.
Numbers with a magnitude strictly larger than are rounded to the corresponding infinity. "Round to nearest, ties to even" is the default for binary floating point and the recommended default for decimal. "Round to nearest, ties to away" is only required for decimal implementations.
IEEE 754 specifies a special value called "Not a Number" (NaN) to be returned as the result of certain "invalid" operations, such as 0/0, , or sqrt(−1). In general, NaNs will be propagated, i.e. most operations involving a NaN will result in a NaN, although functions that would give some defined result for any given floating-point value will do so for NaNs as well, e.g. NaN ^ 0 = 1.
There are two kinds of NaNs: the default quiet NaNs and, optionally, signaling NaNs. A signaling NaN in any arithmetic operation (including numerical comparisons) will cause an "invalid operation" exception to be signaled.
The representation of NaNs specified by the standard has some unspecified bits that could be used to encode the type or source of error; but there is no standard for that encoding. In theory, signaling NaNs could be used by a runtime system to flag uninitialized variables, or extend the floating-point numbers with other special values without slowing down the computations with ordinary values, although such extensions are not common.
A variant of this approach (sometimes called "NaN-boxing") is used by some JavaScript runtimes and LuaJIT to store 64-bit pointer values and IEEE 754 double-precision floating-point values in the same data type, allowing runtimes to eliminate the overhead of extra memory allocations and indirections for floating-point values. The special values such as infinity and NaN ensure that the floating-point arithmetic is algebraically complete: every floating-point operation produces a well-defined result and will not—by default—throw a machine interrupt or trap.
Moreover, the choices of special values returned in exceptional cases were designed to give the correct answer in many cases. For instance, under IEEE 754 arithmetic, a continued fraction such as R(z) := 7 - \cfrac{3}{z - 2 + \cfrac{4}{z - 3 } } can be implemented straightforwardly and will give the correct answer even when there is a division by zero, because any positive number divided by zero results in , e.g. when , .
As noted by Kahan, the unhandled trap consecutive to a floating-point to 16-bit integer conversion overflow that caused the loss of an Ariane 5 rocket would not have happened under the default IEEE 754 floating-point policy. Subnormal numbers ensure that for finite floating-point numbers and , if and only if , as expected, but which did not hold under earlier floating-point representations.
On the design rationale of the x87 80-bit format, Kahan notes: "This Extended format is designed to be used, with negligible loss of speed, for all but the simplest arithmetic with float and double operands. For example, it should be used for scratch variables in loops that implement recurrences like polynomial evaluation, scalar products, partial and continued fractions.
It often averts premature Over/Underflow or severe local cancellation that can spoil simple algorithms". Computing intermediate results in an extended format with high precision and extended exponent has precedents in the historical practice of scientific calculation and in the design of scientific calculators e.g. Hewlett-Packard's financial calculators performed arithmetic and financial functions to three more significant decimals than they stored or displayed.
The implementation of extended precision enabled standard elementary function libraries to be readily developed that normally gave double precision results within one unit in the last place (ULP) at high speed. Correct rounding of values to the nearest representable value avoids systematic biases in calculations and slows the growth of errors.
Rounding ties to even removes the statistical bias that can occur in adding similar figures. Directed rounding was intended as an aid with checking error bounds, for instance in interval arithmetic. It is also used in the implementation of some functions.
The mathematical basis of the operations, in particular correct rounding, allows one to prove mathematical properties and design floating-point algorithms such as 2Sum, Fast2Sum and Kahan summation algorithm, e.g. to improve accuracy or implement multiple-precision arithmetic subroutines relatively easily. A property of the single- and double-precision formats is that their encoding allows one to easily sort them without using floating-point hardware, as if the bits represented sign-magnitude integers, although it is unclear whether this was a design consideration (it seems noteworthy that the earlier IBM hexadecimal floating-point representation also had this property for normalized numbers).
With the prevalent two's-complement representation, interpreting the bits as signed integers sorts the positives correctly, but with the negatives reversed; as one possible correction for that, with an xor to flip the sign bit for positive values and all bits for negative values, all the values become sortable as unsigned integers (with ). \sqrt{x^2 + y^2} (hypot) 1\big/\sqrt{x\vphantom{t}} (rSqrt) (1 + x)^n for x \ge -1 (named compound and used to compute an exponential growth, whose rate cannot be less than −1) x^{\frac{1}{n}} (rootn) x^n (pown, n is an integer), x^y (pow), e^{y \ln x} (powr, exponentiation as a continuous function) \sin x , \cos x , \tan x \arcsin x , \arccos x , \arctan x , \operatorname{atan2}(y, x) \operatorname{sinPi} x = \sin \pi x , \operatorname{cosPi} x = \cos \pi x , \operatorname{tanPi} x = \tan \pi x (see also: Multiples of ) \operatorname{asinPi} x = \tfrac1\pi \arcsin x *, \operatorname{acosPi} x = \tfrac1\pi \arccos x *, \operatorname{atanPi} x = \tfrac1\pi \arctan x *, \operatorname{atan2Pi} (y, x) = \tfrac1\pi \operatorname{atan2}(y, x) (see also: Multiples of ) \sinh x , \cosh x , \tanh x \operatorname{arsinh} x , \operatorname{arcosh} x , \operatorname{artanh} x Additions of the 2019 revision are noted with an asterisk (*); the rest were introduced in 2008. The \operatorname{asinPi} , \operatorname{acosPi} and \operatorname{tanPi} functions were not part of the IEEE 754-2008 standard because they were deemed less necessary. \operatorname{asinPi} and \operatorname{acosPi} were mentioned, but this was regarded as an error.
All three were added in the 2019 revision. The correct-rounding requirement is also an addition of the 2019 revision. The recommended operations also include setting and accessing dynamic mode rounding direction, and implementation-defined vector reduction operations such as sum, scaled product, and dot product (correct rounding is not required).
The standard admits that numerical results for these reduction operations may differ across implementations due to intermediate result width and order of operations. , augmented arithmetic operations for the binary formats are also recommended. These operations, specified for addition, subtraction and multiplication, produce a pair of values consisting of a result correctly rounded to nearest in the format and the error term, which is representable exactly in the format.
At the time of publication of the standard, no hardware implementations are known, but very similar operations were already implemented in software using well-known algorithms (e.g. 2Sum). The history and motivation for their standardization are explained in a background document.
The formerly required minNum, maxNum, minNumMag, and maxNumMag from IEEE 754-2008 have been removed in the 2019 revision due to their non-associativity. Instead, two sets of new minimum and maximum operations are recommended. The first set contains minimum, minimumNumber, maximum and maximumNumber.
The second set contains minimumMagnitude, minimumMagnitudeNumber, maximumMagnitude and maximumMagnitudeNumber. The history and motivation for this change are explained in a background document.
Text derived from https://en.wikipedia.org/w/index.php?oldid=1375008421 under CC BY-SA 4.0; modifications have been made.
References
- ↩Wikipedia “IEEE 754” publication date (×6) · confidence 0.95 revision URL