In the blissful days of today, virtually every computer capable of performing floating point arithmetic implements the IEEE 754 standard. In your numerical analysis class, you have likely been told that “This is how floating point works,” upon which you were introduced to the IEEE 754 standard — as if there couldn’t possibly be an alternative way of doing things. But things were not always this way.

If you were a programmer of floating-point computations on different computers in the 1960’s and 1970’s, you had to cope with a wide variety of floating-point hardware. Each line of computers supported its own range and precision for its floating point numbers, and rounded off arithmetic operations in its own peculiar way. While these differences posed annoying problems, a more challenging problem arose from perplexities that a particular arithmetic could throw up. Some of one fast computer’s numbers behaved as non-zeros during comparison and addition but as zeros during multiplication and division; before a variable could be used safely as a divisor it had to be multiplied by 1.0 and then compared with zero. But another fast computer would trap on overflow if certain of its numbers were multiplied by 1.0 although they were not yet so big that they could not grow bigger by addition. ( This computer also had nonzero numbers so tiny that dividing them by themselves would overflow.) On another computer, multiplying a number by 1.0 could lop off its last four bits. Most computers could get zero from X - Y although X and Y were different; a few computers could get zero even though X and Y were huge and different.
Arithmetic aberrations like these were not derided as bugs; they were “features” of computers too important numercially for programmers to ignore. Programmers coped by inventing bizarre tricks like inserting an assignment ” X = (X + X) - X ” into critical spots in a program that would otherwise havedelivered grossly inaccurate results on a few aberrant computers. And then there were aberrant compilers … .

“Reliable portable numerical software was becoming more expensive to developthan anyone but AT&T and the Pentagon could afford. As microprocessors began toproliferate, so did fears that some day soon nobody would be able to afford it.”

In the late ’70s, Dr. John Palmer convinced Intel that it needed an arithmetic standard to prevent different machines with the label “Intel” from computing differing results. Intel decided to hire Prof. Kahan from Stanford to design its new “Future Compatible” floating point arithmetic. Because of the anarchy ruling in the land of floating point numbers at the time, a new standards committee “p754″ was created by Dr. Robert Stewart. With permission of Intel, Kahan submitted a standards draft proposal to the committee. The proposal was initially met with mixed feelings, due to the standard’s complexity.

“It looked pretty complicated. On the other hand, we had a rationale for everything. What distinguished our proposal from all the others was the reasoning behind every decision. I had worked out the details initially for Intel. My reasoning was based on the requirements of a mass market: A lot of code involving a little floating-point will be written by many people who have never attended my (nor anyone else’s) numerical analysis classes. We had to enhance the likelihood that their programs would get correct results. At the same time we had to ensure that people who really are expert in floating-point could write portable software and prove that it worked, since so many of us would have to rely upon it. There were a lot of almost conflicting requirements on the way to a balanced design.”

So what was so different about this new design? One of the most differing, and most hotly contested design decisions was the inclusion of “subnormal numbers.” In fact, the proposed standard was virtually identical to the standard that DECs machines followed at the time, with the exception of subnormal numbers. Normalized floating point numbers are represented with a sign-bit, an exponent, and a mantissa in the the interval between 1 and 2. A subnormal floating point number has the smallest exponent possible, but the mantissa has a value smaller than 1. Why is this so important? Without these subnormal numbers, there exists a chasm around the number 0. This means that two very small, but unequal numbers X and Y can “fall into” this chasm when they are subtracted, such that X - Y = 0. If this gap is filled with subnormal numbers, the interval between 0 and the smallest representable number is smaller than the interval between any other two numbers. Thus, X - Y = 0 can only be true when X = Y.

How many victims fell into the chasm? Nobody knew. Anecdotal evidence from university computing centers using DEC computers with 8-bit exponents pointed to at least a victim per month per machine during the 1970s. What would happen when computers became over a thousand times more numerous and arithmetic over a thousand times faster?

So if subnormal numbers were so important, why were they such a hotly contested issue? In the 1970’s, subnormal numbers came at a significant cost. They were deemed to cause a significant performance penalty on floating point operations. Not until Intel showed that subnormal numbers could be implemented efficiently in their i8087 floating point co-processor, could the p754 committee be convinced of their inclusion in the IEEE 754 standard.

Nowadays, few people think about gradual underflow anymore. It has become a feature that his taken for granted. But reading about the history of computing, I am thankful to be living in a world with standardized, subnormal floating point numbers.

[1] An Interview with the Old Man of Floating Point: http://www.cs.berkeley.edu/~wkahan/ieee754status/754story.html