tags : Floating Point, Computer Architecture
To come back later
-
finance folks use fixed-point
-
GPUs do not implement fixed-point arithmetic in hardware
-
I would expect dedicated digital signal processor chips to almost always support a high-performance fixed-point multiply-add instruction.
-
A Tutorial on Data Representation - Integers, Floating-point numbers
Deterministic?
- The main appeal of fixed point nowadays is determinism across hardware - so networked games benefit, for instance.
- They are still popular for multiplayer games that has a synced game state. In all clients you want to get exact same results when they run same code which is not possible with fp
Integer Math
- Integer arithmetic is a specialisation of fixed-point arithmetic in which operands have an exponent of zero. This means that the results of all operations also have an exponent of zero. That obviously simplifies operations themselves, but it also means that the results of those operations can be used in subsequent operations (or recurrently in the same operation) without the need to normalise the exponent, i.e. no bit shifting!
- you can perfectly store integers in FP variables. Even banks could store everything in FP instead of integer variables in the lowest unit they support (for example cents).
- Integers at overflow fail catastrophically, so that also has to be controlled for.
Uses
- very low-level network software, radios, and image processing.
- NN inference
- In neural nets, it is usually paired with reduced precision
int8because in addition to getting more operations per second, you can transfer more weights per byte of L1 cache and per byte-per-second of memory bus.- Since navigating those memory limitations is a major part of performance optimization, reduced-precision fixed-point can help a lot.