quantization
/ˌkwɒntɪˈzeɪʃən/
Definition
The process of reducing the precision of the numerical values used in an AI model—often by converting high-precision numbers to lower-precision formats—to significantly lower memory and computational costs without drastically reducing performance.
Etymology
Derived from the English 'quantum,' meaning a discrete amount of energy or matter, which comes from the Latin 'quantus' (how much). In computing, it refers to the mapping of a large set of input values to a smaller set of discrete values.
In the news
In this article, quantization is the underlying mechanism that enables open-weight models to run 6x to 60x more cheaply than proprietary frontier models. By compressing model weights, it allows complex AI to operate efficiently on less expensive infrastructure.
Frontier AI Model Power But 60X Cheaper? It’s Possible, Says Together AI
Read the full article ↗HPCwire