renormalization
/ˌriːnɔːrmələˈzeɪʃən/
Definition
A mathematical technique used in complex systems to simplify data by iteratively coarse-graining or grouping features to focus on high-level abstractions.
Etymology
Derived from the prefix 're-' (again) combined with 'normalize' (to make normal or standard). It originated in physics to describe the process of mathematically adjusting values to eliminate infinities in calculations.
In the news
In this article, renormalization describes how deep layers in an LLM filter out irrelevant input details during training to achieve broader generalization. This physics-inspired concept helps researchers understand how models evolve to form abstract representations.
Understanding large language models demands distinguishing human projection from machine cognition
Read the full article ↗Nature