Research

Deep double descent

Source: OpenAI News 05 Dec 2019

Listen to Brief

AI audio in English, based on the NadiAI brief and original source.

Brief

We show that the double descent phenomenon occurs in CNNs, ResNets, and transformers: performance first improves, then gets worse, and then improves again with increasing model size, data size, or training time. This effect is often avoided through careful regularization. While this behavior appears to be fairly universal, we don’t yet fully understand why it happens, and view further study of this phenomenon as an important research direction.

Why It Matters

Memahami fenomena ini penting untuk mereka bentuk model dan kaedah regularisasi yang lebih boleh dipercayai bagi sistem AI berskala besar.

Keep Reading on NadiAI

Selected Related Articles