Research

Generative modeling with sparse transformers

Source: OpenAI News 23 Apr 2019

Listen to Brief

AI audio in English, based on the NadiAI brief and original source.

Brief

We’ve developed the Sparse Transformer, a deep neural network which sets new records at predicting what comes next in a sequence—whether text, images, or sound. It uses an algorithmic improvement of the attention mechanism to extract patterns from sequences 30x longer than possible previously.

Why It Matters

Ia membolehkan model generatif memproses konteks lebih panjang dan berpotensi meningkatkan kualiti ramalan rentas modaliti seperti teks, imej dan audio.

Keep Reading on NadiAI

Selected Related Articles