Transformer-XL

Transformer-XL is a transformer architecture that introduces the notion of recurrence to the deep self-attention network. Instead of computing the hidden states from scratch for each new segment, Transformer-XL reuses the hidden states obtained in previous segments.

References

Transformer-XL. (n.d.). Papers with Code. Link