Description
Attention is a powerful mechanism developed to enhance the performance of the Encoder-Decoder architecture on neural network-based machine translation tasks. Learn more about how this process works an ...
Summary
- Attention is a powerful mechanism developed to enhance the performance of the Encoder-Decoder architecture on neural network-based machine translation tasks.
- This helps the model to cope efficiently with long input sentences.
- By multiplying each encoder hidden state with its softmax score (scalar), we obtain the alignment vector or the annotation vector.
- The alignment vectors are summed up to produce the context vector.