BERT, RoBERTa, DistilBERT, XLNet: Which one to use

By KDnuggets - 2021-01-20

Description

Lately, varying improvements over BERT have been shown — and here I will contrast the main similarities and differences so you can choose which one to use in your research or application.

Summary

Orthogonal Matrix What is an Orthogonal Matrix?
In terms of geometry, orthogonal means that two vectors are perpendicular to each other.
Additionally you may select any two rows and find that the same property holds, as the transpose of an orthogonal matrix is itself an orthogonal matrix.
Because the transpose preserves the determinant, it is easy to show that the determinant of an orthogonal matrix must be equal to 1 or -1.

Topics

Machine_Learning (0.3)
NLP (0.29)
Backend (0.26)

Similar Articles

NYC Taxi Fare Prediction. Rider Fare Prediction in The Big Apple

By Medium - 2020-12-19

The data for this project can be found on Kaggle in the New York City Taxi Fare Prediction competition held by Google Cloud. The entire training set consists of about 55 million rows of NYC taxi fare…

How to compress a neural network. An introduction to weight pruning

By Medium - 2020-10-08

Modern state-of-the-art neural network architectures are HUGE. For instance, you have probably heard about GPT-3, OpenAI’s newest revolutionary NLP model, capable of writing poetry and interactive…

Zero-Shot Learning in Modern NLP

By Joe Davison Blog - 2020-05-29

State-of-the-art NLP models for text classification without annotated data

SCC: an efficient deep reinforcement learning agent mastering the game of StarCraft II

By DeepAI - 2020-12-24

12/24/20 - AlphaStar, the AI that reaches GrandMaster level in StarCraft II, is a remarkable milestone demonstrating what deep reinforcement ...

How to Build a Machine Learning Model

By Medium - 2020-07-25

A Visual Guide to Learning Data Science

30 Most Asked Machine Learning Questions Answered

By Medium - 2021-03-18

Machine Learning is the path to a better and advanced future. A Machine Learning Developer is the most demanding job in 2021 and it is going to increase by 20–30% in the upcoming 3–5 years. Machine…