lucidrains/vit-pytorch

By GitHub - 2020-10-05

Description

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch - lucidrains/vit-pytorch

Summary

README.md Vision Transformer - Pytorch Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch.
Significance is further explained in Yannic Kilcher's video.
For a Pytorch implementation with pretrained models, please see Ross Wightman's repository here Distillation A recent paper has shown that use of a distillation token for distilling knowledge from convolutional nets to vision transformer can yield small and efficient vision transformers.
A pytorch-lightning script is ready for you to use at the repository link above.

Topics

Machine_Learning (0.1)
NLP (0.07)
Management (0.01)

Similar Articles

Rethinking Attention with Performers

By Google AI Blog - 2020-10-23

Posted by Krzysztof Choromanski and Lucy Colwell, Research Scientists, Google Research Transformer models have achieved state-of-the-art...

fastai/fastbook

By GitHub - 2020-12-07

The fastai book, published as Jupyter Notebooks. Contribute to fastai/fastbook development by creating an account on GitHub.

Creating an Engineering Vision. In this blog post, we share how we

By Medium - 2021-01-12

As a company scales to more people and more technology, it becomes important to have an engineering vision and a strategy to get there. In our experience, as you get to a size of greater than ~40…

Reducing the High Cost of Training NLP Models With SRU

By ASAPP - 2021-02-24

Highly expressive and efficient neural models can be designed using SRU++ with little attention computation needed.

Interaction design is more than just user flows and clicks

By Medium - 2021-02-14

There are three core skills that every modern product design must master: product thinking, visual design, and interaction design. In general, you should be good at all three, but master two. Most…

How Google's Technical Writing Course Helped Me Become a Better Writer

By freeCodeCamp.org - 2021-02-08

Google has a technical writing course that I recently completed and highly enjoyed. It takes roughly 4 hours, and has some exercises along the way so you can test yourself. I'm going to briefly expla ...