​​Dual PatchNorm The authors propose a new method, Dual Patc | Data Science by ODS.ai 🦜

​​Dual PatchNorm

The authors propose a new method, Dual PatchNorm, for Vision Transformers which involves adding two Layer Normalization layers before and after the patch embedding layer. Experiments across three datasets show that this method improves the performance of well-tuned ViT models, and qualitative experiments support this.

Paper: https://arxiv.org/abs/2302.01327

A detailed unofficial overview of the paper: https://andlukyane.com/blog/paper-review-dual-patch-norm

#deeplearning #cv #transformer