Data Science by ODS.ai 🦜

2021-03-06 01:31:34 SEER: The start of a more powerful, flexible, and accessible era for computer vision

#SEER stands for SElf-supERvised architecture which follows the vision of Yan LeCunn that real breakthrough in quality of models is possible only with #selfsupervised learning.

And here it is — model which was trained using some enormous amount of data achieves 84.2 percent top-1 accuracy on ImageNet.

Paramus: 1.3B
Dataset: 1B random images
Hardware: 512 GPUs (unspecified)

Blogpost: https://ai.facebook.com/blog/seer-the-start-of-a-more-powerful-flexible-and-accessible-era-for-computer-vision
ArXiV: https://arxiv.org/pdf/2103.01988.pdf

#facebook #fair #cv #dl Open / Comment

2021-03-04 23:00:09 MIT Introduction to Deep Learning

And specifically, lecture about RNN and its modifications:




The course is excellent as well, but more about image processing. For NLP beginners, such clear and elegant survey about RNNs will be quite useful. So, a lot of architectures in NLP models came from image processing tasks. If you want to recap some theory or get understanding of basics of DL — strong recommendation! Open / Comment

2021-02-27 01:14:30 Recent Advances in Language Model Fine-tuning

By Sebastian Ruder:

https://ruder.io/recent-advances-lm-fine-tuning/ Open / Comment

2021-02-25 13:52:29 Ukrainian Open STT 1000 Hours

Following the path of Open STT in Russian, now you can enjoy a similar dataset in Ukrainian:

- Torrent Link
- GitHub Link

Congratulations to our Ukrainian friends for finally publishing a diverse easily downloadable dataset!

Their pages / dataset UX is still a bit rough on the edges, but compared how fast for example Common Voice accumulates data (130 hours for Russian and 43 hours for Ukrainian), UA Open STT and Open STT remain the best resource for respective languages to date.

Also unlike the majority of STT datasets which are (i) behind a paywall or sponsored by corporations (ii) have limited scope / domains (iii) fit some sort of agenda (i.e. use more GPUs than necessary, use our bloated tools, etc), this dataset is legit made by real people.

Also recently corporations have taken up the trend of rehashing publicly available data, which is cool, but unique data is still nowhere to be seen for obvious reasons (except for Common Voice, which is decent only for English).

#dataset Open / Comment

2021-02-23 14:15:57 ​​Real-World Super-Resolution of Face-Images from Surveillance Cameras

Most SR methods are trained on LR (low resolution) data, which is downsampled from HR (high resolution) data using bicubic interpolation, but real-life LR images are usually different, so models work worse on them. In this paper, the authors suggest using blur kernels, noise, and JPEG compression artifacts to generate LR images similar to the original ones.
Another suggested improvement is using ESRGAN and replacing VGG-loss with LPIPS-loss, as well as adding PatchGAN.
In addition, the authors show that NIMA metric better correlates with human perception (mean opinion rank) than traditional Image Quality Assessment methods.

Paper: https://arxiv.org/abs/2102.03113

#deeplearning #superresolution #gan #facesuperresolution Open / Comment

2021-02-23 13:49:44 Towards Causal Representation Learning

Work on how neural networks derive casual variables from low-level observations.

Link: https://arxiv.org/abs/2102.11107

#casuallearning #bengio #nn #DL Open / Comment

2021-02-22 15:06:10 Introducing Model Search: An Open Source Platform for Finding Optimal ML Models

#Google has released an open source #AutoML framework capable of hyperparameter tuning and ensembling.

Blog post: https://ai.googleblog.com/2021/02/introducing-model-search-open-source.html
Repo: https://github.com/google/model_search Open / Comment

2021-02-19 11:36:03 2020 in ML and NLP publications

By conferences, countries, companies, universities and most productive scientists:
https://www.marekrei.com/blog/ml-and-nlp-publications-in-2020/ Open / Comment

2021-02-16 10:53:47 Data Breakfast for Russian speaking audience on Clubhouse

Залетайте поговорить на любые темы.

Link: https://www.joinclubhouse.com/room/MEkQB83P Open / Comment

2021-02-16 09:52:00 Pie chart naming in different languages.

Credit: https://twitter.com/ElephantEating/status/1360988590814023683 Open / Comment