Deep Neural Nets: 33 years ago and 33 years from now Great po | Data Science by ODS.ai 🦜
Deep Neural Nets: 33 years ago and 33 years from now
Great post by Andrej Karpathy on the progress #CV made in 33 years.
Author's ideas on what would a time traveler from 2055 think about the performance of current networks:
* 2055 neural nets are basically the same as 2022 neural nets on the macro level, except bigger. * Our datasets and models today look like a joke. Both are somewhere around 10,000,000X larger. * One can train 2022 state of the art models in ~1 minute by training naively on their personal computing device as a weekend fun project. * Today’s models are not optimally formulated, and just changing some of the details of the model, loss function, augmentation or the optimizer we can about halve the error. * Our datasets are too small, and modest gains would come from scaling up the dataset alone. * Further gains are actually not possible without expanding the computing infrastructure and investing into some R&D on effectively training models on that scale.
Website: https://karpathy.github.io/2022/03/14/lecun1989/ OG Paper link: http://yann.lecun.com/exdb/publis/pdf/lecun-89e.pdf
First Telegram Data Science channel. Covering all technical and popular staff about anything related to Data Science: AI, Big Data, Machine Learning, Statistics, general Math and the applications of f...