Neural Networks from Scratch - Coding a Layer

A beginner’s guide to understanding the inner workings of Deep Learning

https://morioh.com/p/fb1b9f5a52bc

Video Part 1: https://www.youtube.com/watch?v=Wo5dMEP_BbI

Video Part 2: https://www.youtube.com/watch?v=lGLto9Xd7bU

Transform and Tell: Entity-Aware News Image Captioning

End-to-end model which generates captions for images embedded in news articles.

Github: https://github.com/alasdairtran/transform-and-tell

Demo: https://transform-and-tell.ml/

Paper: https://arxiv.org/abs/2004.08070

Today, on April 22 is Earth day. It’s a right time to look at the climate issues in terms of data storage.
* 90% of all data was created in the last two years
* IoT, Big Data and AI are huge data creators
* 70% of all data stored is copy data2
* 70-80% of data is typically unstructured * 2018 HDD shipments = 869Eb
* 2023 HDD shipments = 2.6Zb
* In a normal DC, 1 watt of HDD consumption = 1 watt of cooling

What can everyone do for the ecology of our planet?

* Migrate suitable workloads to the cloud
* Collect, process and store less data; archive more to reduce carbon storage
* Use backup/archive instead of big data
* Leverage copy management tools
* If you must keep data for longer, use tape or cloud tape

Software solutions for backup, managing and recovering data help to move your data to the cloud and so you can take care of the environment. Commvault - leading experts in software-defined storage. Over 11 Exabytes of customer data are under Commvault management.

ResNeSt: Split-Attention Networks

Split-Attention Network, A New ResNet Variant. It significantly boosts the performance of downstream models such as Mask R-CNN, Cascade R-CNN and DeepLabV3.

Github: https://github.com/zhanghang1989/ResNeSt#pretrained-models

Paper: https://arxiv.org/abs/2004.08955v1

The Illustrated GPT-2 (Visualizing Transformer Language Models)

Visual explaining the inner-workings of transformers, and how they’ve evolved since the original paper

https://jalammar.github.io/illustrated-gpt2/

Habr ru: https://habr.com/ru/post/490842/

OpenAI Implementation: https://github.com/openai/gpt-2

Training with quantization noise for extreme model compression

Quant-Noise is a new technique to enable extreme compression of models that still deliver high performance when deployed in practical applications.

https://ai.facebook.com/blog/training-with-quantization-noise-for-extreme-model-compression/

Paper: https://arxiv.org/abs/2004.07320

GitHub: https://github.com/pytorch/fairseq/tree/master/examples/quant_noise

Building a Real Time Emotion Detection with Python

https://morioh.com/p/801c509dda99

Code: https://github.com/Dhanush45/Realtime-emotion-detectionusing-python

How I taught my computer to play Spot it! using OpenCV and Deep Learning

https://towardsdatascience.com/how-i-learned-my-computer-to-play-spot-it-using-opencv-and-deep-learning-ad1f017a3ec3

Habr ru: https://habr.com/ru/company/otus/blog/498800/

Code: https://github.com/henniedeharder/spotit/tree/master/DeepLearningSpotIt

Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation

Easy and efficient method to extend existing sentence embedding models to new languages. This allows to create multilingual versions from previously monolingual models.

Code: https://github.com/UKPLab/sentence-transformers

Paper: https://arxiv.org/abs/2004.09813v1

NVIDIA Announce MONAI Open Source AI Framework for Healthcare Research

MONAI is a PyTorch-based, open-source framework for deep learning in healthcare imaging.

https://blogs.nvidia.com/blog/2020/04/21/monai-open-source-framework-ai-healthcare/?ncid=so-twit-79443#cid=ix11_so-twit_en-us

Code: https://github.com/Project-MONAI/MONAI

Docs: https://monai.readthedocs.io/en/latest/

Google’s Optimizing Multiple Loss Functions with Loss-Conditional Training

https://ai.googleblog.com/2020/04/optimizing-multiple-loss-functions-with.html

Demo: https://sites.google.com/view/stochss/adjusting-parameters

ADJUSTABLE REAL-TIME STYLE TRANSFER: https://sites.google.com/view/stochss/adjusting-parameters

Jukebox: a new generative model for audio from OpenAI.

Jukebox, a model that generates music with singing in the raw audio domain.

openai.com/blog/jukebox

Article: cdn.openai.com/papers/jukebox.pdf

Examples: https://jukebox.openai.com/

Code: https://github.com/openai/jukebox

Measuring Information Propagation in Literary Social Network

Annotated dataset of 100 works of fiction to support tasks in natural language processing and the computational humanities.

Code: https://github.com/dbamman/litbank

Paper: https://arxiv.org/pdf/2004.13980v1.pdf

📈 Learning Convolutional Neural Networks with Interactive Visualization


Interactive visualization tool designed for non-experts to learn and examine convolutional neural networks (CNNs), a foundational deep learning model architecture.

Video: https://www.youtube.com/watch?v=HnWIHWFbuUQ&feature=youtu.be

Demo: https://poloclub.github.io/cnn-explainer/

Github: https://github.com/poloclub/cnn-explainer

Paper: https://arxiv.org/abs/2004.15004v1

NUBIA (NeUral Based Interchangeability Assessor) is a new SoTA evaluation metric for text generation

Methodology to build automatic evaluation metrics for text generation using only machine learning models as core components

https://wl-research.github.io/blog/

Github: https://github.com/wl-research/nubia

Paper: https://arxiv.org/abs/2004.14667v1

Colab: https://colab.research.google.com/drive/1_K8pOB8fRRnkBPwlcmvUNHgCr4ur8rFg

Why We Need DevOps for ML Data

https://tecton.ai/blog/devops-ml-data/

Хабр: https://habr.com/ru/company/itsumma/blog/500272/

An Implementation of ERNIE For Language Understanding (including Pre-training models and Fine-tuning tools)

ERNIE 2.0 is a continual pre-training framework for language understanding in which pre-training tasks can be incrementally built and learned through multi-task learning.

ERNIE 2.0 from Baidu: https://github.com/PaddlePaddle/ERNIE

Dataset: https://gluebenchmark.com/tasks

Understanding Language using XLNet with autoregressive pre-training

https://medium.com/@zxiao2015/understanding-language-using-xlnet-with-autoregressive-pre-training-9c86e5bea443