Batch Normalization is a Cause of Adversarial Vulnerability

Abstract - Batch normalization (batch norm) is often used in an attempt to stabilize and accelerate training in deep neural networks. In many cases it indeed decreases the number of parameter updates required to achieve low training error. However, it also reduces robustness to small adversarial input perturbations and noise by double-digit percentages, as we show on five standard data-sets. Furthermore, substituting weight decay for batch norm is sufficient to nullify the relationship between adversarial vulnerability and the input dimension. Our work is consistent with a mean-field analysis that found that batch norm causes exploding gradients.

Page - https://arxiv.org/abs/1905.02161

PDF - https://arxiv.org/pdf/1905.02161.pdf

Machine Learning for Physics and the Physics of Learning Tutorials"
Videos and slides, by IPAM (an NSF Math Institute at UCLA dedicated to promoting the interaction of math with other disciplines):
https://www.ipam.ucla.edu/programs/workshops/machine-learning-for-physics-and-the-physics-of-learning-tutorials/?tab=schedule
https://t.iss.one/ArtificialIntelligenceArticles

#MLP2019 #MachineLearning #Physics

DeepPrivacy: A Generative Adversarial Network for Face Anonymization https://arxiv.org/abs/1909.04538

A.I. Expert Andrew Ng Talks Artificial Intelligence
https://fortune.com/2019/09/10/a-i-s-next-big-breakthrough-eye-on-a-i/

Full Stack Deep Learning Bootcamp
(Most of) Lectures of Day 1: https://fullstackdeeplearning.com/march2019
Happy learning!
#ArtificialIntelligence #DeepLearning #MachineLearning https://t.iss.one/ArtificialIntelligenceArticles

Fairness and machine learning
Solon Barocas, Moritz Hardt, Arvind Narayanan : https://fairmlbook.org
#machinelearning #fairness #aigovernance

Meta-Learning with Implicit Gradients
Aravind Rajeswaran, Chelsea Finn, Sham Kakade, Sergey Levine : https://arxiv.org/abs/1909.04630
#MachineLearning #ArtificialIntelligence #Optimization #Control #MetaLearning

Part 1: Neural Networks Explained: Feedforward and Backpropagation
https://mlfromscratch.com/neural-networks-explained/
Part 2: Activation Functions Explained: GELU, SELU, ELU, ReLU etc.
https://mlfromscratch.com/activation-functions-explained/
What do you want to see next?

Adversarial Policy Gradient for Deep Learning Image Augmentation. https://arxiv.org/abs/1909.04108

Geometry-Aware Video Object Detection for Static Cameras. https://arxiv.org/abs/1909.03140

Geoffrey Hinton and Yann LeCun, 2018 ACM A.M. Turing Award Lecture

https://www.youtube.com/watch?v=VsnQf7exv5I&feature=youtu.be

Artificial Intelligence Detects Heart Failure From One Heartbeat With 100% Accuracy - Nicholas Fearn
https://www.forbes.com/sites/nicholasfearn/2019/09/12/artificial-intelligence-detects-heart-failure-from-one-heartbeat-with-100-accuracy

https://t.iss.one/ArtificialIntelligenceArticles

Reference : https://arxiv.org/abs/1710.09829 https://t.iss.one/ArtificialIntelligenceArticles

Master’s student position or internship Machine Learning / Deep Learning

Anomaly Detection on High Dimensional Time Series


Ref. 2019-26

Background

Many application domains increasingly require AD, when anomalies carry critical and actionable information. These include: (1) Cyber-security and intrusion detection in Cloud and IT systems, also in government, defense and security agencies; (2) Fraud detection in financial institutions; (3) Manufacturing, IoT, industry and resource exploration; (4) Healthcare; etc.

Project

We shall address the problem of detecting and predicting general anomalies in high-dimension KPI performance metrics, i.e., high dimension and dynamic range multivariate non-stationary time series collected from large Cloud / IT environments. Using Keras / TF etc., we will build an ML-based AD framework for transfer, attention and meta-learning that must remain robust also with reduced/missing and noisy training data. Besides feature engineering – e.g., selection, reduction, compression techniques – explainability will also be necessary for the model prototype.

Requirements

Data science/mining in general
Feature engineering and DL experience with RNN/CNN/xAE in particular
Hands-on experience with deep neural network models in Python, NumPy, Pandas, SciPy etc., applied to deep RNN/CNN/Autoencoders
Motivation to learn real-life time series and experiment with DL in Keras/TensorFlow/ PyTorch
About the position

The research is to be performed at IBM Research – Zurich Lab, Switzerland.

The expected duration is 3-6 months, starting as soon as possible from June 2019.

Diversity

IBM is committed to diversity at the workplace. With us you will find an open, multicultural environment. Excellent, flexible working arrangements enable both women and men to strike the desired balance between their professional development and their personal lives.

https://ai-jobs.net/job/masters-student-position-or-internship-machine-learning-deep-learning-4/

https://t.iss.one/ArtificialIntelligenceArticles

Google Patents "Generating output sequences from input sequences using neural networks"
https://www.freepatentsonline.com/10402719.html

Open-Unmix for Music Separation

📜Paper: https://joss.theoj.org/papers/571753bc54c5d6dd36382c3d801de41d
🔊Demo: https://open.unmix.app
🔥PyTorch: https://github.com/sigsep/open-unmix-pytorch
🔻NNabla: https://github.com/sigsep/open-unmix-nnabla
🔶TF2: t.b.a.
📓Colab: https://colab.research.google.com/drive/1mijF0zGWxN-KaxTnd0q6hayAlrID5fEQ

A deep learning system for differential diagnosis of skin diseases

Liu et al.

Paper: https://arxiv.org/abs/1909.05382

Blog : https://ai.googleblog.com/2019/09/using-deep-learning-to-inform.html

#deeplearning #machinelearning #healthcare

ICYMI from CVPR 2019: 3D human pose estimation in video with temporal convolutions and semi-supervised training

https://www.profillic.com/paper/arxiv:1811.11742

The authors (Facebook AI researchers) demonstrate that 3D poses in a video can be effectively estimated with a fully convolutional model based on dilated temporal convolutions over 2D keypoints.

Can you classify two class circle data using neural network with only two neurons?

https://arxiv.org/abs/1901.00109