An Approximation of the Universal Intelligence Measure
Abstract: The Universal Intelligence Measure is a recently proposed formal definition of intelligence. It is mathematically specified, extremely general, and captures the essence of many informal definitions of intelligence. It is based on Hutter’s Universal Artificial Intelligence theory, an extension of Ray Solomonoff’s pioneering work on universal induction. Since the Universal Intelligence Measure is only asymptotically computable, building a practical intelligence test from it is not straightforward. This paper studies the practical issues involved in developing a real-world UIM-based performance metric. Based on our investigation, we develop a prototype implementation which we use to evaluate a number of different artificial agents.
https://arxiv.org/pdf/1109.5951v2.pdf
#artificial_intelligence
Abstract: The Universal Intelligence Measure is a recently proposed formal definition of intelligence. It is mathematically specified, extremely general, and captures the essence of many informal definitions of intelligence. It is based on Hutter’s Universal Artificial Intelligence theory, an extension of Ray Solomonoff’s pioneering work on universal induction. Since the Universal Intelligence Measure is only asymptotically computable, building a practical intelligence test from it is not straightforward. This paper studies the practical issues involved in developing a real-world UIM-based performance metric. Based on our investigation, we develop a prototype implementation which we use to evaluate a number of different artificial agents.
https://arxiv.org/pdf/1109.5951v2.pdf
#artificial_intelligence
Keras Website Has been Updated
Quote from its developer (François Chollet): Keras has a new website, which includes a 100% refreshed list of developer guides and code examples.
https://keras.io/
#deep_learning #programming
Quote from its developer (François Chollet): Keras has a new website, which includes a 100% refreshed list of developer guides and code examples.
https://keras.io/
#deep_learning #programming
keras.io
Keras: Deep Learning for humans
Keras documentation
Ilya Sutskever: Deep Learning
Brief Biography: Ilya Sutskever is the co-founder of OpenAI, is one of the most cited computer scientist in history with over 165,000 citations, and to me, is one of the most brilliant and insightful minds ever in the field of deep learning. There are very few people in this world who I would rather talk to and brainstorm with about deep learning, intelligence, and life than Ilya, on and off the mic.
https://www.youtube.com/watch?v=13CZPWmke6A
#deep_learning #artificial_intelligence #reinforcement_learning
Brief Biography: Ilya Sutskever is the co-founder of OpenAI, is one of the most cited computer scientist in history with over 165,000 citations, and to me, is one of the most brilliant and insightful minds ever in the field of deep learning. There are very few people in this world who I would rather talk to and brainstorm with about deep learning, intelligence, and life than Ilya, on and off the mic.
https://www.youtube.com/watch?v=13CZPWmke6A
#deep_learning #artificial_intelligence #reinforcement_learning
YouTube
Ilya Sutskever: Deep Learning | Lex Fridman Podcast #94
Ilya Sutskever is the co-founder of OpenAI, is one of the most cited computer scientist in history with over 165,000 citations, and to me, is one of the most brilliant and insightful minds ever in the field of deep learning. There are very few people in this…
Unsupervised learning models of primary cortical receptive fields and receptive field plasticity
Abstract: The efficient coding hypothesis holds that neural receptive fields are adapted to the statistics of the environment, but is agnostic to the timescale of this adaptation, which occurs on both evolutionary and developmental timescales. In this work we focus on that component of adaptation which occurs during an organism's lifetime, and show that a number of unsupervised feature learning algorithms can account for features of normal receptive field properties across multiple primary sensory cortices. Furthermore, we show that the same algorithms account for altered receptive field properties in response to experimentally altered environmental statistics. Based on these modeling results we propose these models as phenomenological models of receptive field plasticity during an organism's lifetime. Finally, due to the success of the same models in multiple sensory areas, we suggest that these algorithms may provide a constructive realization of the theory, first proposed by Mountcastle (1978), that a qualitatively similar learning algorithm acts throughout primary sensory cortices.
https://papers.nips.cc/paper/4331-unsupervised-learning-models-of-primary-cortical-receptive-fields-and-receptive-field-plasticity
#neural_network #neuroscience
Abstract: The efficient coding hypothesis holds that neural receptive fields are adapted to the statistics of the environment, but is agnostic to the timescale of this adaptation, which occurs on both evolutionary and developmental timescales. In this work we focus on that component of adaptation which occurs during an organism's lifetime, and show that a number of unsupervised feature learning algorithms can account for features of normal receptive field properties across multiple primary sensory cortices. Furthermore, we show that the same algorithms account for altered receptive field properties in response to experimentally altered environmental statistics. Based on these modeling results we propose these models as phenomenological models of receptive field plasticity during an organism's lifetime. Finally, due to the success of the same models in multiple sensory areas, we suggest that these algorithms may provide a constructive realization of the theory, first proposed by Mountcastle (1978), that a qualitatively similar learning algorithm acts throughout primary sensory cortices.
https://papers.nips.cc/paper/4331-unsupervised-learning-models-of-primary-cortical-receptive-fields-and-receptive-field-plasticity
#neural_network #neuroscience
papers.nips.cc
Unsupervised learning models of primary cortical receptive fields and receptive field plasticity
Electronic Proceedings of Neural Information Processing Systems
Reinforcement learning in the brain
Abstract: A wealth of research focuses on the decision-making processes that animals and humans employ when selecting actions in the face of reward and punishment. Initially such work stemmed from psychological investigations of conditioned behavior, and explanations of these in terms of computational models. Increasingly, analysis at the computational level has drawn on ideas from reinforcement learning, which provide a normative framework within which decision-making can be analyzed. More recently, the fruits of these extensive lines of research have made contact with investigations into the neural basis of decision making. Converging evidence now links reinforcement learning to specific neural substrates, assigning them precise computational roles. Specifically, electrophysiological recordings in behaving animals and functional imaging of human decision-making have revealed in the brain the existence of a key reinforcement learning signal, the temporal difference reward prediction error. Here, we first introduce the formal reinforcement learning framework. We then review the multiple lines of evidence linking reinforcement learning to the function of dopaminergic neurons in the mammalian midbrain and to more recent data from human imaging experiments. We further extend the discussion to aspects of learning not associated with phasic dopamine signals, such as learning of goal-directed responding that may not be dopamine-dependent, and learning about the vigor (or rate) with which actions should be performed that has been linked to tonic aspects of dopaminergic signaling. We end with a brief discussion of some of the limitations of the reinforcement learning framework, highlighting questions for future research.
https://psycnet.apa.org/record/2009-07078-003
#reinforcement_learning #neuroscience
Abstract: A wealth of research focuses on the decision-making processes that animals and humans employ when selecting actions in the face of reward and punishment. Initially such work stemmed from psychological investigations of conditioned behavior, and explanations of these in terms of computational models. Increasingly, analysis at the computational level has drawn on ideas from reinforcement learning, which provide a normative framework within which decision-making can be analyzed. More recently, the fruits of these extensive lines of research have made contact with investigations into the neural basis of decision making. Converging evidence now links reinforcement learning to specific neural substrates, assigning them precise computational roles. Specifically, electrophysiological recordings in behaving animals and functional imaging of human decision-making have revealed in the brain the existence of a key reinforcement learning signal, the temporal difference reward prediction error. Here, we first introduce the formal reinforcement learning framework. We then review the multiple lines of evidence linking reinforcement learning to the function of dopaminergic neurons in the mammalian midbrain and to more recent data from human imaging experiments. We further extend the discussion to aspects of learning not associated with phasic dopamine signals, such as learning of goal-directed responding that may not be dopamine-dependent, and learning about the vigor (or rate) with which actions should be performed that has been linked to tonic aspects of dopaminergic signaling. We end with a brief discussion of some of the limitations of the reinforcement learning framework, highlighting questions for future research.
https://psycnet.apa.org/record/2009-07078-003
#reinforcement_learning #neuroscience
A Deep Reinforcement Learning course presented by Stanford University (Winter 2019)
https://web.stanford.edu/class/cs234/index.html
#deep_reinforcement_learning
https://web.stanford.edu/class/cs234/index.html
#deep_reinforcement_learning
probability_cheatsheet.pdf
789.3 KB
Probability Theory Cheat sheet
#probability_theory
#probability_theory
Exploration Strategies in Deep Reinforcement Learning
Summary: Exploitation versus exploration is a critical topic in Reinforcement Learning. We’d like the RL agent to find the best solution as fast as possible. However, in the meantime, committing to solutions too quickly without enough exploration sounds pretty bad, as it could lead to local minima or total failure. Modern RL algorithms that optimize for the best returns can achieve good exploitation quite efficiently, while exploration remains more like an open topic.
Intro: I would like to discuss several common exploration strategies in Deep RL here. As this is a very big topic, my post by no means can cover all the important subtopics. I plan to update it periodically and keep further enriching the content gradually in time.
https://lilianweng.github.io/lil-log/2020/06/07/exploration-strategies-in-deep-reinforcement-learning.html
#deep_reinforcement_learning
Summary: Exploitation versus exploration is a critical topic in Reinforcement Learning. We’d like the RL agent to find the best solution as fast as possible. However, in the meantime, committing to solutions too quickly without enough exploration sounds pretty bad, as it could lead to local minima or total failure. Modern RL algorithms that optimize for the best returns can achieve good exploitation quite efficiently, while exploration remains more like an open topic.
Intro: I would like to discuss several common exploration strategies in Deep RL here. As this is a very big topic, my post by no means can cover all the important subtopics. I plan to update it periodically and keep further enriching the content gradually in time.
https://lilianweng.github.io/lil-log/2020/06/07/exploration-strategies-in-deep-reinforcement-learning.html
#deep_reinforcement_learning
Lil'Log
Exploration Strategies In Deep Reinforcement Learning
TensorFlow Probability: Learning with confidence
TensorFlow Probability (TFP) is a Python library built on TensorFlow that makes it easy to combine probabilistic models and deep learning on modern hardware (TPU, GPU). It's for data scientists, statisticians, and ML researchers/practitioners who want to encode domain knowledge to understand data and make predictions with uncertainty estimates. In this talk we focus on the "layers" module and demonstrate how TFP "distributions" fit naturally with Keras to enable estimating aleatoric and/or epistemic uncertainty.
Website: https://www.tensorflow.org/probability
Introduction Video: https://www.youtube.com/watch?v=BrwKURU-wpk
#tensorflow #machine_learning
TensorFlow Probability (TFP) is a Python library built on TensorFlow that makes it easy to combine probabilistic models and deep learning on modern hardware (TPU, GPU). It's for data scientists, statisticians, and ML researchers/practitioners who want to encode domain knowledge to understand data and make predictions with uncertainty estimates. In this talk we focus on the "layers" module and demonstrate how TFP "distributions" fit naturally with Keras to enable estimating aleatoric and/or epistemic uncertainty.
Website: https://www.tensorflow.org/probability
Introduction Video: https://www.youtube.com/watch?v=BrwKURU-wpk
#tensorflow #machine_learning
TensorFlow
TensorFlow Probability
A library to combine probabilistic models and deep learning on modern hardware (TPU, GPU) for data scientists, statisticians, ML researchers, and practitioners.
Are we done with ImageNet?
Abstract: Yes, and no. We ask whether recent progress on the ImageNet classification benchmark continues to represent meaningful generalization, or whether the community has started to overfit to the idiosyncrasies of its labeling procedure. We therefore develop a significantly more robust procedure for collecting human annotations of the ImageNet validation set. Using these new labels, we reassess the accuracy of recently proposed ImageNet classifiers, and find their gains to be substantially smaller than those reported on the original labels. Furthermore, we find the original ImageNet labels to no longer be the best predictors of this independently-collected set, indicating that their usefulness in evaluating vision models may be nearing an end. Nevertheless, we find our annotation procedure to have largely remedied the errors in the original labels, reinforcing ImageNet as a powerful benchmark for future research in visual recognition.
https://arxiv.org/abs/2006.07159
#benchmark #image_net #computer_vision
Abstract: Yes, and no. We ask whether recent progress on the ImageNet classification benchmark continues to represent meaningful generalization, or whether the community has started to overfit to the idiosyncrasies of its labeling procedure. We therefore develop a significantly more robust procedure for collecting human annotations of the ImageNet validation set. Using these new labels, we reassess the accuracy of recently proposed ImageNet classifiers, and find their gains to be substantially smaller than those reported on the original labels. Furthermore, we find the original ImageNet labels to no longer be the best predictors of this independently-collected set, indicating that their usefulness in evaluating vision models may be nearing an end. Nevertheless, we find our annotation procedure to have largely remedied the errors in the original labels, reinforcing ImageNet as a powerful benchmark for future research in visual recognition.
https://arxiv.org/abs/2006.07159
#benchmark #image_net #computer_vision
Mathematics for Machine Learning
Summary: The fundamental mathematical tools needed to understand machine learning include linear algebra, analytic geometry, matrix decompositions, vector calculus, optimization, probability and statistics. These topics are traditionally taught in disparate courses, making it hard for data science or computer science students, or professionals, to efficiently learn the mathematics. This self-contained textbook bridges the gap between mathematical and machine learning texts, introducing the mathematical concepts with a minimum of prerequisites. It uses these concepts to derive four central machine learning methods: linear regression, principal component analysis, Gaussian mixture models and support vector machines. For students and others with a mathematical background, these derivations provide a starting point to machine learning texts. For those learning the mathematics for the first time, the methods help build intuition and practical experience with applying mathematical concepts. Every chapter includes worked examples and exercises to test understanding. Programming tutorials are offered on the book's web site.
https://mml-book.github.io/book/mml-book.pdf
#machine_learning #mathematics
Summary: The fundamental mathematical tools needed to understand machine learning include linear algebra, analytic geometry, matrix decompositions, vector calculus, optimization, probability and statistics. These topics are traditionally taught in disparate courses, making it hard for data science or computer science students, or professionals, to efficiently learn the mathematics. This self-contained textbook bridges the gap between mathematical and machine learning texts, introducing the mathematical concepts with a minimum of prerequisites. It uses these concepts to derive four central machine learning methods: linear regression, principal component analysis, Gaussian mixture models and support vector machines. For students and others with a mathematical background, these derivations provide a starting point to machine learning texts. For those learning the mathematics for the first time, the methods help build intuition and practical experience with applying mathematical concepts. Every chapter includes worked examples and exercises to test understanding. Programming tutorials are offered on the book's web site.
https://mml-book.github.io/book/mml-book.pdf
#machine_learning #mathematics
SIREN: Implicit Neural Representations with Periodic Activation Functions
Abstract: Implicitly defined, continuous, differentiable signal representations parameterized by neural networks have emerged as a powerful paradigm, offering many possible benefits over conventional representations. However, current network architectures for such implicit neural representations are incapable of modeling signals with fine detail, and fail to represent a signal's spatial and temporal derivatives, despite the fact that these are essential to many physical signals defined implicitly as the solution to partial differential equations. We propose to leverage periodic activation functions for implicit neural representations and demonstrate that these networks, dubbed sinusoidal representation networks or Sirens, are ideally suited for representing complex natural signals and their derivatives. We analyze Siren activation statistics to propose a principled initialization scheme and demonstrate the representation of images, wavefields, video, sound, and their derivatives. Further, we show how Sirens can be leveraged to solve challenging boundary value problems, such as particular Eikonal equations (yielding signed distance functions), the Poisson equation, and the Helmholtz and wave equations. Lastly, we combine Sirens with hypernetworks to learn priors over the space of Siren functions.
Paper: https://arxiv.org/abs/2006.09661
Website: https://vsitzmann.github.io/siren/
Explanatory Video: https://youtu.be/Q5g3p9Zwjrk
#deep_learning #neural_network
Abstract: Implicitly defined, continuous, differentiable signal representations parameterized by neural networks have emerged as a powerful paradigm, offering many possible benefits over conventional representations. However, current network architectures for such implicit neural representations are incapable of modeling signals with fine detail, and fail to represent a signal's spatial and temporal derivatives, despite the fact that these are essential to many physical signals defined implicitly as the solution to partial differential equations. We propose to leverage periodic activation functions for implicit neural representations and demonstrate that these networks, dubbed sinusoidal representation networks or Sirens, are ideally suited for representing complex natural signals and their derivatives. We analyze Siren activation statistics to propose a principled initialization scheme and demonstrate the representation of images, wavefields, video, sound, and their derivatives. Further, we show how Sirens can be leveraged to solve challenging boundary value problems, such as particular Eikonal equations (yielding signed distance functions), the Poisson equation, and the Helmholtz and wave equations. Lastly, we combine Sirens with hypernetworks to learn priors over the space of Siren functions.
Paper: https://arxiv.org/abs/2006.09661
Website: https://vsitzmann.github.io/siren/
Explanatory Video: https://youtu.be/Q5g3p9Zwjrk
#deep_learning #neural_network
Grounding Language in Play: A scalable approach for controlling robots with natural language
https://language-play.github.io/
#nlp #reinforcement_learning #deep_learning
https://language-play.github.io/
#nlp #reinforcement_learning #deep_learning
Synthesis and Stabilization of Complex Behaviors through Online Trajectory Optimization
Abstract: We present an online trajectory optimization method and software platform applicable to complex humanoid robots performing challenging tasks such as getting up from an arbitrary pose on the ground and recovering from large disturbances using dexterous acrobatic maneuvers. The resulting behaviors, illustrated in the attached video, are computed only 7x slower than real time, on a standard PC. The video also shows results on the acrobot problem, planar swimming and one-legged hopping. These simpler problems can already be solved in real time, without pre-computing anything
Video of their experiments: https://youtu.be/anIsw2-Lbco
Paper: https://homes.cs.washington.edu/~todorov/papers/TassaIROS12.pdf
#model_predictive_control #optimal_control #robotics
Abstract: We present an online trajectory optimization method and software platform applicable to complex humanoid robots performing challenging tasks such as getting up from an arbitrary pose on the ground and recovering from large disturbances using dexterous acrobatic maneuvers. The resulting behaviors, illustrated in the attached video, are computed only 7x slower than real time, on a standard PC. The video also shows results on the acrobot problem, planar swimming and one-legged hopping. These simpler problems can already be solved in real time, without pre-computing anything
Video of their experiments: https://youtu.be/anIsw2-Lbco
Paper: https://homes.cs.washington.edu/~todorov/papers/TassaIROS12.pdf
#model_predictive_control #optimal_control #robotics
YouTube
Model Predictive Control with iLQG and MuJoCo
Tassa, Erez and Todorov, "Synthesis and stabilization of complex behaviors through online trajectory optimization," IROS 2012.
PyTorch Internals
Summary: This article is for those of you who have used PyTorch, and thought to yourself, "It would be great if I could contribute to PyTorch," but were scared by PyTorch's behemoth of a C++ codebase. I'm not going to lie: the PyTorch codebase can be a bit overwhelming at times. The purpose of this talk is to put a map in your hands: to tell you about the basic conceptual structure of a "tensor library that supports automatic differentiation", and give you some tools and tricks for finding your way around the codebase. I'm going to assume that you've written some PyTorch before, but haven't necessarily delved deeper into how a machine learning library is written.
https://blog.ezyang.com/2019/05/pytorch-internals/
#pytorch #deep_learning
Summary: This article is for those of you who have used PyTorch, and thought to yourself, "It would be great if I could contribute to PyTorch," but were scared by PyTorch's behemoth of a C++ codebase. I'm not going to lie: the PyTorch codebase can be a bit overwhelming at times. The purpose of this talk is to put a map in your hands: to tell you about the basic conceptual structure of a "tensor library that supports automatic differentiation", and give you some tools and tricks for finding your way around the codebase. I'm going to assume that you've written some PyTorch before, but haven't necessarily delved deeper into how a machine learning library is written.
https://blog.ezyang.com/2019/05/pytorch-internals/
#pytorch #deep_learning
An operator view of policy gradient methods
Abstract: We cast policy gradient methods as the repeated application of two operators: a policy improvement operator I, which maps any policy π to a better one Iπ, and a projection operator P, which finds the best approximation of Iπ in the set of realizable policies. We use this framework to introduce operator-based versions of traditional policy gradient methods such as Reinforce and PPO, which leads to a better understanding of their original counterparts. We also use the understanding we develop of the role of I and P to propose a new global lower bound of the expected return. This new perspective allows us to further bridge the gap between policy-based and value-based methods, showing how Reinforce and the Bellman optimality operator, for example, can be seen as two sides of the same coin.
https://arxiv.org/pdf/2006.11266.pdf
#reinforcement_learning #policy_iteration #value_iteration
Abstract: We cast policy gradient methods as the repeated application of two operators: a policy improvement operator I, which maps any policy π to a better one Iπ, and a projection operator P, which finds the best approximation of Iπ in the set of realizable policies. We use this framework to introduce operator-based versions of traditional policy gradient methods such as Reinforce and PPO, which leads to a better understanding of their original counterparts. We also use the understanding we develop of the role of I and P to propose a new global lower bound of the expected return. This new perspective allows us to further bridge the gap between policy-based and value-based methods, showing how Reinforce and the Bellman optimality operator, for example, can be seen as two sides of the same coin.
https://arxiv.org/pdf/2006.11266.pdf
#reinforcement_learning #policy_iteration #value_iteration
Neural Architecture Search without Training
Abstract: The time and effort involved in hand-designing deep neural networks is immense. This has prompted the development of Neural Architecture Search (NAS) techniques to automate this design. However, NAS algorithms tend to be extremely slow and expensive; they need to train vast numbers of candidate networks to inform the search process. This could be remedied if we could infer a network's trained accuracy from its initial state. In this work, we examine how the linear maps induced by data points correlate for untrained network architectures in the NAS-Bench-201 search space, and motivate how this can be used to give a measure of modelling flexibility which is highly indicative of a network's trained performance. We incorporate this measure into a simple algorithm that allows us to search for powerful networks without any training in a matter of seconds on a single GPU.
Explanatory Video: https://www.youtube.com/watch?v=a6v92P0EbJc
GitHub Repo: https://github.com/BayesWatch/nas-without-training
Paper: https://arxiv.org/abs/2006.04647
#deep_learning #neural_architecture_search
Abstract: The time and effort involved in hand-designing deep neural networks is immense. This has prompted the development of Neural Architecture Search (NAS) techniques to automate this design. However, NAS algorithms tend to be extremely slow and expensive; they need to train vast numbers of candidate networks to inform the search process. This could be remedied if we could infer a network's trained accuracy from its initial state. In this work, we examine how the linear maps induced by data points correlate for untrained network architectures in the NAS-Bench-201 search space, and motivate how this can be used to give a measure of modelling flexibility which is highly indicative of a network's trained performance. We incorporate this measure into a simple algorithm that allows us to search for powerful networks without any training in a matter of seconds on a single GPU.
Explanatory Video: https://www.youtube.com/watch?v=a6v92P0EbJc
GitHub Repo: https://github.com/BayesWatch/nas-without-training
Paper: https://arxiv.org/abs/2006.04647
#deep_learning #neural_architecture_search
YouTube
Neural Architecture Search without Training (Paper Explained)
#ai #research #machinelearning
Neural Architecture Search is typically very slow and resource-intensive. A meta-controller has to train many hundreds or thousands of different models to find a suitable building plan. This paper proposes to use statistics…
Neural Architecture Search is typically very slow and resource-intensive. A meta-controller has to train many hundreds or thousands of different models to find a suitable building plan. This paper proposes to use statistics…
An Introduction to Deep Reinforcement Learning
Abstract: Deep reinforcement learning is the combination of reinforcement learning (RL) and deep learning. This field of research has been able to solve a wide range of complex decisionmaking tasks that were previously out of reach for a machine. Thus, deep RL opens up many new applications in domains such as healthcare, robotics, smart grids, finance, and many more. This manuscript provides an introduction to deep reinforcement learning models, algorithms and techniques. Particular focus is on the aspects related to generalization and how deep RL can be used for practical applications. We assume the reader is familiar with basic machine learning concepts.
Paper: https://arxiv.org/pdf/1811.12560.pdf
#reinforcement_learning
#deep_learning
Abstract: Deep reinforcement learning is the combination of reinforcement learning (RL) and deep learning. This field of research has been able to solve a wide range of complex decisionmaking tasks that were previously out of reach for a machine. Thus, deep RL opens up many new applications in domains such as healthcare, robotics, smart grids, finance, and many more. This manuscript provides an introduction to deep reinforcement learning models, algorithms and techniques. Particular focus is on the aspects related to generalization and how deep RL can be used for practical applications. We assume the reader is familiar with basic machine learning concepts.
Paper: https://arxiv.org/pdf/1811.12560.pdf
#reinforcement_learning
#deep_learning
Yann LeCuN advice for an undergraduate student who aspires to become a Machine Learning Scientist in the field of Deep Learning
(0) take all the continuous math and physics class you can possibly take. If you have the choice between “iOS programming” and “quantum mechanics”, take “quantum mechanics”. In any case, take Calc I, Calc II, Calc III, Linear Algebra, Probability and Statistics, and as many physics courses as you can. But make sure you learn to program.
(1) Take an AI-related problem you are passionate about.
(2) think about it on your own
(3) once you have formed your own idea of it, start reading the literature on the problem
(4) you will find that (a) your ideas were probably a bit naive but (b) your view of the problem is slightly different from what was done before.
(5) Find a professor in your school that can help you make your ideas concrete. It might be difficult. Professors are busy and don’t have much time for undergrads. The ones with the most free time are the very junior, the very senior, and the ones who are not very active in research.
(6) If you don’ find a professor with spare time, hook up with a postdoc or PhD student in his/her lab.
(7) ask the professor if you can attend his/her lab meetings and seminars or sit in his/her class.
(8) Before you graduate, try to write a paper about your research or release a piece of open source code.
(9) Now apply to PhD programs. Forget about the “ranking” of the school for now. Find a reputable professor who works on topics that you are interested in. Pick a person whose papers you like or admire.
(10) Apply to several PhD programs in the schools of the above-mentioned professors and mention in your letter that you’d like to work with that professor but would be open to work with others.
(11) ask your undergrad professor to write a recommendation letter for you. It’s maximally efficient if your undergrad professor is known by your favorite PhD advisor.
(12) if you don’t get accepted in one of your favorite PhD programs, get a job at Facebook or Google and try to get a gig as an engineer assisting research scientists at FAIR or Google Brain.
(13) publish a papers with the research scientists in question. Then re-apply to PhD programs and ask the FAIR or Google scientists you work with to write a recommendation letter for you.
https://www.quora.com/What%E2%80%99s-your-advice-for-undergraduate-student-who-aspires-to-be-a-research-scientist-in-deep-learning-or-related-field-one-day
#machine_learning
(0) take all the continuous math and physics class you can possibly take. If you have the choice between “iOS programming” and “quantum mechanics”, take “quantum mechanics”. In any case, take Calc I, Calc II, Calc III, Linear Algebra, Probability and Statistics, and as many physics courses as you can. But make sure you learn to program.
(1) Take an AI-related problem you are passionate about.
(2) think about it on your own
(3) once you have formed your own idea of it, start reading the literature on the problem
(4) you will find that (a) your ideas were probably a bit naive but (b) your view of the problem is slightly different from what was done before.
(5) Find a professor in your school that can help you make your ideas concrete. It might be difficult. Professors are busy and don’t have much time for undergrads. The ones with the most free time are the very junior, the very senior, and the ones who are not very active in research.
(6) If you don’ find a professor with spare time, hook up with a postdoc or PhD student in his/her lab.
(7) ask the professor if you can attend his/her lab meetings and seminars or sit in his/her class.
(8) Before you graduate, try to write a paper about your research or release a piece of open source code.
(9) Now apply to PhD programs. Forget about the “ranking” of the school for now. Find a reputable professor who works on topics that you are interested in. Pick a person whose papers you like or admire.
(10) Apply to several PhD programs in the schools of the above-mentioned professors and mention in your letter that you’d like to work with that professor but would be open to work with others.
(11) ask your undergrad professor to write a recommendation letter for you. It’s maximally efficient if your undergrad professor is known by your favorite PhD advisor.
(12) if you don’t get accepted in one of your favorite PhD programs, get a job at Facebook or Google and try to get a gig as an engineer assisting research scientists at FAIR or Google Brain.
(13) publish a papers with the research scientists in question. Then re-apply to PhD programs and ask the FAIR or Google scientists you work with to write a recommendation letter for you.
https://www.quora.com/What%E2%80%99s-your-advice-for-undergraduate-student-who-aspires-to-be-a-research-scientist-in-deep-learning-or-related-field-one-day
#machine_learning
Quora
What’s your advice for undergraduate student who aspires to be a research scientist in deep learning or related field one day?
Answer (1 of 8): * (0) take all the continuous math and physics class you can possibly take. If you have the choice between “iOS programming” and “quantum mechanics”, take “quantum mechanics”. In any case, take Calc I, Calc II, Calc III, Linear Algebra,…
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020)
Abstract: In this tutorial article, we aim to provide the reader with the conceptual tools needed to get started on research on offline reinforcement learning algorithms: reinforcement learning algorithms that utilize previously collected data, without additional online data collection. Offline reinforcement learning algorithms hold tremendous promise for making it possible to turn large datasets into powerful decision making engines. Effective offline reinforcement learning methods would be able to extract policies with the maximum possible utility out of the available data, thereby allowing automation of a wide range of decision-making domains, from healthcare and education to robotics. However, the limitations of current algorithms make this difficult. We will aim to provide the reader with an understanding of these challenges, particularly in the context of modern deep reinforcement learning methods, and describe some potential solutions that have been explored in recent work to mitigate these challenges, along with recent applications, and a discussion of perspectives on open problems in the field.
Paper: https://arxiv.org/abs/2005.01643
#reinforcement_learning #offline_reinforcement_learning
Abstract: In this tutorial article, we aim to provide the reader with the conceptual tools needed to get started on research on offline reinforcement learning algorithms: reinforcement learning algorithms that utilize previously collected data, without additional online data collection. Offline reinforcement learning algorithms hold tremendous promise for making it possible to turn large datasets into powerful decision making engines. Effective offline reinforcement learning methods would be able to extract policies with the maximum possible utility out of the available data, thereby allowing automation of a wide range of decision-making domains, from healthcare and education to robotics. However, the limitations of current algorithms make this difficult. We will aim to provide the reader with an understanding of these challenges, particularly in the context of modern deep reinforcement learning methods, and describe some potential solutions that have been explored in recent work to mitigate these challenges, along with recent applications, and a discussion of perspectives on open problems in the field.
Paper: https://arxiv.org/abs/2005.01643
#reinforcement_learning #offline_reinforcement_learning
Backward Feature Correction: How Deep Learning Performs Deep Learning
Summary: How does a 110-layer ResNet learn a high-complexity classifier using relatively few training examples and short training time? We present a theory towards explaining this in terms of hierarchical learning. We refer hierarchical learning as the learner learns to represent a complicated target function by decomposing it into a sequence of simpler functions to reduce sample and time complexity. This paper formally analyzes how multi-layer neural networks can perform such hierarchical learning efficiently and automatically by applying SGD. On the conceptual side, we present, to the best of our knowledge, the FIRST theory result indicating how deep neural networks can be sample and time efficient on certain hierarchical learning tasks, when NO KNOWN non-hierarchical algorithms (such as kernel method, linear regression over feature mappings, tensor decomposition, sparse coding, and their simple combinations) are efficient. We establish a principle called "backward feature correction", where training higher layers in the network can improve the features of lower level ones. We believe this is the key to understand the deep learning process in multi-layer neural networks.
Paper: https://arxiv.org/pdf/2001.04413.pdf
#theory #deep_learning
Summary: How does a 110-layer ResNet learn a high-complexity classifier using relatively few training examples and short training time? We present a theory towards explaining this in terms of hierarchical learning. We refer hierarchical learning as the learner learns to represent a complicated target function by decomposing it into a sequence of simpler functions to reduce sample and time complexity. This paper formally analyzes how multi-layer neural networks can perform such hierarchical learning efficiently and automatically by applying SGD. On the conceptual side, we present, to the best of our knowledge, the FIRST theory result indicating how deep neural networks can be sample and time efficient on certain hierarchical learning tasks, when NO KNOWN non-hierarchical algorithms (such as kernel method, linear regression over feature mappings, tensor decomposition, sparse coding, and their simple combinations) are efficient. We establish a principle called "backward feature correction", where training higher layers in the network can improve the features of lower level ones. We believe this is the key to understand the deep learning process in multi-layer neural networks.
Paper: https://arxiv.org/pdf/2001.04413.pdf
#theory #deep_learning