Video and slides: GNN User Group meeting 2
In the second meeting of GNN user group, there is a discussion of new release of DGL, graph analytics on GPU, as well as new approaches for training GNNs, including those on disassortative graphs. Slides can be found in the slack channel.
In the second meeting of GNN user group, there is a discussion of new release of DGL, graph analytics on GPU, as well as new approaches for training GNNs, including those on disassortative graphs. Slides can be found in the slack channel.
YouTube
Graph Neural Networks User Group Meeting on Feb 25, 2021
Logistics and announcement on the newest release of DGL 0.6.
Our featured speakers are:
4:00 - 4:25 (PST): Gunrock: Graph Analytics on GPU (Dr. John Owens, University of California, Davis)
4:25 - 4:50 (PST): NVIDIA CuGraph - An Open-Source Package for…
Our featured speakers are:
4:00 - 4:25 (PST): Gunrock: Graph Analytics on GPU (Dr. John Owens, University of California, Davis)
4:25 - 4:50 (PST): NVIDIA CuGraph - An Open-Source Package for…
Theoretical Foundations of Graph Neural Networks
Video presentation by Petar Veličković who covers design, history, and applications of GNNs. A lot of interesting concepts such as permutation invariance and equivariance discussed. Slides can be found here.
Video presentation by Petar Veličković who covers design, history, and applications of GNNs. A lot of interesting concepts such as permutation invariance and equivariance discussed. Slides can be found here.
YouTube
Theoretical Foundations of Graph Neural Networks
Deriving graph neural networks (GNNs) from first principles, motivating their use, and explaining how they have emerged along several related research lines.
Computer Laboratory Wednesday Seminar, 17 February 2021
Slide deck: https://petar-v.com/talks/GNN…
Computer Laboratory Wednesday Seminar, 17 February 2021
Slide deck: https://petar-v.com/talks/GNN…
Trapped in a Desert Maze: AlphaZero learns to deal with graphs
A fun video and blog post about AlphaZero playing a simple game of node covering in graphs. I wish there is some human interface to play against these creatures.
A fun video and blog post about AlphaZero playing a simple game of node covering in graphs. I wish there is some human interface to play against these creatures.
YouTube
Battle of A.I. wits: 2 bots trapped in a 3D desert maze
This channel documents our research into the behavior of AI agents, or creatures, living on graphs. In this episode, we describe the development of the "Ringo" test scenario. Then we show two sample "games", set in a 3D environment. The environment is a 3D…
Graph Machine Learning research groups: Tyler Derr
I do a series of posts on the groups in graph research, previous post is here. The 24th is Tyler Derr, a young professor graph ML, who proposed signed GNNs on graphs with negative links.
Tyler Derr (~1992)
- Affiliation: Vanderbilt University
- Education: Ph.D. at Michigan State University in 2020 (advisors: Jiliang Tang)
- h-index 10
- Awards: best papers at SDM
- Interests: adversarial attacks, graph neural networks
I do a series of posts on the groups in graph research, previous post is here. The 24th is Tyler Derr, a young professor graph ML, who proposed signed GNNs on graphs with negative links.
Tyler Derr (~1992)
- Affiliation: Vanderbilt University
- Education: Ph.D. at Michigan State University in 2020 (advisors: Jiliang Tang)
- h-index 10
- Awards: best papers at SDM
- Interests: adversarial attacks, graph neural networks
Telegram
Graph Machine Learning
Graph Machine Learning research groups: Austin R. Benson
I do a series of posts on the groups in graph research, previous post is here. The 23rd is Austin R. Benson, a professor at Cornell, who together with his students recently shook the graph community…
I do a series of posts on the groups in graph research, previous post is here. The 23rd is Austin R. Benson, a professor at Cornell, who together with his students recently shook the graph community…
Paper Notes in Notion
Vitaly Kurin discovered a great format to track notes for the papers he reads. These are short and clean digestions of papers in the intersection of GNNs and RL and I would definitely recommend to look it up if you are studying the same papers. You can also create the same format in Notion by adding a new page (database -> list) and then clicking on New button selectin the properties that are necessary.
Vitaly Kurin discovered a great format to track notes for the papers he reads. These are short and clean digestions of papers in the intersection of GNNs and RL and I would definitely recommend to look it up if you are studying the same papers. You can also create the same format in Notion by adding a new page (database -> list) and then clicking on New button selectin the properties that are necessary.
yobispace on Notion
Paper Notes by Vitaly Kurin | Notion
Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
Fresh picks from ArXiv
This week on ArXiv: unsupervised community detection, reducing variance with sampling, and GNNs for quantum calculations 🧮
If I forgot to mention your paper, please shoot me a message and I will update the post.
Algorithms
* Graph Force Learning
* Recurrent Graph Neural Network Algorithm for Unsupervised Network Community Detection
* ForceNet: A Graph Neural Network for Large-Scale Quantum Calculations
* Autobahn: Automorphism-based Graph Neural Nets
* Graph Convolutional Embeddings for Recommender Systems SIGIR 2021
* On the Importance of Sampling in Learning Graph Convolutional Networks
* Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework
Software
* Implementing graph neural networks with TensorFlow-Keras
* Large Graph Convolutional Network Training with GPU-Oriented Data Communication Architecture
Survey
* Deep Graph Structure Learning for Robust Representations: A Survey
This week on ArXiv: unsupervised community detection, reducing variance with sampling, and GNNs for quantum calculations 🧮
If I forgot to mention your paper, please shoot me a message and I will update the post.
Algorithms
* Graph Force Learning
* Recurrent Graph Neural Network Algorithm for Unsupervised Network Community Detection
* ForceNet: A Graph Neural Network for Large-Scale Quantum Calculations
* Autobahn: Automorphism-based Graph Neural Nets
* Graph Convolutional Embeddings for Recommender Systems SIGIR 2021
* On the Importance of Sampling in Learning Graph Convolutional Networks
* Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework
Software
* Implementing graph neural networks with TensorFlow-Keras
* Large Graph Convolutional Network Training with GPU-Oriented Data Communication Architecture
Survey
* Deep Graph Structure Learning for Robust Representations: A Survey
MLSys 2021 Conference
MLsys is one of the main conferences on applications of ML in real-world. Accepted papers for MLSys 2021 are available here. It will also feature GNN workshop and keynote speakers from NVIDIA, PyTorch, and others. Dates are April 5-9, 2021. Registration is 25$ for students.
MLsys is one of the main conferences on applications of ML in real-world. Accepted papers for MLSys 2021 are available here. It will also feature GNN workshop and keynote speakers from NVIDIA, PyTorch, and others. Dates are April 5-9, 2021. Registration is 25$ for students.
mlsys.org
MLSys 2021 Schedule
MLSys Website
Top-10 Research Papers in AI
A new blog post about the top-10 most cited papers in AI during the last 5 years. I looked at major AI conferences and journals (excluding CV and NLP conferences).
It was quite a refreshing experience to realize that many of what we use today by default have been discovered only within the last few years. Things like Adam, Batch Norm, GCNs, etc.
A new blog post about the top-10 most cited papers in AI during the last 5 years. I looked at major AI conferences and journals (excluding CV and NLP conferences).
It was quite a refreshing experience to realize that many of what we use today by default have been discovered only within the last few years. Things like Adam, Batch Norm, GCNs, etc.
Medium
Top-10 Research Papers in AI
Each year scientists from around the world publish thousands of research papers in AI but only a few of them reach wide audiences and make a global impact in the world. Below are the top-10 most…
Deep Learning and Combinatorial Optimization IPAM Workshop
A great workshop on the intersection of ML, RL, GNNs, and combinatorial optimization. Videos are available. Topics include applications of ML to chip design, TSP, physics, integer programming and more.
A great workshop on the intersection of ML, RL, GNNs, and combinatorial optimization. Videos are available. Topics include applications of ML to chip design, TSP, physics, integer programming and more.
IPAM
Deep Learning and Combinatorial Optimization - IPAM
If We Draw Graphs Like This, We Can Change Computers Forever
The title is catchy, but the article is "only" about improvement for dynamic planarity testing problem. Planarity testing is well-studied problem for testing if a graph can be drawn without crossing edges and O(n) algorithms are known. This article on the other hand studies the case when the edges may be added and removed and the question is how to redraw the graph so that it becomes planar. The results were published at STOC'20.
The title is catchy, but the article is "only" about improvement for dynamic planarity testing problem. Planarity testing is well-studied problem for testing if a graph can be drawn without crossing edges and O(n) algorithms are known. This article on the other hand studies the case when the edges may be added and removed and the question is how to redraw the graph so that it becomes planar. The results were published at STOC'20.
Popular Mechanics
If We Draw Graphs Like This, We Can Change Computers Forever
Engineers could use this breakthrough in graph theory to design wildly efficient quantum computer chips.
A Complete Beginner's Guide to G-Invariant Neural Networks
A tutorial by S. Chandra Mouli and Bruno Ribeiro about G-invariant neural networks, eigenvectors, invariant subspaces, transformation groups, Reynolds operator, and more. Soon, there should be more tutorials on the topic of invariance and linear algebra.
A tutorial by S. Chandra Mouli and Bruno Ribeiro about G-invariant neural networks, eigenvectors, invariant subspaces, transformation groups, Reynolds operator, and more. Soon, there should be more tutorials on the topic of invariance and linear algebra.
www.invariances.org
Foundations of Invariant and Equivariant Representation Learning
Graph Transformer: A Generalization of Transformers to Graphs
A blog post by Vijay Prakash Dwivedi that discusses their paper A Generalization of Transformer Networks to Graphs with Xavier Bresson at 2021 AAAI Workshop (DLG-AAAI’21). It looks like a generalization of GAT network with batch norm and positional encodings. It still though aggregates via local neighborhoods.
My feeling after studying heterophily is that we will see more works that go beyond local neighborhoods and maybe will define neighborhoods not as something that is given by the graph topology but as something we have to learn. For example, we can define attention from each node to all other nodes in the graph and treat the distances in the graph as additional features. It could be difficult to scale so sampling methods should be employed I guess, but it seems allowing the network to decide which nodes are important for aggregation could be a better way to go.
A blog post by Vijay Prakash Dwivedi that discusses their paper A Generalization of Transformer Networks to Graphs with Xavier Bresson at 2021 AAAI Workshop (DLG-AAAI’21). It looks like a generalization of GAT network with batch norm and positional encodings. It still though aggregates via local neighborhoods.
My feeling after studying heterophily is that we will see more works that go beyond local neighborhoods and maybe will define neighborhoods not as something that is given by the graph topology but as something we have to learn. For example, we can define attention from each node to all other nodes in the graph and treat the distances in the graph as additional features. It could be difficult to scale so sampling methods should be employed I guess, but it seems allowing the network to decide which nodes are important for aggregation could be a better way to go.
Medium
Graph Transformer: A Generalization of Transformers to Graphs
We generalize Transformers to arbitrary graphs by extending key design aspects of attention and positional encodings from NLP to graphs.
PyTorch Geometric Temporal
PyG-Temporal is an extension of PyG for temporal graphs. It now includes more than 10 GNN models and several datasets. With world being dynamic I see more and more applications when standard GNN wouldn't work and one needs to resort to dynamic GNNs.
PyG-Temporal is an extension of PyG for temporal graphs. It now includes more than 10 GNN models and several datasets. With world being dynamic I see more and more applications when standard GNN wouldn't work and one needs to resort to dynamic GNNs.
Fresh picks from ArXiv
This week on ArXiv: comparison of using feature vs edge information, SOTA for heterogeneous graphs, surveys on efficient training of GNNs 🏃
If I forgot to mention your paper, please shoot me a message and I will update the post.
GNNs
* Spatio-temporal Modeling for Large-scale Vehicular Networks Using Graph Convolutional Networks
* R-GSN: The Relation-based Graph Similar Network for Heterogeneous Graph
* On the Equivalence Between Temporal and Static Graph Representations for Observational Predictions with Bruno Ribeiro
* Deep Graph Matching under Quadratic Constraint
* Should Graph Neural Networks Use Features, Edges, Or Both?
* Deep graph convolution neural network with non-negative matrix factorization for community discovery
* Graph Neural Networks Inspired by Classical Iterative Algorithms
* Size-Invariant Graph Representations for Graph Classification Extrapolations
Survey
* Sampling methods for efficient training of graph convolutional networks: A survey
* Graph Metrics for Internet Robustness -- A Survey
* A Taxonomy for Classification and Comparison of Dataflows for GNN Accelerators
This week on ArXiv: comparison of using feature vs edge information, SOTA for heterogeneous graphs, surveys on efficient training of GNNs 🏃
If I forgot to mention your paper, please shoot me a message and I will update the post.
GNNs
* Spatio-temporal Modeling for Large-scale Vehicular Networks Using Graph Convolutional Networks
* R-GSN: The Relation-based Graph Similar Network for Heterogeneous Graph
* On the Equivalence Between Temporal and Static Graph Representations for Observational Predictions with Bruno Ribeiro
* Deep Graph Matching under Quadratic Constraint
* Should Graph Neural Networks Use Features, Edges, Or Both?
* Deep graph convolution neural network with non-negative matrix factorization for community discovery
* Graph Neural Networks Inspired by Classical Iterative Algorithms
* Size-Invariant Graph Representations for Graph Classification Extrapolations
Survey
* Sampling methods for efficient training of graph convolutional networks: A survey
* Graph Metrics for Internet Robustness -- A Survey
* A Taxonomy for Classification and Comparison of Dataflows for GNN Accelerators
Large-scale graph machine learning challenge (OGB-LSC) at KDD Cup 2021
OGB-LSC is a collection of three graph datasets—PCQM4M-LSC, WikiKG90M-LSC, and MAG240M-LSC—that are orders of magnitude larger than existing ones. The three datasets correspond to link prediction, graph regression, and node classification tasks, respectively. The goal of OGB-LSC is to empower the community to discover innovative solutions for large-scale graph ML.
The competition will be from March 15th, 2021 until June 8th, 2021 and the winners will be notified by mid-June 2021. The winners will be honored at the KDD 2021 opening ceremony and will present their solutions at the KDD Cup workshop during the conference.
The graphs are indeed big, with the largest size 168 GB, and it's interesting what approaches can be used to solve these problems.
OGB-LSC is a collection of three graph datasets—PCQM4M-LSC, WikiKG90M-LSC, and MAG240M-LSC—that are orders of magnitude larger than existing ones. The three datasets correspond to link prediction, graph regression, and node classification tasks, respectively. The goal of OGB-LSC is to empower the community to discover innovative solutions for large-scale graph ML.
The competition will be from March 15th, 2021 until June 8th, 2021 and the winners will be notified by mid-June 2021. The winners will be honored at the KDD 2021 opening ceremony and will present their solutions at the KDD Cup workshop during the conference.
The graphs are indeed big, with the largest size 168 GB, and it's interesting what approaches can be used to solve these problems.
Open Graph Benchmark
OGB-LSC @ KDD Cup 2021
Why a Large-Scale Graph ML Competiton? Machine Learning (ML) on graphs has attracted immense attention in recent years because of the prevalence of graph-structured data in real-world applications. Modern application domains include web-scale social networks…
A Tale of Three Implicit Planners and the XLVIN agent
A video presentation by Petar Veličković about implicit planners, which could be seen as a middle-ground between model-based and model-free approaches for RL planning problems. The talk covers three popular implicit planners: VIN, ATreeC and XLVIN. All three focus on the recently popularised idea of algorithmically aligning to a planning algorithm, but with different realisations.
A video presentation by Petar Veličković about implicit planners, which could be seen as a middle-ground between model-based and model-free approaches for RL planning problems. The talk covers three popular implicit planners: VIN, ATreeC and XLVIN. All three focus on the recently popularised idea of algorithmically aligning to a planning algorithm, but with different realisations.
YouTube
Petar Veličković - A Tale of Three Implicit Planners and the XLVIN agent
Deep reinforcement learning (RL) is one of the most active topics in today's machine learning landscape. While it offers a remarkably powerful and general blueprint for solving generic problems, it is usually data-hungry, often requiring observations of millions…
Graph Machine Learning research groups: Yaron Lipman
I do a series of posts on the groups in graph research, previous post is here. The 25th is Yaron Lipman, a professor in Israel, who has been co-authoring many papers on equivariances and the power of GNNs.
Yaron Lipman (~1980)
- Affiliation: Weizmann Institute of Science
- Education: Ph.D. at Tel Aviv University in 2008 (advisors: David Levin and Daniel Cohen-Or)
- h-index 41
- Interests: geometric deep learning, meshes, 3d point clouds, equivariant networks
I do a series of posts on the groups in graph research, previous post is here. The 25th is Yaron Lipman, a professor in Israel, who has been co-authoring many papers on equivariances and the power of GNNs.
Yaron Lipman (~1980)
- Affiliation: Weizmann Institute of Science
- Education: Ph.D. at Tel Aviv University in 2008 (advisors: David Levin and Daniel Cohen-Or)
- h-index 41
- Interests: geometric deep learning, meshes, 3d point clouds, equivariant networks
Telegram
Graph Machine Learning
Graph Machine Learning research groups: Tyler Derr
I do a series of posts on the groups in graph research, previous post is here. The 24th is Tyler Derr, a young professor graph ML, who proposed signed GNNs on graphs with negative links.
Tyler Derr (~1992)…
I do a series of posts on the groups in graph research, previous post is here. The 24th is Tyler Derr, a young professor graph ML, who proposed signed GNNs on graphs with negative links.
Tyler Derr (~1992)…
GML Express: large-scale challenge, top papers in AI, and implicit planners.
Another issue of my newsletter. So I finally solved a struggle for me what to write this newsletter about: news or insights. GML express will cover news (which you mostly should get anyway from this channel) and GML In-Depth should cover my insights.
In this GML express you will find a bunch of learning materials, recent video presentations, blog posts, and announcements.
Another issue of my newsletter. So I finally solved a struggle for me what to write this newsletter about: news or insights. GML express will cover news (which you mostly should get anyway from this channel) and GML In-Depth should cover my insights.
In this GML express you will find a bunch of learning materials, recent video presentations, blog posts, and announcements.
Graph Machine Learning
GML Express: large-scale challenge, top papers in AI, and implicit planners.
“Why do old men wake so early? Is it to have one longer day?” Ernest Hemingway
Fresh picks from ArXiv
This week on ArXiv: large-scale node prediction competition, GNNs for 3D objects, and improved performance for imbalanced classification 👯♀️
If I forgot to mention your paper, please shoot me a message and I will update the post.
GNNs
* OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs with Jure Leskovec
* Concentric Spherical GNN for 3D Representation Learning with Le Song
* GraphSMOTE: Imbalanced Node Classification on Graphs with Graph Neural Networks
* Catastrophic Forgetting in Deep Graph Networks: an Introductory Benchmark for Graph Classification
Applications
* A graph theoretical approach to the firebreak locating problem
* Exploiting Isomorphic Subgraphs in SAT (Long version)
* ChronoR: Rotation Based Temporal Knowledge Graph Embedding
* Code Completion by Modeling Flattened Abstract Syntax Trees as Graphs
* NetVec: A Scalable Hypergraph Embedding System
Software
* GNNerator: A Hardware/Software Framework for Accelerating Graph Neural Networks
* Characterizing the Communication Requirements of GNN Accelerators: A Model-Based Approach
This week on ArXiv: large-scale node prediction competition, GNNs for 3D objects, and improved performance for imbalanced classification 👯♀️
If I forgot to mention your paper, please shoot me a message and I will update the post.
GNNs
* OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs with Jure Leskovec
* Concentric Spherical GNN for 3D Representation Learning with Le Song
* GraphSMOTE: Imbalanced Node Classification on Graphs with Graph Neural Networks
* Catastrophic Forgetting in Deep Graph Networks: an Introductory Benchmark for Graph Classification
Applications
* A graph theoretical approach to the firebreak locating problem
* Exploiting Isomorphic Subgraphs in SAT (Long version)
* ChronoR: Rotation Based Temporal Knowledge Graph Embedding
* Code Completion by Modeling Flattened Abstract Syntax Trees as Graphs
* NetVec: A Scalable Hypergraph Embedding System
Software
* GNNerator: A Hardware/Software Framework for Accelerating Graph Neural Networks
* Characterizing the Communication Requirements of GNN Accelerators: A Model-Based Approach
DIG: Dive into Graphs library
A new python library DIG (and paper) in PyTorch for several graph tasks:
* Graph Generation
* Self-supervised Learning on Graphs
* Explainability of Graph Neural Networks
* Deep Learning on 3D Graphs
A new python library DIG (and paper) in PyTorch for several graph tasks:
* Graph Generation
* Self-supervised Learning on Graphs
* Explainability of Graph Neural Networks
* Deep Learning on 3D Graphs