CS224W: Machine Learning with Graphs 2021

CS224W is one of the most popular graph courses by Jure Leskovec at Stanford. This year includes extra topics such as label propagation, scalability of GNNs, and graph nets for science and biology. The slides for the first 6 out of 20 lectures are available.

Video: GNN User Group

The video from the first meeting of GNN user group talks about the usage and next release of DGL and featuring Le Song with combinatorial optimization talk.

GML Newsletter: Interpolation and Extrapolation of Graph Neural Networks

The new issue of the newsletter is about generalization of GNNs. Compared to the study of expressive power, there are fewer works about generalization. Nonetheless, I gathered the most exciting research I found on this topic, which I hope will familiarize you with this research direction.

Fresh picks from ArXiv
This week on ArXiv: tensorflow GNN library, survey on graph-based kNN search, and automation of peer review? 🧐

Conferences
Interpreting and Unifying Graph Neural Networks with An Optimization Framework WWW 2021
A Graph-based Relevance Matching Model for Ad-hoc Retrieval AAAI 2021

Software
Efficient Graph Deep Learning in TensorFlow with tf_geometric

Survey
* A Comprehensive Survey and Experimental Comparison of Graph-Based Approximate Nearest Neighbor Search
* Graph Neural Network for Traffic Forecasting: A Survey
* Can We Automate Scientific Reviewing?

How many paths of length k exist in a graph?

In case you are preparing for the next interview, here is a nice post describing several solutions to a common interview problem: count the number of possible walks between two points in a graph. The problem is not as easy as it seems.

Tutorial: Graph Neural Networks: Models and Applications

A new tutorial covering robustness, attacks, scalability and self-supervised learning for GNN models at AAAI 2021. Slides and video are available.

Sberloga Talk

In case you speak Russian I will be presenting today our ICLR 2021 work about combination of GBDT with GNN on graphs with tabular features. The talk will be 19-00 MSK time. Zoom link will be shared soon at @sberlogawithgraphs. For more videos from Sberloga, subscribe here: https://www.youtube.com/c/SBERLOGA

Cleora Paper

I already wrote about Cleora, an unsupervised embedding library, now there is a paper explaining details of it. The algorithm is just some form of matrix multiplication, yet it shows better performance for link prediction metrics and running time than Pytorch-BigGraph, DeepWalk and others.

Deep Learning on Graphs: Method and Applications (DLG-AAAI’21)

AAAI workshop on GNNs with a amazing list of speakers including Jure Leskovec, Max Welling, Xavier Bresson, and many others. Zoom link is available here, starting today 5pm Europe time.

Fresh picks from ArXiv
This week on ArXiv: link prediction in KGs, unsupervised embedding library, and reconstruction conjecture for up to 13 vertices 💡

Conferences

* Exploring the Subgraph Density-Size Trade-off via the Lovász Extension WSDM 2021
* Effective and Scalable Clustering on Massive Attributed Graphs WebConf 2021

GNN
* Enhance Information Propagation for Graph Neural Network by Heterogeneous Aggregations
* CF-GNNExplainer: Counterfactual Explanations for Graph Neural Networks

Applications
* Exploring the Limits of Few-Shot Link Prediction in Knowledge Graphs with William L. Hamilton
* GNN-RL Compression: Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learning

Software
* Cleora: A Simple, Strong and Scalable Graph Embedding Scheme

Math
* Reconstruction of small graphs and tournaments

How to get started with Graph Machine Learning

In a new post, Aleksa Gordić talks in depth about graph ML, its applications and shares useful resources to get you started in this world.

Graphs and More Complex Structures for Learning and Reasoning Workshop

A workshop at AAAI 2021 featuring the talk about learning knowledge graph representations for zero-shot learning in NLP and vision.

Graph Neural Networks from the First Principles

Petar Veličković will give a talk on 17 Feb about how GNNs appeared in different disciplines and how you can derive GNNs from permutation invariance. Petar has long worked in this field, knowing inside and out graph nets, so I strongly recommend to visit his talk. The link is here.

Job Posting for Research Scientist at NEC Labs Europe

Several researcher positions are available at NEC Lab Europe, a research institute with a focus on CS/ML applications in life sciences. One includes working with Dr. Mathias Niepert who has been publishing many works in graph ML field. Deadline is 31st March.

Graph Machine Learning research groups: Austin R. Benson

I do a series of posts on the groups in graph research, previous post is here. The 23rd is Austin R. Benson, a professor at Cornell, who together with his students recently shook the graph community by showing that label propagation works really well compared to GNN.

Austin R. Benson (~1990)
- Affiliation: Cornell
- Education: Ph.D. at Stanford in 2017 (advisors: Jure Leskovec)
- h-index 21
- Awards: best research papers at KDD, ASONAM, Kavli Fellow
- Interests: label propagation, clustering, network algorithms

Learning mesh-based simulation with Graph Networks

Another work by DeepMind (ICLR '21) on how to simulate physical systems with GNN. The principle is the same as in their previous works: get a graph for a system, process it with GNN, obtain acceleration for each node, and provide it to Euler integrator to obtain positions of each node in the next step. Again, very cool visualizations.

Fresh picks from ArXiv
This week on ArXiv: connection between heterohpily and oversmoothing, SOTA unsupervised model, and MCTS for explainability 📞

GNNs
* Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks with Danai Koutra
* Bootstrapped Representation Learning on Graphs with Petar Veličković
* On Explainability of Graph Neural Networks via Subgraph Explorations
* Spherical Message Passing for 3D Graph Networks
* A Unified Lottery Ticket Hypothesis for Graph Neural Networks
* Graph Convolution for Semi-Supervised Classification: Improved Linear Separability and Out-of-Distribution Generalization
* SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networks

Conferences
* Learning Intents behind Interactions with Knowledge Graph for Recommendation WWW 2021
* Multiplex Bipartite Network Embedding using Dual Hypergraph Convolutional Networks WWW 2021
* Model-Agnostic Graph Regularization for Few-Shot Learning with Jure Leskovec

Recent applications of expanders to graph algorithms

Informally, a graph is expander if the nodes are robustly connected, i.e. removing some edges would not break the connectivity. It has been used a lot to improve the running time of many graph algorithms. In this talk, there is a gentle introduction to expanders and their applications to static, dynamic, iterative, and distributed algorithms on graphs.

Graph Neural Networks for Binding Affinity Prediction

In-depth blog post about applications of GNN to drug discovery, and, in particular, to virtual screening for candidate molecules.

Graph workshop at AAAI 2021: videos

Videos for a recent graph workshop at AAAI 2021 are available online. There are several keynotes including William Hamilton and Stephen Bach, as well as 2-min flash paper presentations.

Postdoc position at EPFL

It's very interesting postdoc position at EPFL to work on molecule design. The following text is by Andreas Loukas.

We are hiring a postdoc to work on the interface between AI and computational protein design. The project will be carried out at EPFL in collaboration with Bruno Correia, Michael Bronstein, Pierre Vandergheynst, and the Swiss Data Science Center.

We offer a 2-year position in EPFL, a vibrant university (well.. post covid) located in one of the most beautiful countries. The salary is very competitive.

The researcher will partake in an interdisciplinary effort to design novel proteins using tools from deep learning. The ideal candidate combines i) practical deep learning/GNN know-how ii) experience with generative models and/or reinforcement learning. Knowledge of biology is not required--but a willingness to learn is.

Relevant work: https://tinyurl.com/1stzxmkj

If you are interested, send me by email: a motivation letter explaining how your expertise fits the current position, a CV, the names/addresses of three references, and three selected publications. We will start reviewing applications on the 15th of March.

Andreas Loukas (find email at andreasloukas.blog)