GNN User Group: meeting 5

Fifth meeting of GNN user group will include talks from:

* 4:00 - 4:25 (PST): Graphite: GRAPH-Induced feaTure Extraction for Point Cloud Registration (Mahdi Saleh, TUM).
* 4:25 - 4:50 (PST): Optimizing Graph Transformer Networks with Graph-based Techniques (Loc Hoang, University of Texas at Austin)
* 4:50 - 5:15 (PST): Encoding the Core Business Entities Using Meituan Brain (Mengdi Zhang, Meituan)
* 5:15 - 5:30 (PST): Open Discussion and Networking

Please join us today, 27 May! Zoom link in the description.

Reinforcement learning for combinatorial optimization: A survey

Our work that surveys recent RL methods for solving combinatorial optimization problems is accepted at Computers & Operations Research journal.

This is very active field right now and it shows a lot of promise. Traditionally, NP-hard problems such as Traveling Salesman Problem were solved by algorithms, that were designed specifically for each problem. With RL, it's possible to extend the toolbox by learning a function on available data. I really hope that in 10 years from now using ML approaches for combinatorial problems will be a commonplace.

Fresh picks from ArXiv
This week on ArXiv: equivariant GNNs to new groups, new metrics for graph similarity, and parsing emotions with GNNs 😢‍

If I forgot to mention your paper, please shoot me a message and I will update the post.

GNNs
* How Attentive are Graph Attention Networks?
* Symmetry-driven graph neural networks
* Graph Similarity Description: How Are These Graphs Similar? KDD 2021
* SLGCN: Structure Learning Graph Convolutional Networks for Graphs under Heterophily
* Linguistic Structures as Weak Supervision for Visual Scene Graph Generation CVPR 2021
* Directed Acyclic Graph Network for Conversational Emotion Recognition ACL 2021
* On the Universality of Graph Neural Networks on Large Random Graphs
* Differentially Private Densest Subgraph Detection ICML 2021

Survey
* Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
* A Comprehensive Survey on Community Detection with Deep Learning
* A Survey of the Bridge Between Combinatorics and Probability

Graph papers at ICML 2021

ICML 2021 papers are announced, here is some analysis on this.

There are about 58 graph papers (if I didn't mention your paper, let me know, I'll fix it).

The top authors are displayed.

Almost Free Inductive Embeddings Out-Perform Trained Graph Neural Networks in Graph Classification in a Range of Benchmarks

A nice blog post by Vadym Safronov (in Russian also here) which shows that you can use not-trained GCN to match or exceed performance of end-to-end trained GCN on graph classification benchmarks.

Graph Machine Learning research groups: Gal Chechik

I do a series of posts on the groups in graph research, previous post is here. The 29th is Gal Chechik, a professor at the Gonda Brain research institute and a director of AI at NVIDIA in Israel.

Gal Chechik (~1976)
- Affiliation: Bar Ilan University, Israel; NVIDIA
- Education: Ph.D. at Hebrew University, Israel in 2004 (advisors: Naftali Tishby and Israel Nelken)
- h-index 37
- Interests: biological systems, theory of GNNs, equivariant functions.
- Awards: best papers at ICML, ISMB; fullbright fellowship, Alon fellowship

Pytorch Geometric tutorial: Special Guest: Matthias Fey

A recent talk by Matthias Fey, a founder of pytorch geometric library, about the news and future directions of the library. Large-scale graphs, sparse tensors, pytorch lightning, torchscript, and more.

Fresh picks from ArXiv
This week on ArXiv: self-supervised approach without negatives, review of generative models, and semantic search at AliBaba 👞

If I forgot to mention your paper, please shoot me a message and I will update the post.

GNNs
* Neural message passing for joint paratope-epitope prediction with Petar Veličković
* Graph Infomax Adversarial Learning for Treatment Effect Estimation with Networked Observational Data KDD 21
* GraphMI: Extracting Private Graph Data from Graph Neural Networks IJCAI 21
* Graph Barlow Twins: A self-supervised representation learning framework for graphs
* Motif Prediction with Graph Neural Networks
* SpreadGNN: Serverless Multi-task Federated Learning for Graph Neural Networks

Algorithms
* AliCG: Fine-grained and Evolvable Conceptual Graph Construction for Semantic Search at Alibaba KDD 2021
* Stochastic Iterative Graph Matching ICML 2021
* Convergent Graph Solvers

Survey
* Evaluation Metrics for Graph Generative Models: Problems, Pitfalls, and Practical Solutions with Karsten Borgwardt
* Laplacian-Based Dimensionality Reduction Including Spectral Clustering, Laplacian Eigenmap, Locality Preserving Projection, Graph Embedding, and Diffusion Map: Tutorial and Survey
* Graph-based Deep Learning for Communication Networks: A Survey

Graph Neural Networking Challenge 2021

An interesting competition, organized by Technical University of Catalonia (UPC) and ITU, about building GNNs to predict source-destination routing time. The goal is to test generalization abilities of GNNs: training on small graphs and testing on much larger graphs.

Udemy Graph Neural Network course

Online course at Udemy that covers the basics of representation learning on graphs (e.g. DeepWalk, node2vec) and popular GNN architectures, plus some PyG implementations.

Deep Learning on Graphs for Natural Language Processing

Interesting tutorial at NAACL 2021 about applications of graph models to NLP tasks such as text classification, semantic parsing, machine translation, and more. It's based on Graph4NLP library and the slides are available here.

Graphs at ICLR 2021

Very good digest of a few graph papers at ICLR 2021. Talks about new GNNS to solve overmoothing, over-squashing, heterophily, and attention problems.

PyTorch-Geometric Tutorial Talk

Today, I will speak about our ICLR work "Boost then Convolve: Gradient Boosting Meets Graph Neural Networks". If you want to learn more about how GBDT and GNN work, and how they can be applied successfully for node prediction tasks, please join here at 15 (Paris time).

GML Express: keynotes at ICLR, topics at ICML 2021, and new GNN tutorials.

The most interesting events in graph ML during the last 2 months are in my new issue of graph ML newsletter.

Fresh picks from ArXiv
This week on ArXiv: analysis of transformers, resolving scalability, and new attacks ⚔️

If I forgot to mention your paper, please shoot me a message and I will update the post.

Embeddings
* Self-supervised Graph-level Representation Learning with Local and Global Structure with Jian Tang
* Do Transformers Really Perform Bad for Graph Representation?
* Order Matters: Probabilistic Modeling of Node Sequence for Graph Generation ICML 2021
* Symmetric Spaces for Graph Embeddings: A Finsler-Riemannian Approach ICML 2021

GNNs
* TDGIA:Effective Injection Attacks on Graph Neural Networks KDD 2021
* Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Prediction with Jian Tang
* Is Homophily a Necessity for Graph Neural Networks?
* Learning to Pool in Graph Neural Networks for Extrapolation
* GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddings with Jure Leskovec
* Scaling Up Graph Neural Networks Via Graph Coarsening
* Rethinking Graph Transformers with Spectral Attention with William L. Hamilton
* Breaking the Limit of Graph Neural Networks by Improving the Assortativity of Graphs with Local Mixing Patterns
* Breaking the Limits of Message Passing Graph Neural Networks

Survey
* Survey of Image Based Graph Neural Networks
* Graph Neural Networks for Natural Language Processing: A Survey

Deep Learning DIY course

A very good deep learning course by Marc Lelarge that among other things cover graph ML: graph embeddings, signal processing, and GNNs. It comes with videos, slides, notebooks, and assignments.

Dynamic GNNs videos

A new YouTube channel that discusses spatio-temporal and dynamic GNNs in an easy and fun manner.

Results of OGB large-scale challenge

OGB team announced the results of KDD 2021 cup challenge where teams competed in node classification, triplet prediction, and graph regression tasks. Short summaries are provided for the winning solutions and it's quite interesting to see the diversity of the proposed methods: some used ensembles of GNNs, some pretrained graph embeddings, some label propagation, among others. Notably, Baidu and DeepMind scored really well on these tasks. Congrats to the winners!

Graphormer - Do Transformers Really Perform Bad for Graph Representation? | Paper Explained

A nice explanation by Aleksa Gordić of the recent paper that shows how enriching node features with some structural information from the graph can help Transformer model to achieve SOTA results on OGB datasets.

Graph Neural Networks User Group: June

June's meeting of GNN user group will include the following talks:

* 4:00 - 4:30 (PST): Binary Graph Neural Networks and Dynamic Graph Models (Mahdi Saleh, Imperial College London).
* 4:30 - 5:00 (PST): Simplifying large-scale visual analysis of tricky data & models with GPUs, graphs, and ML (Leo Meyerovich, Graphistry Inc)
* 5:00 - 5:30 (PST): Open Discussion and Networking

Join this Thursday!