GNN Paper Explained

Looks like a promising YouTube series on graph machine learning, with the first video explaining the GAT paper.

Fresh picks from ArXiv
Today at ArXiv: random fields for graphs, deconvolutional networks, and general routing algorithms 🛣

If I forgot to mention your paper, please shoot me a message and I will update the post.

Conferences
Semi-Supervised Node Classification on Graphs: Markov Random Fields vs. Graph Neural Networks AAAI 2021
Co-GAT: A Co-Interactive Graph Attention Network for Joint Dialog Act Recognition and Sentiment Classification AAAI 2021
Graph and Temporal Convolutional Networks for 3D Multi-person Pose Estimation in Monocular Videos AAAI 2021
Graph-Evolving Meta-Learning for Low-Resource Medical Dialogue Generation AAAI 2021


Applications
On Using Classification Datasets to Evaluate Graph Outlier Detection: Peculiar Observations and New Insights with Leman Akoglu
Motif-Driven Contrastive Learning of Graph Representations
Deep Multi-attribute Graph Representation Learning on Protein Structures
Graph Autoencoders with Deconvolutional Networks
A Generalized A* Algorithm for Finding Globally Optimal Paths in Weighted Colored Graphs

Survey
Knowledge Graphs Evolution and Preservation -- A Technical Report from ISWS 2019

Tsinghua University Releases First AutoML Toolkit for Graph Datasets & Tasks

At least in the research papers, hyperparameter tuning, feature engineering, architecture and model search, stacking and boosting have been largely ignored and I have a good faith that in the coming year there will be more papers that extensively perform all of those to gain additional boost in performance. This AutoGL PyTorch Framework does just this. It's based off PyG, contains a few standard datasets and models, has several HP algorithms, generates graphlet, pagerank, and other features, stacks models' predictions, and more. Looks very promising.

Paper Explained: Principal Neighbourhood Aggregation for Graph Nets

A nice explanation by Andrei Margeloiu about NeurIPS 2020 paper on how to "fix" GNN's expressivity for continuous node features. More video explanations of GML papers!

Graph Machine Learning: Highlights 2020

Here is my short presentation at ODS (video in Russian) about the state of Graph ML in 2020: top-3 applications and perspectives for the next year.

Geometric ML becomes real in fundamental sciences

A new post by Michael Bronstein: top-3 papers in 2020 about applications of graphML to drug development. I agree that this field is getting momentum and more companies, small and big, will look into application of GNNs to molecule predictions. There is even a graph ML researcher position available in the industrial company. Exciting times for those who are interested in graphs, ML, and biology.

Happy New Year 2021!

Thank you all who followed and shared my posts this year! My very first post was a year ago and since then I wrote 380+ more. The community grew to 1700+ subscribers, who motivated me to learn more and share exciting works done in this community. In 2021 I wish you stay connected in this disconnected world! Peace.

Book: Probabilistic Machine Learning: An Introduction

In addition to two books dedicated to graph ML that I described in the past, there is a new draft of ML book that includes a chapter on graph embeddings. This describes graph embeddings as a encoder-decoder problem and dives into unsupervised and supervised ways to define encoder/decoder parts. It covers matrix factorization methods, label propagation, GNNs, and applications of embeddings.

Fresh picks from ArXiv
Today at ArXiv: thesis on graph matching, image search with scene graphs, and decentralized agent's control 👮

If I forgot to mention your paper, please shoot me a message and I will update the post.

Conferences
* Image-to-Image Retrieval by Learning Similarity between Scene Graphs AAAI 21
* A Paragraph-level Multi-task Learning Model for Scientific Fact-Verification Workshop AAAI 21

Applications
* Decentralized Control with Graph Neural Networks with Alejandro Ribeiro
* Graph Networks with Spectral Message Passing with Peter Battaglia


Survey
* Some Algorithms on Exact, Approximate and Error-Tolerant Graph Matching
* Algorithms for Learning Graphs in Financial Markets

TRIPODS Winter School & Workshop on Graph Learning and Deep Learning

A series of tutorials and hands-on sessions, followed by a workshop covering recent results on graph ML by top researchers in this field. Starts today, requires registration (probably directly asking organizers).

SuperGlue: Learning Feature Matching with Graph Neural Network

Another cool application of GNNs, done at Magic Leap, which specializes in 3D computer-generated graphics. They use GNN for graph matching in real-time videos, which is used for tasks such as 3D reconstruction, place recognition, localization, and mapping. The architecture called SuperGlue (presented at CVPR 2020) is an attention based model with Sinkhorn algorithm, similar to other graph matching works, but is that has been successfully integrated into much bigger pipeline that extracts graphs from the images in end-to-end fashion.

What 2021 holds for Graph ML?

Great format of mini interviews with researchers in graph ML about what's important in the field. I participated too, speaking on the great applications of GNNs we had in 2020 and what we may see changing in 2021. It's very interesting to hear what others think is important and while there are some common themes (e.g. drug discovery, graph construction, stronger GNNs), the interviewees share their distinct predictions (e.g new specialized hardware, applications to RL, causal reasoning, decision making).

Cleora: new unsupervised graph embedding model for hypergraphs

A new library Cleora, written in Rust (for efficiency), by Synerise, a startup building AI platform, builds graph embeddings in unsupervised, inductive, and scalable manner. The algorithm itself is very simple, PageRank-like, just iterative multiplication of the adjacency matrix. It claims to be ~5x faster than PyTorch-BigGraph (with better performance) and provides some nice features including real-time updates, determinism of embeddings, independence of each dimension, compositionality of embeddings of the same entity on different datasets. They also claim they use it in production, so worth a try if you have a graph with billions of edges.

Post: Knowledge Graph Insights

A series of posts from 2020 by Giuseppe Futia on construction, performance, and applications of knowledge graphs.

Book: The Atlas for the Aspiring Network Scientist

A new introductory book of network science by Michele Coscia. 760 pages covering Hitting Time Matrix, Kronecker graph model, network measurement error, graph embedding techniques, and more. As the author describes he aims it to be broad, not deep, so there is not much math involved.

Graph Machine Learning research groups: Michele Coscia

I do a series of posts on the groups in graph research, previous post is here. The 21st is Michele Coscia, the author of the atlas of the network science.

Michele Coscia (~1985)
- Affiliation: IT University of Copenhagen
- Education: Ph.D. at University of Pisa in 2012 (advisor: Dino Pedreschi)
- h-index 22
- Awards: KDD Dissertation Award, ERCIM Cor Baayen Award
- Interests: homophily, community detection, network science

Survey: Utilising Graph Machine Learning within Drug Discovery and Development

A new survey with Michael Bronstein and his colleagues on application of GNNs to drug discovery. This is very exciting line of research and I bet there will be much more effort in 2021 not only from the academia but also from the startups and big pharmacies. In this domain graphs appear as a natural structure to model relationships in molecules or more complex bio entities, for examples protein to protein interactions. There are also many valuable tasks such as target identification, molecule property prediction, de-novo drug design and more. Relation Therapeutics, a London-based startup that also participates in writing this survey, even has an opening for Graph ML researcher.