Graph Machine Learning
6.71K subscribers
53 photos
11 files
808 links
Everything about graph theory, computer science, machine learning, etc.


If you have something worth sharing with the community, reach out @gimmeblues, @chaitjo.

Admins: Sergey Ivanov; Michael Galkin; Chaitanya K. Joshi
Download Telegram
KDD 2020: Workshop on Deep Learning on Graphs

If you miss ICML deadlines, there is another good workshop for GML at KDD.
Deadline: 15 June
5 pages, double-blind
Graph Representation Learning for Algorithmic Reasoning

Another idea coming more frequently in recent graph papers is to learn particular graph algorithm such as Bellman-Ford or Breadth-First Search, instead of doing node classification or link prediction. Here is a video from WebConf'20 by Petar VeličkoviΔ‡ (DeepMind) motivating this approach.
Graph Machine Learning research groups: Michael Bronstein

I do a series of posts on the groups in graph research. The fifth is Michael Bronstein. He founded a company Fabula AI on detecting fake news in social networks, which was acquired by Twitter. Also, he was a committee member of my PhD defense πŸ™‚

Michael Bronstein (1980)
- Affiliation: Imperial College London; Twitter
- Education: Ph.D. at Israel Institute of Technology in Israel in 2007 (supervised by Ron Kimmel);
- h-index: 61;
- Awards: IEEE and IARP fellow, Dalle Molle prize, Royal Society Wolfson Merit award;
- Interests: computer graphics, geometrical deep learning, graph neural networks.
Max Welling Talk GNN

I recently thought about what are other types of GNN exist beyond message-passing. I think one of them can be equivariant networks, i.e. neural networks that have permutation-equivariant properties, but I think there are other possible powerful graph models that are yet to be discovered.

In this video, Max Welling discusses his recent works on equivariant NNs for meshes and factor GNNs.
Introduction to Deep Learning (I2DL)

There is a course on deep learning by Technical University of Munich. Recordings, slides, and exercises are available online.
Secrets of the Surface: The Mathematical Vision of Maryam Mirzakhani

There is a documentary that you can watch on the life of Maryam Mirzakhani. In 2014, she was awarded the Fields medal for the work in "the dynamics and geometry of Riemann surfaces and their moduli spaces." You can read about her in this article. For the film, you can register here and they will send a link to the Vimeo, which will be available until 19th May.
AI and Theorem Proving

One of the topics that caught my attention was on using AI to automate theorem proving. Apparently, there is already a conference on this. At ICLR there was a paper on using graph networks for theorem proving.

I think besides this conference, which mainly explores how you can model mathematical logic using embeddings, another type of theorem proving is on smart pruning of combinatorial spaces (e.g. you have large space of graphs, from which you need to pick some particular examples).
Learning graph structure to help classification

I just recently discussed an idea whether it's possible to create a graph from a non-graph classification data set and improve classification performance by doing it and I found two works on it.

First approach just tries different values for knn to connect the points into a graph, obtains a graph for each parameter setting, and verifies the performance of classification of a graph model on the obtained graph. Clearly the problem with it is that you have to do classification many many times for different parameters of your knn.

Second approach (ICML 2019, link to presentation) is more data-driven: instead of freezing the parameters of knn, it uses a graph generative model that would generate a graph from the points. Then a graph neural network would make a classification prediction and, together with parameters of generative model, would be updated by backpropagation. It's still quite heavy as you need to update parameters of two different models instead of one. Perhaps, there are future works that would create the graphs at little computational cost and would boost the results for classification pipelines.
ACL 2020 stats

ACL (Association for Computational Linguistics) is the top conferences in NLP. Each year there are more and more papers that use graphs with natural language.

Dates: July 5-10
Where: Online

β€’ 3088 submissions (2906 submissions in 2019)
β€’ 571/208 long and short papers (25% overall acceptance rate; 447/213 in 2019)
β€’ 42/6 long/short graph papers (7%/3% of total)
CVPR 2020

Computer Vision and Pattern Recognition (CVPR) conference is the top conference in CV and has had increasing focus on application of graphs to images. Here are some facts about CVPR 2020.

Papers link
Dates: June 14-19
Where: Virtual

β€’ 6,656 total papers
β€’ 1,470 accepted papers
β€’ 22% acceptance rate
β€’ ~69 graph papers (~5% of total)
Graph Machine Learning research groups: Christos Faloutsos

I do a series of posts on the groups in graph research. The sixth is Christos Faloutsos. He was an advisor for many current professors in GML such as Jure Leskovec, Leman Akoglu, Stephan Guennemman, and Bruno Ribeiro.


Christos Faloutsos (~1960)
- Affiliation: Carnegie Mellon University; Amazon
- Education: Ph.D. at University of Toronto in 1987 (supervised by Stavros Christodoulakis);
- h-index: 131;
- Awards: ACM fellow, best paper awards at KDD, SIGMOD, ICDM;
- Interests: data mining; database; anomaly detection in graphs.
Social network data set

Anton @xgfsru shared a data set VK1M (password 1234), with first 1M users from social network vk.com (data is taken via public API). In addition to the friends of each user, the file contains meta-information such as education, country, birthday of each node. It can be useful for node classification or regression tasks as well as community or anomaly detection.
How hard is graph isomorphism for graph neural networks?

This is a new work by Andreas Loukas that sheds a little bit more light into the theory behind GNN. The analysis relies on the amount of information nodes should exchange in order to detect isomorphism class of each graph. This problem of finding isomorphism class is called graph canonization problem, which is probably even harder than graph isomorphism. As such a GNN model needs to output a number for each possible graph up to isomorphism, and needless to say the number of non-isomorphic graphs grows closely to factorial terms. Hence the experiments, while support the theory, are done only on very small graphs ~10 nodes.
On the universal equivariant functions

Yesterday's post was followed by a conversation with Andreas Loukas on the power of graph neural networks. There is one detail about the current analysis of GNN, which I didn't pay much attention before, even though I encountered it (it's a recurring phenomenon for me, when some major insight is given one sentence in the paper and is not highlighted in CAPITAL letters).

The insight is that there are two types of GNN, anonymous and non-anonymous. Anonymous case means you are invariant to the order of nodes, for example when you use the sum aggregation over nodes. It was shown that anonymous case is equivalent to WL algorithm and therefore has a lot limitations such as not being able to count subgraphs or distinguish graphons, etc. So the current anonymous models are not universal: they cannot compute all the functions of the inputs. It's a weaker model than non-anonymous case, when you give some orderings to the nodes and then you iterate over all orderings.

Non-anonymous models have additional node features, for example one-hot encodings of their position in adjacency matrix, which is one of the sufficient conditions for GNN to be universal. There are then two scenarios. Either you consider all possible permutations, in which case it grows in factorial terms and essentially it's a cheat. Or you resort to a single permutation, but then do not enjoy the invariance property of GNN, i.e. for different orderings it can give different set of embeddings for the same nodes.

So it's interesting to see if there are universal equivariant functions that do not use all node permutations, which is still an open question.
Why `True is False is False` -> False?

Stumbled upon this little question why True is False is False evaluates to False in python. The answer is as simple as the question, but not obvious and many people could not answer it. So I wrote a quick post about it.