How can we create general-purpose graph foundation models?
(by Dmitry Eremeev)

For a long time, we believed that general-purpose graph foundation models were impossible to create. Indeed, graphs are used to represent data across many different domains, and thus graph machine learning must handle tasks on extremely diverse datasets, such as social, information, transportation, and co-purchasing networks, or models of various physical, biological, or engineering systems. Given the vast differences in structure, features, and labels among these datasets, it seemed unlikely that a single model could achieve robust cross-domain generalization and perform well on all of them.

However, we noticed that tabular machine learning faces a similar challenge of working with diverse datasets containing different features and labels. And yet, this field has recently witnessed the emergence of first successful foundation models such as TabPFNv2, which are based on the prior-data fitted networks (PFNs) paradigm. Thus, we have decided to try to bring their success to the graph domain.

Our first attempt, G2T-FM, was relatively straightforward. We manually injected graph information into node features by computing structural and positional encodings, along with neighborhood-aggregated features. We then applied tabular foundation models (TabPFNv2 and LimiX) to these enriched features. Even this simple approach delivered impressive results. G2T-FM not only strongly outperforms previous graph foundation models on the GraphLand benchmark and classic datasets, but also often outperforms architecturally-improved and carefully tuned GNNs trained from scratch.

Building on this, our next step was to create GraphPFN – the first graph foundation model in the PFN framework. Moving beyond manual feature engineering of the previous approach, we first integrated message passing modules into the LimiX model so that it could learn graph-based dependencies directly, and then continually pretrained it on 4,000,000 synthetic graph datasets sampled from our specially designed attributed graph prior. The obtained model can perform node property prediction on graph datasets in a single forward pass via in-context learning and produces strong results, substantially outperforming both G2T-FM and classic GNNs on several datasets.

There remains much work to be done, including scaling to larger graphs, improving model architectures and designing better graph priors for synthetic dataset generation. However, we are now convinced that building general-purpose graph foundation models is indeed possible, and a prior-data fitted network approach is a promising path towards this goal.

For more details, check out our papers:
Turning Tabular Foundation Models into Graph Foundation Models
GraphPFN: A Prior-Data Fitted Graph Foundation Model

Tired of evaluating your graph ML models on Cora, CiteSeer, and PubMed? We have a better benchmark for you!
(by Oleg Platonov)

Paper: link (NeurIPS 2025 D&B track)
Datasets: Zenodo and PyG (in PyG, all the necessary feature preprocessing can be done automatically)
Code: GitHub

Recently, there has been a lot of criticism of existing popular graph ML benchmark datasets concerning such aspects as lacking practical relevance, low structural diversity that leaves most of the possible graph structure space not represented, low application domain diversity, graph structure not being beneficial for the considered tasks, and potential bugs in the data collection processes. Some of these criticisms previously appeared on this channel.

To provide the community with better benchmarks, we present GraphLand: a collection of 14 graph datasets for node property prediction coming from diverse real-world industrial applications of graph ML. What makes this benchmark stand out?

Diverse application domains: social networks, web graphs, road networks, and more. Importantly, half of the datasets feature node-level regression tasks that are currently underrepresented in graph ML benchmarks, but are often encountered in real-world applications.

Range of sizes: from thousands to millions of nodes, providing opportunities for researchers with different computational resources.

Rich node attributes that contain numerical and categorical features — these are more typical for industrial applications than textual descriptions that are standard for current benchmarks.

Different learning scenarios. For all datasets, we provide two random data splits with low and high label rate. Further, many of our networks are evolving over time, and for them we additionally provide more challenging temporal data splits and an opportunity to evaluate models in the inductive setting where only an early snapshot of the evolving network is available at train time.

We evaluated a range of models on our datasets and found that, while GNNs achieve strong performance on industrial datasets, they can sometimes be rivaled by popular in the industry gradient boosted decision trees which are provided with additional graph-based input features.

Further, we evaluated several graph foundation models (GFMs). Despite much attention being paid to GFMs recently, we found that there are currently only a few GFMs that can handle arbitrary node features (which is required for true generalization between different graphs) and that these GFMs produce very weak results on our benchmark. So it seemed like the problem of developing general-purpose graph foundation models was far from being solved, which motivated our research in this direction (see the previous post).