Awesome Collection of Graph Neural Network Theory Papers

Theoretical Foundations of Graph Neural Networks, Jiaqing Xie (ETH), Yuxin Wang (Fudan), Ziheng Chi (ETH)

This collection of Graph Neural Network (GNN) works distinguishes itself from other existing collections of GNN survey papers. It assists in crafting theoretical references related to GNNs and, more crucially, offers a more expansive viewpoint on the evolution of GNN theories, such as expressive power and SE(3) equivariant power, in the past few years.

In the second part, using domain graph neural networks to explain important natural science theories such as PDE, biochemistry are included. Ultimately we won't consider any social science relateds networks.

Some General Theoretical GNN Surveys

1. Zhou, Yu, et al. Graph neural networks: Taxonomy, advances, and trends. ACM Transactions on Intelligent Systems and Technology (TIST) 13.1 (2022): 1-54.

2. Liao, Renjie. Deep Learning on Graphs: Theory, Models, Algorithms and Applications. University of Toronto (Canada), 2021.

3. Ritzert, Martin. Learning on graphs with logic and neural networks. Diss. Dissertation, RWTH Aachen University, 2021, 2021.

4. Buffelli, Davide. Improving the Effectiveness of Graph Neural Networks in Practical Scenarios. (2023).

5. Yuan, Hao, et al. Explainability in graph neural networks: A taxonomic survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (2022).

6. Khoshraftar, Shima, and Aijun An. A survey on graph representation learning methods. arXiv preprint arXiv:2204.01855 (2022).

Expressive Power of Graph Neural Networks (WL-test, Equivariance, Invariance)

1. Souza, Amauri, et al. Provably expressive temporal graph networks. Advances in Neural Information Processing Systems 35 (2022): 32257-32269.

2. Balcilar, Muhammet, et al. Analyzing the expressive power of graph neural networks in a spectral perspective. Proceedings of the International Conference on Learning Representations (ICLR). 2021.

3. Sato, Ryoma. A survey on the expressive power of graph neural networks. arXiv preprint arXiv:2003.04078 (2020).

4. Rampášek, Ladislav, et al. Recipe for a general, powerful, scalable graph transformer. Advances in Neural Information Processing Systems 35 (2022): 14501-14515.

5. Bodnar, Cristian, et al. Weisfeiler and lehman go cellular: Cw networks. Advances in Neural Information Processing Systems 34 (2021): 2625-2640.

6. Bouritsas, Giorgos, et al. Improving graph neural network expressivity via subgraph isomorphism counting. IEEE Transactions on Pattern Analysis and Machine Intelligence 45.1 (2022): 657-668.

7. Frasca, Fabrizio, et al. Understanding and extending subgraph gnns by rethinking their symmetries. Advances in Neural Information Processing Systems 35 (2022): 31376-31390.

8. Vignac, Clement, Andreas Loukas, and Pascal Frossard. Building powerful and equivariant graph neural networks with structural message-passing. Advances in neural information processing systems 33 (2020): 14143-14155.

9. Ma, Jiaqi, Junwei Deng, and Qiaozhu Mei. Subgroup generalization and fairness of graph neural networks. Advances in Neural Information Processing Systems 34 (2021): 1048-1061.

10. Fatemi, Bahare, Layla El Asri, and Seyed Mehran Kazemi. SLAPS: Self-supervision improves structure learning for graph neural networks. Advances in Neural Information Processing Systems 34 (2021): 22667-22681.

11. Wijesinghe, Asiri, and Qing Wang. A new perspective on" how graph neural networks go beyond weisfeiler-lehman?". International Conference on Learning Representations. 2022.

12. Papp, Pál András, and Roger Wattenhofer. A theoretical comparison of graph neural network extensions. International Conference on Machine Learning. PMLR, 2022.

13. Suresh, Susheel, et al. Breaking the limit of graph neural networks by improving the assortativity of graphs with local mixing patterns. arXiv preprint arXiv:2106.06586 (2021).

14. Zhao, Lingxiao, Neil Shah, and Leman Akoglu. A practical, progressively-expressive GNN. Advances in Neural Information Processing Systems 35 (2022): 34106-34120.

15. Zhang, Bohang, et al. Rethinking the expressive power of gnns via graph biconnectivity. arXiv preprint arXiv:2301.09505 (2023).

16. Chen, Lei, Zhengdao Chen, and Joan Bruna. On graph neural networks versus graph-augmented mlps. arXiv preprint arXiv:2010.15116 (2020).

17. Long, Qingqing, et al. Theoretically improving graph neural networks via anonymous walk graph kernels. Proceedings of the Web Conference 2021. 2021.

18. Dash, Tirtharaj, Ashwin Srinivasan, and Lovekesh Vig. Incorporating symbolic domain knowledge into graph neural networks. Machine Learning 110.7 (2021): 1609-1636.

19. Cahill, Jameson, et al. Group-invariant max filtering. arXiv preprint arXiv:2205.14039 (2022).

20. Maehara, Takanori, and Hoang NT. Learning on random balls is sufficient for estimating (some) graph parameters. Advances in Neural Information Processing Systems 34 (2021): 1126-1141.

21. Wang, Hanchen, et al. Neural Subgraph Counting with Wasserstein Estimator.Proceedings of the 2022 International Conference on Management of Data. 2022.

22. Zopf, Markus. 1-wl expressiveness is (almost) all you need. 2022 International Joint Conference on Neural Networks (IJCNN). IEEE, 2022.

23. Yang, Mingqi, et al. Breaking the expression bottleneck of graph neural networks. IEEE Transactions on Knowledge and Data Engineering (2022).

24. Bianchi, Filippo Maria, and Veronica Lachi. The expressive power of pooling in Graph Neural Networks. arXiv preprint arXiv:2304.01575 (2023).

25. Feldman, Or, et al. Weisfeiler and leman go infinite: Spectral and combinatorial pre-colorings. arXiv preprint arXiv:2201.13410 (2022).

26. Liu, Linfeng, et al. Towards Accurate Subgraph Similarity Computation via Neural Graph Pruning. arXiv preprint arXiv:2210.10643 (2022).

27. Xia, Wenwen, Yuchen Li, and Shenghong Li. On the Substructure Countability of Graph Neural Networks. IEEE Transactions on Knowledge and Data Engineering (2022).

28. Beddar-Wiesing, Silvia, et al. Weisfeiler--Lehman goes Dynamic: An Analysis of the Expressive Power of Graph Neural Networks for Attributed and Dynamic Graphs. arXiv preprint arXiv:2210.03990 (2022).

29. Wang, Zhaohui, et al. Twin Weisfeiler-Lehman: High Expressive GNNs for Graph Classification. arXiv preprint arXiv:2203.11683 (2022).

30. Fang, Zhongxi, et al. Wasserstein Graph Distance based on $ L_1 $-Approximated Tree Edit Distance between Weisfeiler-Lehman Subtrees. arXiv preprint arXiv:2207.04216 (2022).

31. Imaduwage, Sarith, P. P. N. V. Kumara, and W. J. Samaraweera. The Mismatch between Graph Neural Networks’ Expressivity and Propagation Graphs in Fake News Detection. 2022 IEEE/ACIS 7th International Conference on Big Data, Cloud Computing, and Data Science (BCD). IEEE, 2022.

32. Murphy, Ryan L. A Novel Framework for Invariant Neural Networks Applied to Graph and Set Data. Diss. Purdue University Graduate School, 2021.

33. Tahmasebi, Behrooz, and Stefanie Jegelka. On the Effect of Input Perturbations for Graph Neural Networks.

34. Tingyang, Yu. Subgraph Sampling Strategy for Equivariant Subgraph Aggregation Network Based on Weisfeiler-Lehman Similarity.

35. Azizian, Waiss, and Marc Lelarge. Expressive power of invariant and equivariant graph neural networks. arXiv preprint arXiv:2006.15646 (2020).

36. Joshi, Chaitanya K., et al. On the expressive power of geometric graph neural networks. arXiv preprint arXiv:2301.09308 (2023).

37. Balcilar, Muhammet, et al. Analyzing the expressive power of graph neural networks in a spectral perspective. Proceedings of the International Conference on Learning Representations (ICLR). 2021.

38. Xu, Keyulu, et al. How powerful are graph neural networks?. arXiv preprint arXiv:1810.00826 (2018).

39. Li, Pan, and Jure Leskovec. The expressive power of graph neural networks. Graph Neural Networks: Foundations, Frontiers, and Applications (2022): 63-98.

40. Wang, Xiyuan, and Muhan Zhang. How powerful are spectral graph neural networks. International Conference on Machine Learning. PMLR, 2022.

41. You, Jiaxuan, et al. Identity-aware graph neural networks. Proceedings of the AAAI conference on artificial intelligence. Vol. 35. No. 12. 2021.

42. Feng, Jiarui, et al. How powerful are k-hop message passing graph neural networks. arXiv preprint arXiv:2205.13328 (2022).

43. Balcilar, Muhammet, et al. Breaking the limits of message passing graph neural networks. International Conference on Machine Learning. PMLR, 2021.

44. Bouritsas, Giorgos, et al. Improving graph neural network expressivity via subgraph isomorphism counting. IEEE Transactions on Pattern Analysis and Machine Intelligence 45.1 (2022): 657-668.

45. Papp, Pál András, et al. DropGNN: Random dropouts increase the expressiveness of graph neural networks. Advances in Neural Information Processing Systems 34 (2021): 21997-22009.

46. Liu, Songtao, et al. Local augmentation for graph neural networks. International Conference on Machine Learning. PMLR, 2022.

47. Kanatsoulis, Charilaos I., and Alejandro Ribeiro. Graph neural networks are more powerful than we think. arXiv preprint arXiv:2205.09801 (2022).

48. Xu, Keyulu, et al. Optimization of graph neural networks: Implicit acceleration by skip connections and more depth. International Conference on Machine Learning. PMLR, 2021.

49. Du, Weitao, et al. A new perspective on building efficient and expressive 3D equivariant graph neural networks. arXiv preprint arXiv:2304.04757 (2023).

50. Barceló, Pablo, et al. The logical expressiveness of graph neural networks. 8th International Conference on Learning Representations (ICLR 2020). 2020.

51. Guo, Alan JX, Qing-Hu Hou, and Ou Wu. Improving the Expressive Power of Graph Neural Network with Tinhofer Algorithm. arXiv preprint arXiv:2104.01848 (2021).

52. Geerts, Floris. The expressive power of kth-order invariant graph networks. arXiv preprint arXiv:2007.12035 (2020).

53. Bo, Deyu, et al. A Survey on Spectral Graph Neural Networks. arXiv preprint arXiv:2302.05631 (2023).

54. Michel, Gaspard, et al. Path Neural Networks: Expressive and Accurate Graph Neural Networks. arXiv preprint arXiv:2306.05955 (2023).

55. Aamand, Anders, et al. Exponentially improving the complexity of simulating the Weisfeiler-Lehman test with graph neural networks. Advances in Neural Information Processing Systems 35 (2022): 27333-27346.

56. Geerts, Floris, and Juan L. Reutter. Expressiveness and approximation properties of graph neural networks. arXiv preprint arXiv:2204.04661 (2022).

57. Maron, Haggai, et al. Provably powerful graph networks. Advances in neural information processing systems 32 (2019).

58. Zhang, Muhan, and Pan Li. Nested graph neural networks. Advances in Neural Information Processing Systems 34 (2021): 15734-15747.

59. Morris, Christopher, et al. Speqnets: Sparsity-aware permutation-equivariant graph networks. International Conference on Machine Learning. PMLR, 2022.

60. Villar, Soledad, et al. Scalars are universal: Equivariant machine learning, structured like classical physics. Advances in Neural Information Processing Systems 34 (2021): 28848-28863.

61. Cotta, Leonardo, Christopher Morris, and Bruno Ribeiro. Reconstruction for powerful graph representations. Advances in Neural Information Processing Systems 34 (2021): 1713-1726.

62. Han, Jiaqi, et al. Geometrically equivariant graph neural networks: A survey. arXiv preprint arXiv:2202.07230 (2022).

63. Morris, Christopher, et al. Weisfeiler and leman go machine learning: The story so far. arXiv preprint arXiv:2112.09992 (2021).

64. Gao, Jianfei, and Bruno Ribeiro. On the equivalence between temporal and static equivariant graph representations. International Conference on Machine Learning. PMLR, 2022.

65. Huang, Zhongyu, et al. Going Deeper into Permutation-Sensitive Graph Neural Networks. International Conference on Machine Learning. PMLR, 2022.

66. Cai, Chen, and Yusu Wang. Convergence of invariant graph networks. International Conference on Machine Learning. PMLR, 2022.

67. Chen, Samantha, et al. Weisfeiler-Lehman Meets Gromov-Wasserstein. International Conference on Machine Learning. PMLR, 2022.

68. Puny, Omri, et al. Beddar-Wiesing, Silvia, et al. "Weisfeiler--Lehman goes Dynamic: An Analysis of the Expressive Power of Graph Neural Networks for Attributed and Dynamic Graphs." arXiv preprint arXiv:2210.03990 (2022). for graph neural networks. arXiv preprint arXiv:2302.11556 (2023).

69. Huang, Yinan, et al. Boosting the Cycle Counting Power of Graph Neural Networks with I $^ 2$-GNNs. arXiv preprint arXiv:2210.13978 (2022).

70. Feng, Jiarui, et al. Towards Arbitrarily Expressive GNNs in $ O (n^ 2) $ Space by Rethinking Folklore Weisfeiler-Lehman. arXiv preprint arXiv:2306.03266 (2023).

71. Böker, Jan, et al. Fine-grained Expressivity of Graph Neural Networks. arXiv preprint arXiv:2306.03698 (2023).

72. Luo, Zhezheng, et al. On the Expressiveness and Generalization of Hypergraph Neural Networks. arXiv preprint arXiv:2303.05490 (2023).

73. Zhang, Bohang, et al. A Complete Expressiveness Hierarchy for Subgraph GNNs via Subgraph Weisfeiler-Lehman Tests. arXiv preprint arXiv:2302.07090 (2023).

74. Zhu, Wenhao, et al. On Structural Expressive Power of Graph Transformers." arXiv preprint arXiv:2305.13987 (2023).

75. Liu, Chang, et al. EDEN: A Plug-in Equivariant Distance Encoding to Beyond the 1-WL Test. arXiv preprint arXiv:2211.10739 (2022).

76. Le, Thien, and Stefanie Jegelka. Limits, approximation and size transferability for GNNs on sparse graphs via graphops. arXiv preprint arXiv:2306.04495 (2023).

77. Mitton, Joshua, and Roderick Murray-Smith. Subgraph Permutation Equivariant Networks. arXiv preprint arXiv:2111.11840 (2021).

78. Liu, Zhiyuan, et al. Towards Equivariant Graph Contrastive Learning via Cross-Graph Augmentation. (2023).

79. Dasoulas, Georgios. Towards Expressive Graph Neural Networks: Theory, Algorithms, and Applications. Diss. Institut Polytechnique de Paris, 2022.

80. Geerts, Floris, and Juan L. Reutter. Expressiveness and approximation properties of graph neural networks. arXiv preprint arXiv:2204.04661 (2022).

81. Lim, Derek, et al. Sign and basis invariant networks for spectral graph representation learning. arXiv preprint arXiv:2202.13013 (2022).

82. Morris, Christopher, et al. Speqnets: Sparsity-aware permutation-equivariant graph networks. International Conference on Machine Learning. PMLR, 2022.

83. Kong, Lecheng, Yixin Chen, and Muhan Zhang. Geodesic Graph Neural Network for Efficient Graph Representation Learning. arXiv preprint arXiv:2210.02636 (2022).

Geometric Graph Neural Networks, Topology and Lie Group

Many of the papers that bridges topology and equivariant graphNN that mentioned above.

1. Sahbi, Hichem. Topologically-Consistent Magnitude Pruning for Very Lightweight Graph Convolutional Networks. 2022 IEEE International Conference on Image Processing (ICIP). IEEE, 2022.

2. Paolino, Raffaele, et al. Unveiling the Sampling Density in Non-Uniform Geometric Graphs. arXiv preprint arXiv:2210.08219 (2022).

3. Sahbi, Hichem. Lightweight Graph Convolutional Networks with Topologically Consistent Magnitude Pruning. arXiv preprint arXiv:2203.13616 (2022).

4. Feng, Fan, et al. Community Channel-Net: Efficient channel-wise interactions via community graph topology. Pattern Recognition 141 (2023): 109536.

5. Dehmamy, Nima, Albert-László Barabási, and Rose Yu. Understanding the representation power of graph neural networks in learning graph topology. Advances in Neural Information Processing Systems 32 (2019).

6. Gagrani, Mukul, et al. Neural Topological Ordering for Computation Graphs. Advances in Neural Information Processing Systems 35 (2022): 17327-17339.

7. Dehmamy, Nima, Albert-László Barabási, and Rose Yu. Understanding the representation power of graph neural networks in learning graph topology. Advances in Neural Information Processing Systems 32 (2019).

8. Ye, Xue, Fang Sun, and Shiming Xiang. TREPH: A Plug-In Topological Layer for Graph Neural Networks. Entropy 25.2 (2023): 331.

9. Zheng, Shuai, et al. Node-oriented Spectral Filtering for Graph Neural Networks. arXiv preprint arXiv:2212.03654 (2022).

10. Horn, Max, et al. Topological graph neural networks. arXiv preprint arXiv:2102.07835 (2021).

11. Wei, Lanning, Huan Zhao, and Zhiqiang He. Designing the topology of graph neural networks: A novel feature fusion perspective. Proceedings of the ACM Web Conference 2022. 2022.

12. Liu, Shaohui, Chengyang Wu, and Hao Zhu. Topology-aware graph neural networks for learning feasible and adaptive AC-OPF solutions. IEEE Transactions on Power Systems (2022).

13. Zhang, Jiying, et al. Fine-tuning graph neural networks via graph topology induced optimal transport. arXiv preprint arXiv:2203.10453 (2022).

14. Southern, Joshua, et al. Curvature filtrations for graph generative model evaluation. arXiv preprint arXiv:2301.12906 (2023).

15. Zhang, Simon, Soham Mukherjee, and Tamal K. Dey. GEFL: Extended Filtration Learning for Graph Classification. Learning on Graphs Conference. PMLR, 2022.

16. Xu, Jiaxing, et al. Union Subgraph Neural Networks. arXiv preprint arXiv:2305.15747 (2023).

17. Chen, Yuzhou, Elena Sizikova, and Yulia R. Gel. TopoAttn-Nets: Topological Attention in Graph Representation Learning. Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Cham: Springer International Publishing, 2022.

18. Gao, He, et al. Topological Graph Convolutional Network Based on Complex Network Characteristics. IEEE Access 10 (2022): 64465-64472.

19. Horoi, Stefan, et al. Exploring the geometry and topology of neural network loss landscapes. Advances in Intelligent Data Analysis XX: 20th International Symposium on Intelligent Data Analysis, IDA 2022, Rennes, France, April 20–22, 2022, Proceedings. Cham: Springer International Publishing, 2022.

20. Rieck, Bastian. On the Expressivity of Persistent Homology in Graph Learning. arXiv preprint arXiv:2302.09826 (2023).

21. Aktas, Mehmet Emin, Thu Nguyen, and Esra Akbas. Homology Preserving Graph Compression. 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA). IEEE, 2021.

22. Bamberger, Jacob. A Topological characterisation of Weisfeiler-Leman equivalence classes. Topological, Algebraic and Geometric Learning Workshops 2022. PMLR, 2022.

23. Xin, Cheng, et al. A $2 $-parameter Persistence Layer for Learning. (2023).

24. Segovia-Dominguez, Ignacio, et al. EMP: Effective Multidimensional Persistence for Graph Representation Learning. (2023).

25. Satorras, Vıctor Garcia, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural networks. International conference on machine learning. PMLR, 2021.

*26. Stärk, Hannes, et al. Equibind: Geometric deep learning for drug binding structure prediction. International Conference on Machine Learning. PMLR, 2022.

27. Liu, Shengchao, et al. Pre-training molecular graph representation with 3d geometry. arXiv preprint arXiv:2110.07728 (2021).

28. Villar, Soledad, et al. Scalars are universal: Equivariant machine learning, structured like classical physics. Advances in Neural Information Processing Systems 34 (2021): 28848-28863.

29. Satorras, Victor Garcia, et al. E (n) equivariant normalizing flows. arXiv preprint arXiv:2105.09016 (2021).

30. Ganea, Octavian-Eugen, et al. Independent se (3)-equivariant models for end-to-end rigid protein docking. arXiv preprint arXiv:2111.07786 (2021).

31. Brandstetter, Johannes, et al. Geometric and physical quantities improve e (3) equivariant message passing. arXiv preprint arXiv:2110.02905 (2021).

32. Gasteiger, Johannes, Chandan Yeshwanth, and Stephan Günnemann. Directional message passing on molecular graphs via synthetic coordinates. Advances in Neural Information Processing Systems 34 (2021): 15421-15433.

33. Shuaibi, Muhammed, et al. Rotation invariant graph neural networks using spin convolutions. arXiv preprint arXiv:2106.09575 (2021).

34. Bogatskiy, Alexander, et al. Symmetry group equivariant architectures for physics. arXiv preprint arXiv:2203.06153 (2022).

Graph Neural Networks & Algorithms on solving NP-hard problems

From my observation, SAT is the most researhed topicm then TSP, CNF, DNF, and coloring. May be overlapped with some of the spectral methods with aim of graph approximations.

1. Prates, Marcelo, et al. Learning to solve np-complete problems: A graph neural network for decision tsp. Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 33. No. 01. 2019.

2. Cappart, Quentin, et al. Combinatorial optimization and reasoning with graph neural networks. arXiv preprint arXiv:2102.09544 (2021).

3. Xing, Zhihao, and Shikui Tu. A graph neural network assisted monte carlo tree search approach to traveling salesman problem. IEEE Access 8 (2020): 108418-108428.

4. Hu, Yujiao, et al. A bidirectional graph neural network for traveling salesman problems on arbitrary symmetric graphs. Engineering Applications of Artificial Intelligence 97 (2021): 104061.

5. Abboud, Ralph, Ismail Ceylan, and Thomas Lukasiewicz. Learning to reason: Leveraging neural networks for approximate DNF counting. Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 34. No. 04. 2020.

6. Yow, Kai Siong, and Siqiang Luo. Learning-based approaches for graph problems: a survey. arXiv preprint arXiv:2204.01057 (2022).

7. Liu, Minghao, et al. Can Graph Neural Networks Learn to Solve MaxSAT Problem?. arXiv preprint arXiv:2111.07568 (2021).

8. Selsam, Daniel, et al. Learning a SAT solver from single-bit supervision. arXiv preprint arXiv:1802.03685 (2018).

9. Kothapalli, Abihith, Mudassir Shabbir, and Xenofon Koutsoukos. Learning-Based Heuristic for Combinatorial Optimization of the Minimum Dominating Set Problem using Graph Convolutional Networks. arXiv preprint arXiv:2306.03434 (2023).

10. Liu, Minghao, et al. Learning the satisfiability of pseudo-Boolean problem with graph neural networks. Principles and Practice of Constraint Programming: 26th International Conference, CP 2020, Louvain-la-Neuve, Belgium, September 7–11, 2020, Proceedings 26. Springer International Publishing, 2020.

11. Panagopoulos, George, et al. Learning Graph Representations for Influence Maximization. arXiv preprint arXiv:2108.04623 (2021).

12. Lamb, Luis C., et al. Graph neural networks meet neural-symbolic computing: A survey and perspective. arXiv preprint arXiv:2003.00330 (2020).

13. Joshi, Chaitanya K., et al. Learning TSP requires rethinking generalization. 27th International Conference on Principles and Practice of Constraint Programming (CP 2021). Schloss Dagstuhl-Leibniz-Zentrum für Informatik, 2021.

14. Yolcu, Emre, and Barnabás Póczos. Learning local search heuristics for boolean satisfiability. Advances in Neural Information Processing Systems 32 (2019). (SAT)

15. Joshi, Chaitanya K., et al. Learning the travelling salesperson problem requires rethinking generalization. Constraints 27.1-2 (2022): 70-98.

16. Kurin, Vitaly, et al. Can Q-learning with graph networks learn a generalizable branching heuristic for a SAT solver?. Advances in Neural Information Processing Systems 33 (2020): 9608-9621.

17. Zhang, Wenjie, et al. NLocalSAT: Boosting local search with solution prediction. arXiv preprint arXiv:2001.09398 (2020).

18. Zhang, Lisa, et al. Neural guided constraint logic programming for program synthesis. Advances in Neural Information Processing Systems 31 (2018).

19. Abdelaziz, Ibrahim, et al. Learning to guide a saturation-based theorem prover. IEEE Transactions on Pattern Analysis and Machine Intelligence 45.1 (2022): 738-751.

20. Angelini, Maria Chiara, and Federico Ricci-Tersenghi. Cracking nuts with a sledgehammer: when modern graph neural networks do worse than classical greedy algorithms. arXiv preprint arXiv:2206.13211 (2022).

21. Tönshoff, Jan, et al. One model, any CSP: Graph neural networks as fast global search heuristics for constraint satisfaction. arXiv preprint arXiv:2208.10227 (2022).

22. Yang, Lei, and Lei Zou. Noah: Neural-optimized A Search Algorithm for Graph Edit Distance Computation. 2021 IEEE 37th International Conference on Data Engineering (ICDE). IEEE, 2021.

23. Wang, Wenxi, et al. NeuroComb: Improving SAT Solving with Graph Neural Networks. arXiv preprint arXiv:2110.14053 (2021).

24. Liu, Minghao, et al. Can Graph Neural Networks Learn to Solve MaxSAT Problem?. arXiv preprint arXiv:2111.07568 (2021).

25. Grötschla, Florian, Joël Mathys, and Roger Wattenhofer. Learning Graph Algorithms With Recurrent Graph Neural Networks. arXiv preprint arXiv:2212.04934 (2022).

26. Glorot, Xavier, et al. Learning representations of logical formulae using graph neural networks. Neural Information Processing Systems, Workshop on Graph Representation Learning. 2019.

27. Saveri, Gaia, and Luca Bortolussi. Graph Neural Networks for Propositional Model Counting. arXiv preprint arXiv:2205.04423 (2022).

28. Zhang, Congsong, Yong Gao, and James Nastos. Learning Branching Heuristics from Graph Neural Networks. arXiv preprint arXiv:2211.14405 (2022).

29. Sun, Changzhi, et al. Probabilistic graph reasoning for natural proof generation. arXiv preprint arXiv:2107.02418 (2021).

30. Chen, Ziliang, and Zhanfu Yang. Graph neural reasoning may fail in certifying boolean unsatisfiability. arXiv preprint arXiv:1909.11588 (2019).

31. Yan, Zhiyuan, et al. Addressing Variable Dependency in GNN-based SAT Solving. arXiv preprint arXiv:2304.08738 (2023).

32. Mukherjee, Prasita, and Tiark Rompf. A GNN Based Approach to LTL Model Checking. arXiv preprint arXiv:2110.14824 (2021).

33. Xu, Feifan, et al. Fast OBDD reordering using neural message passing on hypergraph. arXiv preprint arXiv:1811.02178 (2018).

34. Malmqvist, Lars. Approximate Solutions to Abstract Argumentation Problems Using Graph Neural Networks. Diss. University of York, 2022.

35. Hagström, Fredrik, and Fredrik Hagström. Finding Solutions to the Vehicle Routing Problem using a Graph Neural Network. (2022).

36. Ghose, Amur, Amit Levi, and Yingxue Zhang. Graph neural networks for Ramsey graphs.

37. Hula, Jan, David Mojzıšek, and Mikoláš Janota. Towards Graph Neural Networks for SMT Portfolios.

38. Amizadeh, Saeed, Sergiy Matusevych, and Markus Weimer. PDP: A General Neural Framework for Learning SAT Solvers.

39. de Aquino Afonso, Bruno Klaus, et al. Kdd-br 2021: Using graph neural networks for link prediction in tsp problem. Anais do XVIII Encontro Nacional de Inteligência Artificial e Computacional. SBC, 2021.

40. Hudson, Benjamin, et al. Graph neural network guided local search for the traveling salesperson problem. arXiv preprint arXiv:2110.05291 (2021).

41. Feldman, Or, et al. Weisfeiler and leman go infinite: Spectral and combinatorial pre-colorings. arXiv preprint arXiv:2201.13410 (2022).

Spectral Graph Theories, Laplacian Approximations & Combinatorical Optimization with GNN

1. Cappart, Quentin, et al. Combinatorial optimization and reasoning with graph neural networks. arXiv preprint arXiv:2102.09544 (2021).

2. Yadati, Naganand, et al. Hypergcn: A new method for training graph convolutional networks on hypergraphs. Advances in neural information processing systems 32 (2019).

3. Peng, Yun, Byron Choi, and Jianliang Xu. Graph learning for combinatorial optimization: a survey of state-of-the-art. Data Science and Engineering 6.2 (2021): 119-141.

4. Sahbi, Hichem. Learning laplacians in chebyshev graph convolutional networks. Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.

5. Duan, Haonan, et al. Augment with care: Contrastive learning for combinatorial problems. International Conference on Machine Learning. PMLR, 2022.

6. Yan, Junchi, Shuang Yang, and Edwin R. Hancock. Learning for graph matching and related combinatorial optimization problems. Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20. International Joint Conferences on Artificial Intelligence Organization, 2020.

7. Kuck, Jonathan, et al. Belief propagation neural networks. Advances in Neural Information Processing Systems 33 (2020): 667-678.

8. Sun, Haoran, Etash K. Guha, and Hanjun Dai. Annealed Training for Combinatorial Optimization on Graphs. arXiv preprint arXiv:2207.11542 (2022).

9. Lee, Mengyuan, et al. A fast graph neural network-based method for winner determination in multi-unit combinatorial auctions. IEEE Transactions on Cloud Computing 10.4 (2020): 2264-2280.

10. Gatti, Alice, et al. Graph Partitioning and Sparse Matrix Ordering using Reinforcement Learning and Graph Neural Networks. Journal of Machine Learning Research 23.303 (2022): 1-28.

11. Jovanovic, Raka, et al. Applying graph neural networks to the decision version of graph combinatorial optimization problems. IEEE Access (2023).

12. Li, Zhuwen, Qifeng Chen, and Vladlen Koltun. Combinatorial optimization with graph convolutional networks and guided tree search. Advances in neural information processing systems 31 (2018).

13. Gasse, Maxime, et al. Exact combinatorial optimization with graph convolutional neural networks. Advances in neural information processing systems 32 (2019).

14. Vesselinova, Natalia, et al. Learning combinatorial optimization on graphs: A survey with applications to networking. IEEE Access 8 (2020): 120388-120416.

15. Drori, Iddo, et al. Learning to solve combinatorial optimization problems on real-world graphs in linear time. 2020 19th IEEE International Conference on Machine Learning and Applications (ICMLA). IEEE, 2020.

16. Luz, Ilay, et al. Learning algebraic multigrid using graph neural networks. International Conference on Machine Learning. PMLR, 2020.

17. Khalil, Elias B., Christopher Morris, and Andrea Lodi. Mip-gnn: A data-driven framework for guiding combinatorial solvers. Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 9. 2022.

18. Ju, Haotian, et al. Generalization in Graph Neural Networks: Improved PAC-Bayesian Bounds on Graph Diffusion. International Conference on Artificial Intelligence and Statistics. PMLR, 2023.

19. Angelini, Maria Chiara, and Federico Ricci-Tersenghi. Modern graph neural networks do worse than classical greedy algorithms in solving combinatorial optimization problems like maximum independent set. Nature Machine Intelligence 5.1 (2023): 29-31.

20. Khalil, Elias, et al. Learning combinatorial optimization algorithms over graphs. Advances in neural information processing systems 30 (2017).

21. Schuetz, Martin JA, J. Kyle Brubaker, and Helmut G. Katzgraber. Combinatorial optimization with physics-inspired graph neural networks. Nature Machine Intelligence 4.4 (2022): 367-377.

22. Chen, Ziang, et al. On representing linear programs by graph neural networks. arXiv preprint arXiv:2209.12288 (2022).

23. D'Inverno, Giuseppe Alessio, et al. A new perspective on the approximation capability of GNNs. arXiv preprint arXiv:2106.08992 (2021).

24. Balcilar, Muhammet, et al. Analyzing the expressive power of graph neural networks in a spectral perspective. Proceedings of the International Conference on Learning Representations (ICLR). 2021.

25. Bo, Deyu, et al. A Survey on Spectral Graph Neural Networks. arXiv preprint arXiv:2302.05631 (2023).

Gradient Descent & Error Bounds on Graph Neural Networks (Convex/Concave)

1. Izadi, Mohammad Rasool, et al. Optimization of graph neural networks with natural gradient descent. 2020 IEEE international conference on big data (big data). IEEE, 2020.

2. Awasthi, Pranjal, Abhimanyu Das, and Sreenivas Gollapudi. A Convergence Analysis of Gradient Descent on Graph Neural Networks. Advances in Neural Information Processing Systems 34 (2021): 20385-20397.

3. Lee, Junghyun, et al. A Statistical Analysis of Stochastic Gradient Noises for GNNs. 한국정보과학회 학술발표논문집 (2022): 1025-1027.

4. Li, Qunwei, Shaofeng Zou, and Wenliang Zhong. Learning graph neural networks with approximate gradient descent. Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 35. No. 10. 2021.

5. Yadati, Naganand. A Convex Formulation for Graph Convolutional Training: Two Layer Case. 2022 IEEE International Conference on Data Mining (ICDM). IEEE, 2022.

6. Park, Junyoung, Chihyeon Song, and Jinkyoo Park. Input Convex Graph Neural Networks: An Application to Optimal Control and Design Optimization.

7. Xu, Keyulu, et al. Optimization of graph neural networks: Implicit acceleration by skip connections and more depth. International Conference on Machine Learning. PMLR, 2021.

Graph Representation Learning on Theoretical Natural Sciences

Particle Physics

Almost all the papers are from this link.

1. Hao, Zhongkai, et al. Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications. arXiv preprint arXiv:2211.08064 (2022).

2. Henrion, Isaac, et al. Neural message passing for jet physics. (2017).

3. Ju, Xiangyang, et al. Graph neural networks for particle reconstruction in high energy physics detectors. arXiv preprint arXiv:2003.11603 (2020).

4. Abdughani, Murat, et al. Probing stop pair production at the LHC with graph neural networks. Journal of High Energy Physics 2019.8 (2019): 1-14.

5. Martínez, J. Arjona, et al. Pileup mitigation at the Large Hadron Collider with graph neural networks. The European Physical Journal Plus 134.7 (2019): 333.

6. Ren, Jie, Lei Wu, and Jin Min Yang. Unveiling CP property of top-Higgs coupling with graph neural networks at the LHC. Physics Letters B 802 (2020): 135198.

7. Moreno, Eric A., et al. JEDI-net: a jet identification algorithm based on interaction networks. The European Physical Journal C 80 (2020): 1-15.

8. Qasim, Shah Rukh, et al. Learning representations of irregular particle-detector geometry with distance-weighted graph networks. The European Physical Journal C 79.7 (2019): 1-11.

9. Chakraborty, Amit, et al. Neural network-based top tagger with two-point energy correlations and geometry of soft emissions. Journal of High Energy Physics 2020.7 (2020): 1-47.

10. Abdughani, Murat, et al. Probing the triple Higgs boson coupling with machine learning at the LHC. Physical Review D 104.5 (2021): 056003.

11. Bernreuther, Elias, et al. Casting a graph net to catch dark showers. SciPost Physics 10.2 (2021): 046.

12. Shlomi, Jonathan, Peter Battaglia, and Jean-Roch Vlimant. Graph neural networks in particle physics. Machine Learning: Science and Technology 2.2 (2020): 021001.

13. Iiyama, Yutaro, et al. Distance-weighted graph neural networks on FPGAs for real-time particle reconstruction in high energy physics. Frontiers in big Data 3 (2021): 598927.

14. Choma, Nicholas, et al. Track seeding and labelling with embedded-space graph neural networks. arXiv preprint arXiv:2007.00149 (2020).

15. Guo, Jun, et al. Boosted Higgs boson jet reconstruction via a graph neural network. Physical Review D 103.11 (2021): 116025.

16. Verma, Yogesh, and Satyajit Jena. Particle Track Reconstruction using Geometric Deep Learning. arXiv preprint arXiv:2012.08515 (2020).

17. Qian, Zhen, et al. Vertex and energy reconstruction in JUNO with machine learning methods. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 1010 (2021): 165527.

18. Pata, Joosep, et al. MLPF: efficient machine-learned particle-flow reconstruction using graph neural networks. The European Physical Journal C 81 (2021): 1-14.

19. Biscarat, Catherine, et al. Towards a realistic track reconstruction algorithm based on graph neural networks for the HL-LHC. EPJ Web of Conferences. Vol. 251. EDP Sciences, 2021.

20. Thais, Savannah, and Gage DeZoort. Instance Segmentation Gnns for One-Shot Conformal Tracking at the LHC. arXiv preprint arXiv:2103.06509 (2021).

21. Verma, Yogesh, and Satyajit Jena. Jet characterization in Heavy Ion Collisions by QCD-Aware Graph Neural Networks. arXiv preprint arXiv:2103.14906 (2021).

22. Hariri, Ali, Darya Dyachkova, and Sergei Gleyzer. Graph generative models for fast detector simulations in high energy physics. arXiv preprint arXiv:2104.01725 (2021).

23. Konar, Partha, Vishal S. Ngairangbam, and Michael Spannowsky. Energy-weighted message passing: an infra-red and collinear safe graph neural network algorithm. Journal of High Energy Physics 2022.2 (2022): 1-30.

24. Tsan, Steven, et al. Particle Graph Autoencoders and Differentiable, Learned Energy Mover's Distance. arXiv preprint arXiv:2111.12849 (2021).

25. Gong, Shiqi, et al. An efficient Lorentz equivariant graph neural network for jet tagging. Journal of High Energy Physics 2022.7 (2022): 1-22.

26. Qasim, Shah Rukh, et al. End-to-end multi-particle reconstruction in high occupancy imaging calorimeters with graph neural networks. The European Physical Journal C 82.8 (2022): 753.

27. Ma, Fei, Feiyi Liu, and Wei Li. A jet tagging algorithm of graph network with HaarPooling message passing. arXiv preprint arXiv:2210.13869 (2022).

28. Bogatskiy, Alexander, et al. PELICAN: Permutation Equivariant and Lorentz Invariant or Covariant Aggregator Network for Particle Physics. arXiv preprint arXiv:2211.00454 (2022).

29. Di Bello, Francesco Armando, et al. Reconstructing particles in jets using set transformer and hypergraph prediction networks. arXiv preprint arXiv:2212.01328 (2022).

30. Ehrke, Lukas, et al. Topological Reconstruction of Particle Physics Processes using Graph Neural Networks. arXiv preprint arXiv:2303.13937 (2023).

Theoretical Biochemistry

There are huge amounts of GNN works done in this domain. Here we extract some important works for quick references, including molecule, drug, protein, retrosynthesis and single cell RNA seq.

1. Gilmer, Justin, et al. Neural message passing for quantum chemistry. International conference on machine learning. PMLR, 2017.

2. Thölke, Philipp, and Gianni De Fabritiis. Equivariant transformers for neural network based molecular potentials. International Conference on Learning Representations. 2022.

3. Schütt, Kristof, et al. Schnet: A continuous-filter convolutional neural network for modeling quantum interactions. Advances in neural information processing systems 30 (2017).

4. Feinberg, Evan N., et al. PotentialNet for molecular property prediction. ACS central science 4.11 (2018): 1520-1530.

5. Lu, Chengqiang, et al. Molecular property prediction: A multilevel quantum interactions modeling perspective. Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 33. No. 01. 2019.

6. You, Jiaxuan, et al. Graph convolutional policy network for goal-directed molecular graph generation. Advances in neural information processing systems 31 (2018).

7. Mercado, Rocío, et al. Graph networks for molecular design. Machine Learning: Science and Technology 2.2 (2021): 025023.

8. Liu, Qi, et al. Constrained graph variational autoencoders for molecule design. Advances in neural information processing systems 31 (2018).

9. Liu, Shengchao, Hongyu Guo, and Jian Tang. Molecular geometry pretraining with se (3)-invariant denoising distance matching. arXiv preprint arXiv:2206.13602 (2022).

10. Wang, Juexin, et al. scGNN is a novel graph neural network framework for single-cell RNA-Seq analyses. Nature communications 12.1 (2021): 1882.

11. Gu, Haocheng, et al. scGNN 2.0: a graph neural network tool for imputation and clustering of single-cell RNA-Seq data. Bioinformatics 38.23 (2022): 5322-5325.

12. Gan, Yanglan, et al. Deep structural clustering for single-cell RNA-seq data jointly through autoencoder and graph neural network. Briefings in Bioinformatics 23.2 (2022).

13. Wen, Hongzhi, et al. Graph neural networks for multimodal single-cell data integration. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2022.

14. Shao, Xin, et al. scDeepSort: a pre-trained cell-type annotation method for single-cell transcriptomics using deep learning with a weighted graph neural network. Nucleic acids research 49.21 (2021): e122-e122.

15. Zeng, Yuansong, et al. A robust and scalable graph neural network for accurate single-cell classification. Briefings in Bioinformatics 23.2 (2022): bbab570.

16. Liu, Cheng-Hao, et al. RetroGNN: Approximating retrosynthesis by graph neural networks for de novo drug design. arXiv preprint arXiv:2011.13042 (2020).

17. Dai, Hanjun, et al. Retrosynthesis prediction with conditional graph logic network. Advances in Neural Information Processing Systems 32 (2019).

18. Tu, Zhengkai, and Connor W. Coley. Permutation invariant graph-to-sequence model for template-free retrosynthesis and reaction prediction. Journal of chemical information and modeling 62.15 (2022): 3503-3513.

19. Somnath, Vignesh Ram, et al. Learning graph models for retrosynthesis prediction. Advances in Neural Information Processing Systems 34 (2021): 9405-9415.

20. Mao, Kelong, et al. Molecular graph enhanced transformer for retrosynthesis prediction. Neurocomputing 457 (2021): 193-202.

21. Shi, Chence, et al. A graph to graphs framework for retrosynthesis prediction. International conference on machine learning. PMLR, 2020.

22. Han, Peng, et al. Gnn-retro: Retrosynthetic planning with graph neural networks. Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 4. 2022.

23. Strokach, Alexey, et al. Fast and flexible protein design using deep graph neural networks. Cell systems 11.4 (2020): 402-411.

24. You, Ronghui, et al. DeepGraphGO: graph neural network for large-scale, multispecies protein function prediction. Bioinformatics 37.Supplement_1 (2021): i262-i271.

25. Ingraham, John, et al. Generative models for graph-based protein design. Advances in neural information processing systems 32 (2019).

26. Fout, Alex, et al. Protein interface prediction using graph convolutional networks. Advances in neural information processing systems 30 (2017).

27. Zhang, Zuobai, et al. Protein representation learning by geometric structure pretraining. arXiv preprint arXiv:2203.06125 (2022).

28. Zhang, Zuobai, et al. Enhancing protein language models with structure-based encoder and pre-training. arXiv preprint arXiv:2303.06275 (2023).

29. Chen, Can, et al. Structure-aware protein self-supervised learning. Bioinformatics 39.4 (2023): btad189.

30. Yu, Yue, et al. DAG-GNN: DAG structure learning with graph neural networks. International Conference on Machine Learning. PMLR, 2019.

Partial / Ordinary Differential Equations

1. Poli, Michael, et al. Graph neural ordinary differential equations. arXiv preprint arXiv:1911.07532 (2019).

2. Rusch, T. Konstantin, et al. Graph-coupled oscillator networks. International Conference on Machine Learning. PMLR, 2022.

3. Xhonneux, Louis-Pascal, Meng Qu, and Jian Tang. Continuous graph neural networks. International Conference on Machine Learning. PMLR, 2020.

4. Zhuang, Juntang, et al. Ordinary differential equations on graph networks. (2020).

5. Chamberlain, Ben, et al. Grand: Graph neural diffusion. International Conference on Machine Learning. PMLR, 2021.

6. Thorpe, Matthew, et al. GRAND++: Graph neural diffusion with a source term. International Conference on Learning Representations. 2022.

7. Di Giovanni, Francesco, et al. Graph neural networks as gradient flows. arXiv preprint arXiv:2206.10991 (2022).

8. Jin, Ming, et al. Multivariate time series forecasting with dynamic graph neural ODEs. IEEE Transactions on Knowledge and Data Engineering (2022).

9. Hadou, Samar, Charilaos I. Kanatsoulis, and Alejandro Ribeiro. Space-time graph neural networks. arXiv preprint arXiv:2110.02880 (2021).

10. Baker, Justin, et al. Proximal Implicit ODE Solvers for Accelerating Learning Neural ODEs. arXiv preprint arXiv:2204.08621 (2022).

11. Elhag, Ahmed AA, et al. Graph Anisotropic Diffusion. arXiv preprint arXiv:2205.00354 (2022).

12. Qin, Yifang, et al. Learning Graph ODE for Continuous-Time Sequential Recommendation. arXiv preprint arXiv:2304.07042 (2023).

13. Nguyen, Khang, et al. DeepGRAND: Deep Graph Neural Diffusion. (2023).

14. Huang, Chuyu. STR-GODEs: Spatial-Temporal-Ridership Graph ODEs for Metro Ridership Prediction. arXiv preprint arXiv:2107.04980 (2021).

15. Xia, Hedi. Towards Faster and More Accurate Neural ODEs. Diss. University of California, Los Angeles, 2023.

16. Hua, Chuanbo, et al. Efficient Continuous Spatio-Temporal Simulation with Graph Spline Networks. ICML 2022 2nd AI for Science Workshop.

17. Di Giovanni, Francesco. Graph neural networks as dynamical systems.

18. Pal, Soumyasundar. Monte Carlo algorithms for nonlinear filtering, bayesian graph neural networks, and probabilistic forecasting. (2022).

19. Brandstetter, Johannes, Daniel Worrall, and Max Welling. Message passing neural PDE solvers. arXiv preprint arXiv:2202.03376 (2022).

20. Qin, Tiexin, et al. Learning Dynamic Graph Embeddings with Neural Controlled Differential Equations. arXiv preprint arXiv:2302.11354 (2023).

21. Takamoto, Makoto, et al. PDEBench: An extensive benchmark for scientific machine learning Advances in Neural Information Processing Systems 35 (2022): 1596-1611.

22. Sanchez-Gonzalez, Alvaro, et al. Hamiltonian graph networks with ode integrators. arXiv preprint arXiv:1909.12790 (2019).

23. Ruhe, David, et al. Geometric clifford algebra networks. arXiv preprint arXiv:2302.06594 (2023).

24. Wu, Tailin, Takashi Maruyama, and Jure Leskovec. Learning to accelerate partial differential equations via latent global evolution. arXiv preprint arXiv:2206.07681 (2022).

25. Lütjens, Björn, et al. Multiscale neural operator: Learning fast and grid-independent pde solvers. arXiv preprint arXiv:2207.11417 (2022).

26. Horie, Masanobu, and Naoto Mitsume. Physics-Embedded Neural Networks: Graph Neural PDE Solvers with Mixed Boundary Conditions. Advances in Neural Information Processing Systems 35 (2022): 23218-23229.

27. Equer, Léonard, T. Konstantin Rusch, and Siddhartha Mishra. Multi-scale message passing neural pde solvers. arXiv preprint arXiv:2302.03580 (2023).

28. Lötzsch, Winfried, Simon Ohler, and Johannes S. Otterbach. Learning the solution operator of boundary value problems using graph neural networks. arXiv preprint arXiv:2206.14092 (2022).

29. Wang, Yuelin, et al. ACMP: Allen-Cahn Message Passing for Graph Neural Networks with Particle Phase Transition. arXiv preprint arXiv:2206.05437 (2022).

30. Horie, Masanobu, and Naoto Mitsume. Physics-Embedded Neural Networks: $\boldsymbol {\mathrm {E}(n)} $-Equivariant Graph Neural PDE Solvers. arXiv preprint arXiv:2205.11912 (2022).

31. Toshev, Artur P., et al. Learning Lagrangian Fluid Mechanics with E ($3 $)-Equivariant Graph Neural Networks. arXiv preprint arXiv:2305.15603 (2023).

32. Serrano, Louis, et al. Operator Learning with Neural Fields: Tackling PDEs on General Geometries. arXiv preprint arXiv:2306.07266 (2023).

33. Huang, Xinquan, et al. NeuralStagger: accelerating physics-constrained neural PDE solver with spatial-temporal decomposition. arXiv preprint arXiv:2302.10255 (2023).

34. Nastorg, Matthieu, et al. An Implicit GNN Solver for Poisson-like problems. arXiv preprint arXiv:2302.10891 (2023).

35. Boussif, Oussama, et al. MAgnet: Mesh agnostic neural PDE solver. Advances in Neural Information Processing Systems 35 (2022): 31972-31985.

36. Hua, Chuanbo, et al. Graph Spline Networks for Efficient Continuous Simulation of Dynamical Systems. (2023).

37. Li, Yichen, et al. NeuralPCG: Learning Preconditioner for Solving Partial Differential Equations with Graph Neural Network. (2023).

38. Li, Qing, et al. Solving Nonlinear Conservation Laws of Partial Differential Equations Using Graph Neural Networks. Proceedings of the Northern Lights Deep Learning Workshop. Vol. 4. 2023.

39. Wu, Tailin, et al. Learning Controllable Adaptive Simulation for Multi-scale Physics. NeurIPS 2022 AI for Science: Progress and Promises.

Geology, Transportation and Seismology (Selected)

1. Hillier, Michael, et al. Three-dimensional structural geological modeling using graph neural networks. Mathematical Geosciences 53.8 (2021): 1725-1749.

2. Kuang, Ping, et al. Landslide Displacement Prediction via Attentive Graph Neural Network. Remote Sensing 14.8 (2022): 1919.

3. Sun, Alexander Y., et al. Explore Spatio‐Temporal Learning of Large Sample Hydrology Using Graph Neural Networks. Water Resources Research 57.12 (2021): e2021WR030394.

4. McBrearty, Ian W., and Gregory C. Beroza. Earthquake phase association with graph neural networks. Bulletin of the Seismological Society of America 113.2 (2023): 524-547.

5. Gong, Renbin, and Yanzi Yang. Application of Graph Neural Network in Lithology Recognition. 2022 4th International Conference on Natural Language Processing (ICNLP). IEEE, 2022.

6. Zhou, Fan, et al. Land deformation prediction via slope-aware graph neural networks. Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 35. No. 17. 2021.

7. Li, Rongfan, et al. Mining Spatio-Temporal Relations via Self-Paced Graph Contrastive Learning. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2022.

8. Li, Yun, et al. Graph Neural Network for spatiotemporal data: methods and applications. arXiv preprint arXiv:2306.00012 (2023).

9. Choi, Yongjin, and Krishna Kumar. Graph Neural Network-based surrogate model for granular flows. arXiv preprint arXiv:2305.05218 (2023).

10. McBrearty, Ian W., and Gregory C. Beroza. Earthquake location and magnitude estimation with graph neural networks. 2022 IEEE International Conference on Image Processing (ICIP). IEEE, 2022.

11. McBrearty, Ian W., and Gregory C. Beroza. Earthquake phase association with graph neural networks. Bulletin of the Seismological Society of America 113.2 (2023): 524-547.

12. Bloemheuvel, Stefan, et al. Multivariate time series regression with graph neural networks. arXiv preprint arXiv:2201.00818 (2022).

13. Bilal, Muhammad Atif, et al. Early Earthquake Detection Using Batch Normalization Graph Convolutional Neural Network (BNGCNN). Applied Sciences 12.15 (2022): 7548.

14. Ruiz, Luana, Fernando Gama, and Alejandro Ribeiro. Gated graph convolutional recurrent neural networks. 2019 27th European Signal Processing Conference (EUSIPCO). IEEE, 2019.

15. Zhao, Wenzhi, et al. Contextual-Aware Land Cover Classification with U-Shaped Object Graph Neural Network. IEEE Geoscience and Remote Sensing Letters 19 (2022): 1-5.

16. Zhou, Fan, et al. Identifying user geolocation with Hierarchical Graph Neural Networks and explainable fusion. Information Fusion 81 (2022): 1-13.

17. Jiang, Weiwei, and Jiayun Luo. Graph neural network for traffic forecasting: A survey. Expert Systems with Applications (2022): 117921.

18. Wu, Ning, et al. Learning effective road network representation with hierarchical graph neural networks. Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. 2020.

19. Peng, Hao, et al. Spatial temporal incidence dynamic graph neural networks for traffic flow forecasting. Information Sciences 521 (2020): 277-290.

20. Ma, Zhaorui, et al. GWS-Geo: A graph neural network based model for street-level IPv6 geolocation. Journal of Information Security and Applications 75 (2023): 103511.

21. Sarlin, Paul-Edouard, et al. Superglue: Learning feature matching with graph neural networks. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.

22. Wang, Xiaoyang, et al. Traffic flow prediction via spatial temporal graph neural network. Proceedings of the web conference 2020. 2020.

23. Zhang, Fang-Hao, and Zhi-Gang Shao. ST-GRF: Spatiotemporal graph neural networks for rainfall forecasting. Digital Signal Processing 136 (2023): 103989.

24. Jin, Guangyin, et al. Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey. arXiv preprint arXiv:2303.14483 (2023).

25. Chang, Buru, et al. Learning graph-based geographical latent representation for point-of-interest recommendation. Proceedings of the 29th ACM International conference on information & knowledge management. 2020.

26. Tulczyjew, Lukasz, et al. Graph neural networks extract high-resolution cultivated land maps from Sentinel-2 image series. IEEE Geoscience and Remote Sensing Letters 19 (2022): 1-5.

27. Saha, Sudipan, Shan Zhao, and Xiao Xiang Zhu. Multitarget domain adaptation for remote sensing classification using graph neural network. IEEE Geoscience and Remote Sensing Letters 19 (2022): 1-5.

28. Yan, Jingjing, Shunping Ji, and Yao Wei. A combination of convolutional and graph neural networks for regularized road surface extraction. IEEE transactions on geoscience and remote sensing 60 (2022): 1-13.

29. Seastream, Grant, and Huseyin Denli. Geoscientific reasoning via graph neural networks. SEG/AAPG International Meeting for Applied Geoscience & Energy. OnePetro, 2022.

30. Zhao, Wenzhi, et al. Contextual-Aware Land Cover Classification with U-Shaped Object Graph Neural Network. IEEE Geoscience and Remote Sensing Letters 19 (2022): 1-5.

31. Klemmer, Konstantin, Nathan S. Safir, and Daniel B. Neill. Positional encoder graph neural networks for geographic data. International Conference on Artificial Intelligence and Statistics. PMLR, 2023.

Cosmology, Astrophysics

1. Cranmer, Miles, Peter Melchior, and Brian Nord. Unsupervised resource allocation with graph neural networks. NeurIPS 2020 Workshop on Pre-registration in Machine Learning. PMLR, 2021.

2. Jespersen, Christian Kragh, et al. $\texttt {Mangrove} $: Learning Galaxy Properties from Merger Trees. arXiv preprint arXiv:2210.13473 (2022).

3. Wang, Tianshu, and Peter Melchior. Graph neural network-based resource allocation strategies for multi-object spectroscopy. Machine Learning: Science and Technology 3.1 (2022): 015023.

4. Farsian, Farida, et al. New applications of Graph Neural Networks in Cosmology. arXiv preprint arXiv:2210.11487 (2022).

5. Villanueva-Domingo, Pablo, et al. Inferring Halo Masses with Graph Neural Networks. The Astrophysical Journal 935.1 (2022): 30.

6. Villanueva-Domingo, Pablo, and Francisco Villaescusa-Navarro. Learning cosmology and clustering with cosmic graphs. The Astrophysical Journal 937.2 (2022): 115.

7. Nguyen, Tri, et al. Uncovering dark matter density profiles in dwarf galaxies with graph neural networks. Physical Review D 107.4 (2023): 043015.

8. Koundal, Paras, and IceCube Collaboration. Composition Analysis of cosmic-rays at IceCube Observatory, using Graph Neural Networks. Workshop on Machine Learning for Cosmic-Ray Air Showers (2022), Newark, DE, USA, 31.01. 2022–03.02. 2022. 2022.

9. Jagvaral, Yesukhei, et al. Galaxies on graph neural networks: towards robust synthetic galaxy catalogs with deep generative models. arXiv preprint arXiv:2212.05596 (2022).

10. Choma, Nicholas, et al. Graph neural networks for icecube signal classification. 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA). IEEE, 2018.

11. Reck, S., et al. Graph neural networks for reconstruction and classification in KM3NeT. Journal of Instrumentation 16.10 (2021): C10011.

12. Koundal, Paras, Matthias Plum, and Julian Saffer. Study of mass composition of cosmic rays with IceTop and IceCube. arXiv preprint arXiv:2107.09626 (2021).

13. ...

TODO List:

1. Energy based Graph Neural Networks (First Part)
2. Finding more references on spectral graph theories with graph neural networks (Traditional Analysis + Learning Based Methods) in first part
3. Add neural SDE (stochastic differential equation. e.g. Molecular Dynamics) solvers 12/10/2023

This repo is going to be frequently updated. Some new theoretical papers will be added. Old papers or unimportant papers will be removed after careful considerations.