Awesome-LLMs-in-Graph-tasks
A curated collection of research papers exploring the utilization of LLMs for graph-related tasks.
Stars: 506
This repository is a collection of papers on leveraging Large Language Models (LLMs) in Graph Tasks. It provides a comprehensive overview of how LLMs can enhance graph-related tasks by combining them with traditional Graph Neural Networks (GNNs). The integration of LLMs with GNNs allows for capturing both structural and contextual aspects of nodes in graph data, leading to more powerful graph learning. The repository includes summaries of various models that leverage LLMs to assist in graph-related tasks, along with links to papers and code repositories for further exploration.
README:
This is a collection of papers on leveraging Large Language Models in Graph Tasks. It's based on our survey paper: A Survey of Graph Meets Large Language Model: Progress and Future Directions.
We will try to make this list updated frequently. If you found any error or any missed paper, please don't hesitate to open issues or pull requests.
Our survey has been accepted by IJCAI 2024 survey track.
With the help of LLMs, there has been a notable shift in the way we interact with graphs, particularly those containing nodes associated with text attributes. The integration of LLMs with traditional GNNs can be mutually beneficial and enhance graph learning. While GNNs are proficient at capturing structural information, they primarily rely on semantically constrained embeddings as node features, limiting their ability to express the full complexities of the nodes. Incorporating LLMs, GNNs can be enhanced with stronger node features that effectively capture both structural and contextual aspects. On the other hand, LLMs excel at encoding text but often struggle to capture structural information present in graph data. Combining GNNs with LLMs can leverage the robust textual understanding of LLMs while harnessing GNNs' ability to capture structural relationships, leading to more comprehensive and powerful graph learning.
Figure 1. The overview of Graph Meets LLMs.
Table 1. A summary of models that leverage LLMs to assist graph-related tasks in literature, ordered by their release time. Fine-tuning denotes whether it is necessary to fine-tune the parameters of LLMs, and ♥ indicates that models employ parameter-efficient fine-tuning (PEFT) strategies, such as LoRA and prefix tuning. Prompting indicates the use of text-formatted prompts in LLMs, done manually or automatically. Acronyms in Task: Node refers to node-level tasks; Link refers to link-level tasks; Graph refers to graph-level tasks; Reasoning refers to Graph Reasoning; Retrieval refers to Graph-Text Retrieval; Captioning refers to Graph Captioning.
-
(2022.03) [ICLR' 2022] Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction [Paper | Code]
-
(2023.02) [ICLR' 2023] Edgeformers: Graph-Empowered Transformers for Representation Learning on Textual-Edge Networks [Paper | Code]
-
(2023.05) [KDD' 2023] Graph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applications [Paper]
-
(2023.06) [KDD' 2023] Heterformer: Transformer-based Deep Node Representation Learning on Heterogeneous Text-Rich Networks [Paper | Code]
-
(2023.05) [ICLR' 2024] Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning [Paper | Code]
-
(2023.08) [Arxiv' 2023] Exploring the potential of large language models (llms) in learning on graphs [Paper]
-
(2023.07) [Arxiv' 2023] Can Large Language Models Empower Molecular Property Prediction? [Paper | Code]
-
(2023.08) [Arxiv' 2023] Simteg: A frustratingly simple approach improves textual graph learning [Paper | Code]
-
(2023.09) [Arxiv' 2023] Prompt-based Node Feature Extractor for Few-shot Learning on Text-Attributed Graphs [Paper]
-
(2023.09) [Arxiv' 2023] TouchUp-G: Improving Feature Representation through Graph-Centric Finetuning [Paper]
-
(2023.09) [ICLR' 2024] One for All: Towards Training One Graph Model for All Classification Tasks [Paper | Code]
-
(2023.10) [Arxiv' 2023] Learning Multiplex Embeddings on Text-rich Networks with One Text Encoder [Paper | Code]
-
(2023.11) [WSDM' 2024] LLMRec: Large Language Models with Graph Augmentation for Recommendation [Paper | Code]
-
(2023.11) [NeurIPS' 2023] WalkLM: A Uniform Language Model Fine-tuning Framework for Attributed Graph Embedding [Paper | Code]
-
(2024.01) [IJCAI' 2024] Efficient Tuning and Inference for Large Language Models on Textual Graphs [Paper]
-
(2024.02) [KDD' 2024] ZeroG: Investigating Cross-dataset Zero-shot Transferability in Graphs [Paper]
-
(2024.02) [Arxiv' 2024] UniGraph: Learning a Cross-Domain Graph Foundation Model From Natural Language [Paper]
-
(2024.02) [CIKM' 2024] Distilling Large Language Models for Text-Attributed Graph Learning [Paper]
-
(2023.05) [NeurIPS' 2023] Can language models solve graph problems in natural language? [Paper | Code]
-
(2023.05) [Arxiv' 2023] GPT4Graph: Can Large Language Models Understand Graph Structured Data? An Empirical Evaluation and Benchmarking [Paper | Code]
-
(2023.06) [NeurIPS' 2023] GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learning [Paper | Code]
-
(2023.07) [Arxiv' 2023] Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs [Paper | Code]
-
(2023.08) [Arxiv' 2023] GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Text [Paper]
-
(2023.08) [Arxiv' 2023] Natural Language is All a Graph Needs [Paper | Code]
-
(2023.08) [Arxiv' 2023] Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysis [Paper | Code]
-
(2023.09) [Arxiv' 2023] Can LLMs Effectively Leverage Graph Structural Information: When and Why [Paper | Code]
-
(2023.10) [Arxiv' 2023] GraphText: Graph Reasoning in Text Space [Paper] | Code]
-
(2023.10) [Arxiv' 2023] Talk like a Graph: Encoding Graphs for Large Language Models [Paper]
-
(2023.10) [Arxiv' 2023] GraphLLM: Boosting Graph Reasoning Ability of Large Language Model [Paper | Code]
-
(2023.10) [Arxiv' 2023] Beyond Text: A Deep Dive into Large Language Model [Paper]
-
(2023.10) [EMNLP' 2023] MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter [Paper | Code]
-
(2023.10) [Arxiv' 2023] GraphGPT: Graph Instruction Tuning for Large Language Models [Paper | Code]
-
(2023.10) [EMNLP' 2023] ReLM: Leveraging Language Models for Enhanced Chemical Reaction Prediction [Paper | Code]
-
(2023.10) [Arxiv' 2023] LLM4DyG: Can Large Language Models Solve Problems on Dynamic Graphs? [Paper]
-
(2023.10) [Arxiv' 2023] Disentangled Representation Learning with Large Language Models for Text-Attributed Graphs [Paper]
-
(2023.11) [Arxiv' 2023] Which Modality should I use -- Text, Motif, or Image? : Understanding Graphs with Large Language Models [Paper]
-
(2023.11) [Arxiv' 2023] InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discovery [Paper]
-
(2023.12) [Arxiv' 2023] When Graph Data Meets Multimodal: A New Paradigm for Graph Understanding and Reasoning [Paper]
-
(2024.02) [Arxiv' 2024] Let Your Graph Do the Talking: Encoding Structured Data for LLMs [Paper]
-
(2024.02) [Arxiv' 2024] Rendering Graphs for Graph Reasoning in Multimodal Large Language Models [Paper]
-
(2024.02) [WWW' 2024] GraphTranslator: Aligning Graph Model to Large Language Model for Open-ended Tasks [Paper | Code]
-
(2024.02) [Arxiv' 2024] InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignment [Paper | Code]
-
(2024.02) [Arxiv' 2024] LLaGA: Large Language and Graph Assistant [Paper | Code]
-
(2024.02) [WWW' 2024] Can GNN be Good Adapter for LLMs? [Paper]
-
(2024.02) [Arxiv' 2024] HiGPT: Heterogeneous Graph Language Model [Paper | Code]
-
(2024.02) [Arxiv' 2024] GraphWiz: An Instruction-Following Language Model for Graph Problems [Paper | Code]
-
(2024.03) [Arxiv' 2024] OpenGraph: Towards Open Graph Foundation Models [Paper | Code]
-
(2024.07) [Arxiv' 2024] GOFA: A Generative One-For-All Model for Joint Graph Language Modeling [Paper | Code]
- (2020.08) [Arxiv' 2020] Graph-based Modeling of Online Communities for Fake News Detection [Paper | Code]
- (2021.05) [NeurIPS' 2021] GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph [Paper | Code]
- (2021.11) [EMNLP' 2021] Text2Mol: Cross-Modal Molecule Retrieval with Natural Language Queries [Paper | Code]
- (2022.07) [ACL' 2023] Hidden Schema Networks [Paper | Code]
- (2022.09) [Arxiv' 2022] A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language [Paper | Code]
- (2022.10) [ICLR' 2023] Learning on Large-scale Text-attributed Graphs via Variational Inference [Paper | Code]
- (2022.12) [NMI' 2023] Multi-modal Molecule Structure-text Model for Text-based Editing and Retrieval [Paper | Code]
- (2023.04) [Arxiv' 2023] Train Your Own GNN Teacher: Graph-Aware Distillation on Textual Graphs [Paper | Code]
- (2023.05) [ACL' 2023] PATTON : Language Model Pretraining on Text-Rich Networks [Paper | Code]
- (2023.05) [Arxiv' 2023] ConGraT: Self-Supervised Contrastive Pretraining for Joint Graph and Text Embeddings [Paper | Code]
- (2023.07) [Arxiv' 2023] Prompt Tuning on Graph-augmented Low-resource Text Classification [Paper | Code]
- (2023.10) [EMNLP' 2023] GRENADE: Graph-Centric Language Model for Self-Supervised Representation Learning on Text-Attributed Graphs [Paper | Code]
- (2023.10) [WWW' 2024] Representation Learning with Large Language Models for Recommendation [Paper | Code]
- (2023.10) [EMNLP' 2023] Pretraining Language Models with Text-Attributed Heterogeneous Graphs [Paper | Code]
-
(2023.10) [ICLR' 2024] Label-free Node Classification on Graphs with Large Language Models (LLMs) [Paper | Code]
-
(2024.09) [NeurIPS' 2024] Entity Alignment with Noisy Annotations from Large Language Models [Paper | Code]
- (2023.10) [Arxiv' 2023] Graph Neural Architecture Search with GPT-4 [Paper]
- (2023.10) [Arxiv' 2023] Empower Text-Attributed Graphs Learning with Large Language Models (LLMs) [Paper]
- (2023.11) [Arxiv' 2023] Large Language Models as Topological Structure Enhancers for Text-Attributed Graphs [Paper]
-
(2024.05) [Arxiv' 2024] Intruding with Words: Towards Understanding Graph Injection Attacks at the Text Level [Paper]
-
(2024.08) [Arxiv' 2024] Can Large Language Models Improve the Adversarial Robustness of Graph Neural Networks? [Paper]
We note that several repos also summarize papers on the integration of LLMs and graphs. However, we differentiate ourselves by organizing these papers leveraging a new and more granular taxonomy. We recommend researchers to explore some repositories for a comprehensive survey.
-
Awesome-Graph-LLM, created by Xiaoxin He from NUS.
-
Awesome-Large-Graph-Model, created by Ziwei Zhang from THU.
-
Awesome-Language-Model-on-Graphs, created by Bowen Jin from UIUC.
We highly recommend a repository that summarizes the work on Graph Prompt, which is very close to Graph-LLM.
- Awesome-Graph-Prompt, created by Xixi Wu from CUHK.
If you have come across relevant resources, feel free to open an issue or submit a pull request.
* (_time_) [conference] **paper_name** [[Paper](link) | [Code](link)]
<details close>
<summary>Model name</summary>
<p align="center"><img width="75%" src="Figures/xxx.jpg" /></p>
<p align="center"><em>The framework of model name.</em></p>
</details>
Feel free to cite this work if you find it useful to you!
@article{li2023survey,
title={A Survey of Graph Meets Large Language Model: Progress and Future Directions},
author={Li, Yuhan and Li, Zhixun and Wang, Peisong and Li, Jia and Sun, Xiangguo and Cheng, Hong and Yu, Jeffrey Xu},
journal={arXiv preprint arXiv:2311.12399},
year={2023}
}
For Tasks:
Click tags to check more tools for each tasksFor Jobs:
Alternative AI tools for Awesome-LLMs-in-Graph-tasks
Similar Open Source Tools
Awesome-LLMs-in-Graph-tasks
This repository is a collection of papers on leveraging Large Language Models (LLMs) in Graph Tasks. It provides a comprehensive overview of how LLMs can enhance graph-related tasks by combining them with traditional Graph Neural Networks (GNNs). The integration of LLMs with GNNs allows for capturing both structural and contextual aspects of nodes in graph data, leading to more powerful graph learning. The repository includes summaries of various models that leverage LLMs to assist in graph-related tasks, along with links to papers and code repositories for further exploration.
prompt-in-context-learning
An Open-Source Engineering Guide for Prompt-in-context-learning from EgoAlpha Lab. 📝 Papers | ⚡️ Playground | 🛠 Prompt Engineering | 🌍 ChatGPT Prompt | ⛳ LLMs Usage Guide > **⭐️ Shining ⭐️:** This is fresh, daily-updated resources for in-context learning and prompt engineering. As Artificial General Intelligence (AGI) is approaching, let’s take action and become a super learner so as to position ourselves at the forefront of this exciting era and strive for personal and professional greatness. The resources include: _🎉Papers🎉_: The latest papers about _In-Context Learning_ , _Prompt Engineering_ , _Agent_ , and _Foundation Models_. _🎉Playground🎉_: Large language models(LLMs)that enable prompt experimentation. _🎉Prompt Engineering🎉_: Prompt techniques for leveraging large language models. _🎉ChatGPT Prompt🎉_: Prompt examples that can be applied in our work and daily lives. _🎉LLMs Usage Guide🎉_: The method for quickly getting started with large language models by using LangChain. In the future, there will likely be two types of people on Earth (perhaps even on Mars, but that's a question for Musk): - Those who enhance their abilities through the use of AIGC; - Those whose jobs are replaced by AI automation. 💎EgoAlpha: Hello! human👤, are you ready?
chatgpt-auto-refresh
ChatGPT Auto Refresh is a userscript that keeps ChatGPT sessions fresh by eliminating network errors and Cloudflare checks. It removes the 10-minute time limit from conversations when Chat History is disabled, ensuring a seamless experience. The tool is safe, lightweight, and a time-saver, allowing users to keep their sessions alive without constant copy/paste/refresh actions. It works even in background tabs, providing convenience and efficiency for users interacting with ChatGPT. The tool relies on the chatgpt.js library and is compatible with various browsers using Tampermonkey, making it accessible to a wide range of users.
LLM-for-misinformation-research
LLM-for-misinformation-research is a curated paper list of misinformation research using large language models (LLMs). The repository covers methods for detection and verification, tools for fact-checking complex claims, decision-making and explanation, claim matching, post-hoc explanation generation, and other tasks related to combating misinformation. It includes papers on fake news detection, rumor detection, fact verification, and more, showcasing the application of LLMs in various aspects of misinformation research.
bravegpt
BraveGPT is a userscript that brings the power of ChatGPT to Brave Search. It allows users to engage with a conversational AI assistant directly within their search results, providing instant and personalized responses to their queries. BraveGPT is powered by GPT-4, the latest and most advanced language model from OpenAI, ensuring accurate and comprehensive answers. With BraveGPT, users can ask questions, get summaries, generate creative content, and more, all without leaving the Brave Search interface. The tool is easy to install and use, making it accessible to users of all levels. BraveGPT is a valuable addition to the Brave Search experience, enhancing its capabilities and providing users with a more efficient and informative search experience.
Awesome-Graph-LLM
Awesome-Graph-LLM is a curated collection of research papers exploring the intersection of graph-based techniques with Large Language Models (LLMs). The repository aims to bridge the gap between LLMs and graph structures prevalent in real-world applications by providing a comprehensive list of papers covering various aspects of graph reasoning, node classification, graph classification/regression, knowledge graphs, multimodal models, applications, and tools. It serves as a valuable resource for researchers and practitioners interested in leveraging LLMs for graph-related tasks.
NewEraAI-Papers
The NewEraAI-Papers repository provides links to collections of influential and interesting research papers from top AI conferences, along with open-source code to promote reproducibility and provide detailed implementation insights beyond the scope of the article. Users can stay up to date with the latest advances in AI research by exploring this repository. Contributions to improve the completeness of the list are welcomed, and users can create pull requests, open issues, or contact the repository owner via email to enhance the repository further.
chatgpt.js
chatgpt.js is a powerful JavaScript library that allows for super easy interaction w/ the ChatGPT DOM. * Feature-rich * Object-oriented * Easy-to-use * Lightweight (yet optimally performant)
awesome-LLM-AIOps
The 'awesome-LLM-AIOps' repository is a curated list of academic research and industrial materials related to Large Language Models (LLM) and Artificial Intelligence for IT Operations (AIOps). It covers various topics such as incident management, log analysis, root cause analysis, incident mitigation, and incident postmortem analysis. The repository provides a comprehensive collection of papers, projects, and tools related to the application of LLM and AI in IT operations, offering valuable insights and resources for researchers and practitioners in the field.
googlegpt
GoogleGPT is a browser extension that brings the power of ChatGPT to Google Search. With GoogleGPT, you can ask ChatGPT questions and get answers directly in your search results. You can also use GoogleGPT to generate text, translate languages, and more. GoogleGPT is compatible with all major browsers, including Chrome, Firefox, Edge, and Safari.
chatgpt-widescreen
ChatGPT Widescreen Mode is a browser extension that adds widescreen and fullscreen modes to ChatGPT, enhancing chat sessions by reducing scrolling and creating a more immersive viewing experience. Users can experience clearer programming code display, view multi-step instructions or long recipes on a single page, enjoy original content in a visually pleasing format, customize features like a larger chatbox and hidden header/footer, and use the tool with chat.openai.com and poe.com. The extension is compatible with various browsers and relies on code from the chatgpt.js library under the MIT license.
chatgpt-infinity
ChatGPT Infinity is a free and powerful add-on that makes ChatGPT generate infinite answers on any topic. It offers customizable topic selection, multilingual support, adjustable response interval, and auto-scroll feature for a seamless chat experience.
duckduckgpt
DuckDuckGPT brings the magic of ChatGPT to DDG (powered by GPT-4!). DuckDuckGPT is a browser extension that allows you to use ChatGPT within DuckDuckGo. This means you can ask ChatGPT questions, get help with tasks, and generate creative content, all without leaving DuckDuckGo. DuckDuckGPT is easy to use. Once you have installed the extension, simply type your question into the DuckDuckGo search bar and hit enter. ChatGPT will then generate a response that will appear below the search results. DuckDuckGPT is a powerful tool that can help you with a wide variety of tasks. Here are just a few examples of what you can use it for: * Get help with research * Write essays and other creative content * Translate languages * Get coding help * Answer trivia questions * And much more! DuckDuckGPT is still in development, but it is already a very powerful tool. As GPT-4 continues to improve, DuckDuckGPT will only get better. So if you are looking for a way to make your DuckDuckGo searches more productive, be sure to give DuckDuckGPT a try.
FastGPT
FastGPT is a knowledge base Q&A system based on the LLM large language model, providing out-of-the-box data processing, model calling and other capabilities. At the same time, you can use Flow to visually arrange workflows to achieve complex Q&A scenarios!
chatgpt-auto-continue
ChatGPT Auto-Continue is a userscript that automatically continues generating ChatGPT responses when chats cut off. It relies on the powerful chatgpt.js library and is easy to install and use. Simply install Tampermonkey and ChatGPT Auto-Continue, and visit chat.openai.com as normal. Multi-reply conversations will automatically continue generating when cut-off!
For similar tasks
Awesome-LLMs-in-Graph-tasks
This repository is a collection of papers on leveraging Large Language Models (LLMs) in Graph Tasks. It provides a comprehensive overview of how LLMs can enhance graph-related tasks by combining them with traditional Graph Neural Networks (GNNs). The integration of LLMs with GNNs allows for capturing both structural and contextual aspects of nodes in graph data, leading to more powerful graph learning. The repository includes summaries of various models that leverage LLMs to assist in graph-related tasks, along with links to papers and code repositories for further exploration.
For similar jobs
sweep
Sweep is an AI junior developer that turns bugs and feature requests into code changes. It automatically handles developer experience improvements like adding type hints and improving test coverage.
teams-ai
The Teams AI Library is a software development kit (SDK) that helps developers create bots that can interact with Teams and Microsoft 365 applications. It is built on top of the Bot Framework SDK and simplifies the process of developing bots that interact with Teams' artificial intelligence capabilities. The SDK is available for JavaScript/TypeScript, .NET, and Python.
ai-guide
This guide is dedicated to Large Language Models (LLMs) that you can run on your home computer. It assumes your PC is a lower-end, non-gaming setup.
classifai
Supercharge WordPress Content Workflows and Engagement with Artificial Intelligence. Tap into leading cloud-based services like OpenAI, Microsoft Azure AI, Google Gemini and IBM Watson to augment your WordPress-powered websites. Publish content faster while improving SEO performance and increasing audience engagement. ClassifAI integrates Artificial Intelligence and Machine Learning technologies to lighten your workload and eliminate tedious tasks, giving you more time to create original content that matters.
chatbot-ui
Chatbot UI is an open-source AI chat app that allows users to create and deploy their own AI chatbots. It is easy to use and can be customized to fit any need. Chatbot UI is perfect for businesses, developers, and anyone who wants to create a chatbot.
BricksLLM
BricksLLM is a cloud native AI gateway written in Go. Currently, it provides native support for OpenAI, Anthropic, Azure OpenAI and vLLM. BricksLLM aims to provide enterprise level infrastructure that can power any LLM production use cases. Here are some use cases for BricksLLM: * Set LLM usage limits for users on different pricing tiers * Track LLM usage on a per user and per organization basis * Block or redact requests containing PIIs * Improve LLM reliability with failovers, retries and caching * Distribute API keys with rate limits and cost limits for internal development/production use cases * Distribute API keys with rate limits and cost limits for students
uAgents
uAgents is a Python library developed by Fetch.ai that allows for the creation of autonomous AI agents. These agents can perform various tasks on a schedule or take action on various events. uAgents are easy to create and manage, and they are connected to a fast-growing network of other uAgents. They are also secure, with cryptographically secured messages and wallets.
griptape
Griptape is a modular Python framework for building AI-powered applications that securely connect to your enterprise data and APIs. It offers developers the ability to maintain control and flexibility at every step. Griptape's core components include Structures (Agents, Pipelines, and Workflows), Tasks, Tools, Memory (Conversation Memory, Task Memory, and Meta Memory), Drivers (Prompt and Embedding Drivers, Vector Store Drivers, Image Generation Drivers, Image Query Drivers, SQL Drivers, Web Scraper Drivers, and Conversation Memory Drivers), Engines (Query Engines, Extraction Engines, Summary Engines, Image Generation Engines, and Image Query Engines), and additional components (Rulesets, Loaders, Artifacts, Chunkers, and Tokenizers). Griptape enables developers to create AI-powered applications with ease and efficiency.