KG-LLM-Papers

[Paper List] Papers integrating knowledge graphs (KGs) and large language models (LLMs)

Stars: 1232

Visit

KG-LLM-Papers is a repository that collects papers integrating knowledge graphs (KGs) and large language models (LLMs). It serves as a comprehensive resource for research on the role of KGs in the era of LLMs, covering surveys, methods, and resources related to this integration.

README:

KG-LLM-Papers

What can LLMs do for KGs? Or, in other words, what role can KG play in the era of LLMs?

🙌 This repository collects papers integrating knowledge graphs (KGs) and large language models (LLMs).

😎 Welcome to recommend missing papers through Adding Issues or Pull Requests.

🔔 News

2024-05 Our paper Knowledgeable Preference Alignment for LLMs in Domain-specific Question Answering has been accepted by ACL 2024. [Repo]
2024-02 We preprint our Survey Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey [Repo].
2023-10 We preprint our paper Making Large Language Models Perform Better in Knowledge Graph Completion and release the [Repo].
2023-06 We create this repository to maintain a paper list on Intergrating Knowledge Graphs and Large Language Models.

Content

📜 Papers

Papers

Surveys

[arxiv] Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey. 2024.02
[arxiv] Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey. 2023.11
[arxiv] Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity. 2023.10
[arxiv] On the Evolution of Knowledge Graphs: A Survey and Perspective. 2023.10
[arxiv] Benchmarking the Abilities of Large Language Models for RDF Knowledge Graph Creation and Comprehension: How Well Do LLMs Speak Turtle? 2023.09
[arxiv] Explainability for Large Language Models: A Survey. 2023.09
[arxiv] Generations of Knowledge Graphs: The Crazy Ideas and the Business Impact. 2023.08
[arxiv] Large Language Models and Knowledge Graphs: Opportunities and Challenges. 2023.08
[TKDE] Unifying Large Language Models and Knowledge Graphs: A Roadmap. 2023.06 [Repo]
[arxiv] ChatGPT is not Enough: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modeling. 2023.06
[arxiv] A Survey of Knowledge-Enhanced Pre-trained Language Models. 2023.05

Method

[arxiv] Chain-of-Knowledge: Integrating Knowledge Reasoning into Large Language Models by Learning from Knowledge Graphs. 2024.07
[arxiv] GraphEval: A Knowledge-Graph Based LLM Hallucination Evaluation Framework. 2024.07
[arxiv] Think-on-Graph 2.0: Deep and Interpretable Large Language Model Reasoning with Knowledge Graph-guided Retrieval. 2024.07
[ISWC 2024] Finetuning Generative Large Language Models with Discrimination Instructions for Knowledge Graph Completion. 2024.07
[ACL 2024 findings] Two-stage Generative Question Answering on Temporal Knowledge Graph Using Large Language Models. 2024.07
[arxiv] Tree-of-Traversals: A Zero-Shot Reasoning Algorithm for Augmenting Black-box Language Models with Knowledge Graphs. 2024.07
[NAACL 2024 findings] GenTKG: Generative Forecasting on Temporal Knowledge Graph with Large Language Models. 2024.06
[ACL 2024 findings] Two-stage Generative Question Answering on Temporal Knowledge Graph Using Large Language Models. 2024.06
[arxiv] Efficient Knowledge Infusion via KG-LLM Alignment. 2024.06
[arxiv] Knowledge Graph Enhanced Retrieval-Augmented Generation for Failure Mode and Effects Analysis. 2024.06
[arxiv] Knowledge Graph-Enhanced Large Language Models via Path Selection. 2024.06
[arxiv] Learning to Plan for Retrieval-Augmented Large Language Models from Knowledge Graphs. 2024.06
[arxiv] Docs2KG: Unified Knowledge Graph Construction from Heterogeneous Documents Assisted by Large Language Models. 2024.06
[arxiv] UniOQA: A Unified Framework for Knowledge Graph Question Answering with Large Language Model. 2024.06
[arxiv] Multimodal Reasoning with Multimodal Knowledge Graph. 2024.06
[arxiv] Knowledge Graph in Astronomical Research with Large Language Models: Quantifying Driving Forces in Interdisciplinary Scientific Discovery. 2024.06
[arxiv] EffiQA: Efficient Question-Answering with Strategic Multi-Model Collaboration on Knowledge Graphs. 2024.06
[arxiv] Explore then Determine: A GNN-LLM Synergy Framework for Reasoning over Knowledge Graph. 2024.06
[arxiv] EMERGE: Integrating RAG for Improved Multimodal EHR Predictive Modeling. 2024.06
[arxiv] DepsRAG: Towards Managing Software Dependencies using Large Language Models. 2024.06 [Repo]
[arxiv] HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models 2024.05 [Repo]
[arxiv] KG-FIT: Knowledge Graph Fine-Tuning Upon Open-World Knowledge. 2024.05
[arxiv] Timeline-based Sentence Decomposition with In-Context Learning for Temporal Fact Extraction. 2024.05
[arxiv] SOK-Bench: A Situated Video Reasoning Benchmark with Aligned Open-World Knowledge. 2024.05
[arxiv] Mitigating Hallucinations in Large Language Models via Self-Refinement-Enhanced Knowledge Retrieval. 2024.05
[arxiv] Prompting Large Language Models with Knowledge Graphs for Question Answering Involving Long-tail Facts. 2024.05
[arxiv] DALK: Dynamic Co-Augmentation of LLMs and KG to answer Alzheimer's Disease Questions with Scientific Literature. 2024.05
[arxiv] BiasKG: Adversarial Knowledge Graphs to Induce Bias in Large Language Models. 2024.05
[arxiv] AttacKG+:Boosting Attack Knowledge Graph Construction with Large Language Models. 2024.05
[arxiv] Sora Detector: A Unified Hallucination Detection for Large Text-to-Video Models. 2024.05
[arxiv] FOKE: A Personalized and Explainable Education Framework Integrating Foundation Models, Knowledge Graphs, and Prompt Engineering. 2024.05
[arxiv] Relations Prediction for Knowledge Graph Completion using Large Language Models. 2024.05
[arxiv] Evaluating Large Language Models for Structured Science Summarization in the Open Research Knowledge Graph. 2024.05
[arxiv] Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuning. 2024.05
[arxiv] RAG-based Explainable Prediction of Road Users Behaviors for Automated Driving using Knowledge Graphs and Large Language Models. 2024.05
[arxiv] PrivComp-KG : Leveraging Knowledge Graph and Large Language Models for Privacy Policy Compliance Verification. 2024.04
[arxiv] Multi-hop Question Answering over Knowledge Graphs using Large Language Models. 2024.04
[arxiv] Automated Construction of Theme-specific Knowledge Graphs. 2024.04
[arxiv] Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answering. 2024.04
[arxiv] Evaluating Class Membership Relations in Knowledge Graphs using Large Language Models. 2024.04
[arxiv] KGValidator: A Framework for Automatic Validation of Knowledge Graph Construction. 2024.04
[arxiv] Context-Enhanced Language Models for Generating Multi-Paper Citations. 2024.04
[arxiv] Reasoning on Efficient Knowledge Paths:Knowledge Graph Guides Large Language Model for Domain Question Answering. 2024.04
[arxiv] KG-CTG: Citation Generation through Knowledge Graph-guided Large Language Models. 2024.04
[arxiv] CuriousLLM: Elevating Multi-Document QA with Reasoning-Infused Knowledge Graph Prompting. 2024.04
[arxiv] ODA: Observation-Driven Agent for integrating LLMs and Knowledge Graphs. 2024.04
[arxiv] Building A Knowledge Graph to Enrich ChatGPT Responses in Manufacturing Service Discovery. 2024.04
[arxiv] Logic Query of Thoughts: Guiding Large Language Models to Answer Complex Logic Queries with Knowledge Graphs. 2024.04
[arxiv] Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction. 2024.04
[arxiv] Unveiling LLMs: The Evolution of Latent Representations in a Temporal Knowledge Graph. 2024.04
[arxiv] Construction of Functional Materials Knowledge Graph in Multidisciplinary Materials Science via Large Language Model. 2024.04
[arxiv] On Linearizing Structured Data in Encoder-Decoder Language Models: Insights from Text-to-SQL. 2024.04
[arxiv] Self-Improvement Programming for Temporal Knowledge Graph Question Answering. 2024.04
[arxiv] A Preliminary Roadmap for LLMs as Assistants in Exploring, Analyzing, and Visualizing Knowledge Graphs. 2024.04
[arxiv] Evaluating the Factuality of Large Language Models using Large-Scale Knowledge Graphs. 2024.04
[arxiv] Harnessing the Power of Large Language Model for Uncertainty Aware Graph Processing. 2024.04
[arxiv] EventGround: Narrative Reasoning by Grounding to Eventuality-centric Knowledge Graphs. 2024.04
[arxiv] Generate-on-Graph: Treat LLM as both Agent and KG in Incomplete Knowledge Graph Question Answering. 2024.04
[arxiv] From Local to Global: A Graph RAG Approach to Query-Focused Summarization. 2024.04
[arxiv] HyKGE: A Hypothesis Knowledge Graph Enhanced Framework for Accurate and Reliable Medical LLMs Responses. 2024.04
[arxiv] Evaluating the Factuality of Large Language Models using Large-Scale Knowledge Graphs. 2024.04
[arxiv] Evaluating the Factuality of Large Language Models using Large-Scale Knowledge Graphs. 2024.04
[arxiv] KnowLA: Enhancing Parameter-efficient Finetuning with Knowledgeable Adaptation. 2024.03
[LREC-COLING 2024] KC-GenRe: A Knowledge-constrained Generative Re-ranking Method Based on Large Language Models for Knowledge Graph Completion. 2024.03
[arxiv] K-Act2Emo: Korean Commonsense Knowledge Graph for Indirect Emotional Expression. 2024.03
[arxiv] Fusing Domain-Specific Content from Large Language Models into Knowledge Graphs for Enhanced Zero Shot Object State Classification. 2024.03
[arxiv] Construction of Hyper-Relational Knowledge Graphs Using Pre-Trained Large Language Models. 2024.03
[arxiv] Call Me When Necessary: LLMs can Efficiently and Faithfully Reason over Structured Environments. 2024.03
[arxiv] From human experts to machines: An LLM supported approach to ontology and knowledge graph construction. 2024.03
[arxiv] Complex Reasoning over Logical Queries on Commonsense Knowledge Graphs. 2024.03
[arxiv] Knowledge Graph Large Language Model (KG-LLM) for Link Prediction. 2024.03
[arxiv] KG-Rank: Enhancing Large Language Models for Medical QA with Knowledge Graphs and Ranking Techniques. 2024.03
[arxiv] Advancing Biomedical Text Mining with Community Challenges. 2024.03
[arxiv] Knowledge Graphs as Context Sources for LLM-Based Explanations of Learning Recommendations. 2024.03
[arxiv] Evidence-Focused Fact Summarization for Knowledge-Augmented Zero-Shot Question Answering. 2024.03
[arxiv] AceMap: Knowledge Discovery through Academic Graph. 2024.03
[arxiv] KnowPhish: Large Language Models Meet Multimodal Knowledge Graphs for Enhancing Reference-Based Phishing Detection. 2024.03
[arxiv] Unveiling Hidden Links Between Unseen Security Entities. 2024.03
[LREC-COLING 2024] Multi-perspective Improvement of Knowledge Graph Completion with Large Language Models. 2024.03
[arxiv] Infusing Knowledge into Large Language Models with Contextual Prompts. 2024.03
[arxiv] CR-LT-KGQA: A Knowledge Graph Question Answering Dataset Requiring Commonsense Reasoning and Long-Tail Knowledge. 2024.03
[arxiv] Right for Right Reasons: Large Language Models for Verifiable Commonsense Knowledge Graph Question Answering. 2024.03
[arxiv] Automatic Question-Answer Generation for Long-Tail Knowledge. 2024.03
[arxiv] AutoRD: An Automatic and End-to-End System for Rare Disease Knowledge Graph Construction Based on Ontologies-enhanced Large Language Models. 2024.03
[arxiv] Stepwise Self-Consistent Mathematical Reasoning with Large Language Models. 2024.02
[arxiv] Two-stage Generative Question Answering on Temporal Knowledge Graph Using Large Language Models. 2024.02
[arxiv] Unlocking the Power of Large Language Models for Entity Alignment. 2024.02
[arxiv] Enhancing Temporal Knowledge Graph Forecasting with Large Language Models via Chain-of-History Reasoning. 2024.02
[arxiv] Breaking the Barrier: Utilizing Large Language Models for Industrial Recommendation Systems through an Inferential Knowledge Graph. 2024.02
[arxiv] Knowledge Graph Enhanced Large Language Model Editing. 2024.02
[arxiv] Modality-Aware Integration with Large Language Models for Knowledge-based Visual Question Answering. 2024.02
[arxiv] Graph-Based Retriever Captures the Long Tail of Biomedical Knowledge. 2024.02
[arxiv] LLM as Prompter: Low-resource Inductive Reasoning on Arbitrary Knowledge Graphs. 2024.02
[arxiv] Counter-intuitive: Large Language Models Can Better Understand Knowledge Graphs Than We Thought. 2024.02
[arxiv] InfuserKI: Enhancing Large Language Models with Knowledge Graphs via Infuser-Guided Knowledge Integration. 2024.02
[arxiv] Towards Development of Automated Knowledge Maps and Databases for Materials Engineering using Large Language Models. 2024.02
[arxiv] KG-Agent: An Efficient Autonomous Agent Framework for Complex Reasoning over Knowledge Graph. 2024.02
[arxiv] PAT-Questions: A Self-Updating Benchmark for Present-Anchored Temporal Question-Answering. 2024.02
[arxiv] A Condensed Transition Graph Framework for Zero-shot Link Prediction with Large Language Models. 2024.02
[arxiv] Enhancing Large Language Models with Pseudo- and Multisource- Knowledge Graphs for Open-ended Question Answering. 2024.02
[arxiv] G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering. 2024.02
[arxiv] X-LoRA: Mixture of Low-Rank Adapter Experts, a Flexible Framework for Large Language Models with Applications in Protein Mechanics and Design. 2024.02
[arxiv] REALM: RAG-Driven Enhancement of Multimodal Electronic Health Records Analysis via Large Language Models. 2024.02
[arxiv] UrbanKGent: A Unified Large Language Model Agent Framework for Urban Knowledge Graph Construction. 2024.02
[arxiv] GLaM: Fine-Tuning Large Language Models for Domain Knowledge Graph Alignment via Neighborhood Partitioning and Generative Subgraph Encoding. 2024.02
[arxiv] Let Your Graph Do the Talking: Encoding Structured Data for LLMs. 2024.02
[arxiv] CADReN: Contextual Anchor-Driven Relational Network for Controllable Cross-Graphs Node Importance Estimation. 2024.02
[arxiv] An Enhanced Prompt-Based LLM Reasoning Scheme via Knowledge Graph-Integrated Collaboration. 2024.02
[arxiv] SPARQL Generation: an analysis on fine-tuning OpenLLaMA for Question Answering over a Life Science Knowledge Graph. 2024.02
[arxiv] Interplay of Semantic Communication and Knowledge Learning. 2024.02
[arxiv] GUARD: Role-playing to Generate Natural-language Jailbreakings to Test Guideline Adherence of Large Language Models. 2024.02
[arxiv] Rendering Graphs for Graph Reasoning in Multimodal Large Language Models. 2024.02
[arxiv] Evaluating LLM -- Generated Multimodal Diagnosis from Medical Images and Symptom Analysis. 2024.02
[EACL 2024] Contextualization Distillation from Large Language Model for Knowledge Graph Completion. 2024.02
[EACL 2024] A Comparative Analysis of Conversational Large Language Models in Knowledge-Based Text Generation. 2024.02
[arxiv] Prompt-Time Symbolic Knowledge Capture with Large Language Models. 2024.02
[arxiv] Effective Bug Detection in Graph Database Engines: An LLM-based Approach. 2024.02
[arxiv] Two Heads Are Better Than One: Integrating Knowledge from Knowledge Graphs and Large Language Models for Entity Alignment. 2024.01
[arxiv] Benchmarking Large Language Models in Complex Question Answering Attribution using Knowledge Graphs. 2024.01
[arxiv] Clue-Guided Path Exploration: An Efficient Knowledge Base Question-Answering Framework with Low Computational Resource Consumption. 2024.01
[AAAI 2024] KAM-CoT: Knowledge Augmented Multimodal Chain-of-Thoughts Reasoning. 2024.01
[arxiv] Context Matters: Pushing the Boundaries of Open-Ended Answer Generation with Graph-Structured Knowledge Context. 2024.01
[arxiv] Supporting Student Decisions on Learning Recommendations: An LLM-Based Chatbot with Knowledge Graph Contextualization for Conversational Explainability and Mentoring. 2024.01
[arxiv] Distilling Event Sequence Knowledge From Large Language Models. 2024.01
[ACL 24] Large Language Models Can Learn Temporal Reasoning. 2024.01 [Repo]
[arxiv] Chain of History: Learning and Forecasting with LLMs for Temporal Knowledge Graph Completion. 2024.01
[arxiv] TechGPT-2.0: A large language model project to solve the task of knowledge graph construction. 2024.01 [Repo]
[arxiv] Evaluating Large Language Models in Semantic Parsing for Conversational Question Answering over Knowledge Graphs. 2024.01
[arxiv] The Earth is Flat? Unveiling Factual Errors in Large Language Models. 2024.01
[arxiv] keqing: knowledge-based question answering is a nature chain-of-thought mentor of LLM. 2024.01
[arxiv] Quartet Logic: A Four-Step Reasoning (QLFR) framework for advancing Short Text Classification. 2024.01
[arxiv] Conversational Question Answering with Reformulations over Knowledge Graph. 2023.12
[arxiv] Think and Retrieval: A Hypothesis Knowledge Graph Enhanced Medical Large Language Models. 2023.12
[arxiv] KnowledgeNavigator: Leveraging Large Language Models for Enhanced Reasoning over Knowledge Graph. 2023.12
[arxiv] Urban Generative Intelligence (UGI): A Foundational Platform for Agents in Embodied City Environment. 2023.12
[arxiv] Zero-Shot Fact-Checking with Semantic Triples and Knowledge Graphs. 2023.12
[arxiv] KGLens: A Parameterized Knowledge Graph Solution to Assess What an LLM Does and Doesn't Know. 2023.12
[arxiv] LLM-ARK: Knowledge Graph Reasoning Using Large Language Models via Deep Reinforcement Learning. 2023.12
[arxiv] Towards Trustworthy AI Software Development Assistance. 2023.12
[arxiv] KnowGPT: Black-Box Knowledge Injection for Large Language Models. 2023.12
[arxiv] Making Large Language Models Better Knowledge Miners for Online Marketing with Progressive Prompting Augmentation. 2023.12
[arxiv] Conceptual Engineering Using Large Language Models. 2023.12
[arxiv] Beyond Isolation: Multi-Agent Synergy for Improving Knowledge Graph Construction. 2023.12
[arxiv] Zero- and Few-Shots Knowledge Graph Triplet Extraction with Large Language Models. 2023.12
[arxiv] On Exploring the Reasoning Capability of Large Language Models with Knowledge Graphs. 2023.11
[arxiv] Biomedical knowledge graph-optimized prompt generation for large language models. 2023.11 [Repo]
[arxiv] A Graph-to-Text Approach to Knowledge-Grounded Response Generation in Human-Robot Interaction. 2023.11
[EMNLP 2023]Revisiting the Knowledge Injection Frameworks. 2023.12
[EMNLP 2023]Does the Correctness of Factual Knowledge Matter for Factual Knowledge-Enhanced Pre-trained Language Models? 2023.12
[EMNLP 2023]ReasoningLM: Enabling Structural Subgraph Reasoning in Pre-trained Language Models for Question Answering over Knowledge Graph. 2023.12
[EMNLP 2023 Findings]KICGPT: Large Language Model with Knowledge in Context for Knowledge Graph Completion. 2023.12
[arxiv] $R^3$-NL2GQL: A Hybrid Models Approach for for Accuracy Enhancing and Hallucinations Mitigation. 2023.11
[arxiv] Mitigating Large Language Model Hallucinations via Autonomous Knowledge Graph-based Retrofitting. 2023.11
[EMNLP 2023] Fine-tuned LLMs Know More, Hallucinate Less with Few-Shot Sequence-to-Sequence Semantic Parsing over Wikidata. 2023.11
[arxiv] Leveraging LLMs in Scholarly Knowledge Graph Question Answering. 2023.11
[arxiv] Knowledgeable Preference Alignment for LLMs in Domain-specific Question Answering. 2023.11
[arxiv] OLaLa: Ontology Matching with Large Language Models. 2023.11
[arxiv] In-Context Learning for Knowledge Base Question Answering for Unmanned Systems based on Large Language Models. 2023.11
[arxiv] Let's Discover More API Relations: A Large Language Model-based AI Chain for Unsupervised API Relation Inference. 2023.11
[arxiv] Form follows Function: Text-to-Text Conditional Graph Generation based on Functional Requirements. 2023.11
[arxiv] Large Language Models Meet Knowledge Graphs to Answer Factoid Questions. 2023.10
[arxiv] Answer Candidate Type Selection: Text-to-Text Language Model for Closed Book Question Answering Meets Knowledge Graphs. 2023.10
[arxiv] Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models. 2023.10
[arxiv] DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text. 2023.10
[arxiv] Generative retrieval-augmented ontologic graph and multi-agent strategies for interpretive large language model-based materials design. 2023.10
[arxiv] A Multimodal Ecological Civilization Pattern Recommendation Method Based on Large Language Models and Knowledge Graph. 2023.10
[arxiv] LoRAShear: Efficient Large Language Model Structured Pruning and Knowledge Recovery. 2023.10
[arxiv] Graph Agent: Explicit Reasoning Agent for Graphs. 2023.10
[arxiv] An In-Context Schema Understanding Method for Knowledge Base Question Answering. 2023.10
[arxiv] GraphGPT: Graph Instruction Tuning for Large Language Models. 2023.10
[EMNLP 2023 Findings] Systematic Assessment of Factual Knowledge in Large Language Models. 2023.10
[EMNLP 2023 Findings] KG-GPT: A General Framework for Reasoning on Knowledge Graphs Using Large Language Models. 2023.10
[arxiv] MechGPT, a language-based strategy for mechanics and materials modeling that connects knowledge across scales, disciplines and modalities. 2023.10
[arxiv] Qilin-Med: Multi-stage Knowledge Injection Advanced Medical Large Language Model. 2023.10
[arxiv] ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models. 2023.10
[arxiv] From Large Language Models to Knowledge Graphs for Biomarker Discovery in Cancer. 2023.10
[arxiv] Making Large Language Models Perform Better in Knowledge Graph Completion. 2023.10
[arxiv] CP-KGC: Constrained-Prompt Knowledge Graph Completion with Large Language Models. 2023.10
[arxiv] PHALM: Building a Knowledge Graph from Scratch by Prompting Humans and a Language Model. 2023.10
[arxiv] InstructProtein: Aligning Human and Protein Language via Knowledge Instruction. 2023.10
[arxiv] Large Language Models Meet Knowledge Graphs to Answer Factoid Questions. 2023.10
[arxiv] Knowledge Crosswords: Geometric Reasoning over Structured Knowledge with Large Language Models. 2023.10
[ICLR 2024] Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning. 2023.10 [Repo]
[arxiv] RelBERT: Embedding Relations with Language Models. 2023.10
[arxiv] Benchmarking the Abilities of Large Language Models for RDF Knowledge Graph Creation and Comprehension: How Well Do LLMs Speak Turtle?. 2023.09
[arxiv] Let's Chat to Find the APIs: Connecting Human, LLM and Knowledge Graph through AI Chain. 2023.09
[arxiv] Graph Neural Prompting with Large Language Models. 2023.09
[arxiv] A knowledge representation approach for construction contract knowledge modeling. 2023.09
[arxiv] Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answering. 2023.09
[arxiv] "Merge Conflicts!" Exploring the Impacts of External Distractors to Parametric Knowledge Graphs. 2023.09
[arxiv] FactLLaMA: Optimizing Instruction-Following Language Models with External Knowledge for Automated Fact-Checking. 2023.09
[arxiv] ChatRule: Mining Logical Rules with Large Language Models for Knowledge Graph Reasoning. 2023.09
[AAAI 2024] Code-Style In-Context Learning for Knowledge-Based Question Answering. 2023.09
[arxiv] Unleashing the Power of Graph Learning through LLM-based Autonomous Agents. 2023.09
[arxiv] Knowledge-tuning Large Language Models with Structured Medical Knowledge Bases for Reliable Response Generation in Chinese. 2023.09
[arxiv] Knowledge Solver: Teaching LLMs to Search for Domain Knowledge from Knowledge Graphs. 2023.09
[arxiv] Biomedical Entity Linking with Triple-aware Pre-Training. 2023.08
[arxiv] Exploring Large Language Models for Knowledge Graph Completion. 2023.08 [Repo]
[arxiv] Developing a Scalable Benchmark for Assessing Large Language Models in Knowledge Graph Engineering. 2023.08
[arxiv] Leveraging A Medical Knowledge Graph into Large Language Models for Diagnosis Prediction. 2023.08
[arxiv] LKPNR: LLM and KG for Personalized News Recommendation Framework. 2023.08
[arxiv] Knowledge Graph Prompting for Multi-Document Question Answering. 2023.08
[arxiv] Head-to-Tail: How Knowledgeable are Large Language Models (LLM)? A.K.A. Will LLMs Replace Knowledge Graphs?. 2023.08
[arxiv] MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Models. 2023.08
[arxiv] Towards Semantically Enriched Embeddings for Knowledge Graph Completion. 2023.07
[TKDE 2024] AutoAlign: Fully Automatic and Effective Knowledge Graph Alignment enabled by Large Language Models. 2023.07
[arxiv] Using Large Language Models for Zero-Shot Natural Language Generation from Knowledge Graphs. 2023.07
[ICLR 2024] Think-on-Graph: Deep and Responsible Reasoning of Large Language Model with Knowledge Graph. 2023.07
[SIGKDD 2024 Explorations] Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs. 2023.07
[arxiv] Exploring Large Language Model for Graph Data Understanding in Online Job Recommendations. 2023.07
[arxiv] RecallM: An Architecture for Temporal Context Understanding and Question Answering. 2023.07
[arxiv] LLM-assisted Knowledge Graph Engineering: Experiments with ChatGPT. 2023.07
[arxiv] Iterative Zero-Shot LLM Prompting for Knowledge Graph Construction. 2023.07
[arxiv] Snowman: A Million-scale Chinese Commonsense Knowledge Graph Distilled from Foundation Model . 2023.06
[arxiv] Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering. 2023.06
[arxiv] Fine-tuning Large Enterprise Language Models via Ontological Reasoning. 2023.06
[NeurIPS 2023] Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks. 2023.05
[arxiv] Enhancing Knowledge Graph Construction Using Large Language Models. 2023.05
[arxiv] ChatGraph: Interpretable Text Classification by Converting ChatGPT Knowledge to Graphs. 2023.05
[ACL 2023] FactKG: Fact Verification via Reasoning on Knowledge Graphs. 2023.05
[arxiv] HiPrompt: Few-Shot Biomedical Knowledge Fusion via Hierarchy-Oriented Prompting. 2023.04
[EMNLP 2023] StructGPT: A General Framework for Large Language Model to Reason over Structured Data. 2023.05
[ICLR 2024] Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning. 2023.05
[arxiv] LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities. 2023.05 [Repo]
[NeurIPS 2023] Can Language Models Solve Graph Problems in Natural Language? 2023.05
[arxiv] Knowledge Graph Completion Models are Few-shot Learners: An Empirical Study of Relation Labeling in E-commerce with LLMs. 2023.05
[arxiv] Can large language models generate salient negative statements? 2023.05
[arxiv] GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarking. 2023.05
[arxiv] Complex Logical Reasoning over Knowledge Graphs using Large Language Models. 2023.05 [Repo]
[arxiv] Causal Reasoning and Large Language Models: Opening a New Frontier for Causality. 2023.04
[arxiv] Can large language models build causal graphs? 2023.04
[arxiv] Using Multiple RDF Knowledge Graphs for Enriching ChatGPT Responses. 2023.04
[arxiv] Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPT. 2023.04
[arxiv] Structured prompt interrogation and recursive extraction of semantics (SPIRES): A method for populating knowledge bases using zero-shot learning. 2023.04 [Repo]

Resources and Benchmarking

[arxiv] STaRK: Benchmarking LLM Retrieval on Textual and Relational Knowledge Bases. 2024.04 [Repo]
[arxiv] RareBench: Can LLMs Serve as Rare Diseases Specialists?. 2024.02
[arxiv] Benchmarking Large Language Models in Complex Question Answering Attribution using Knowledge Graphs. 2024.01
[ACL 24] Large Language Models Can Learn Temporal Reasoning. 2024.01 [Repo]
[arxiv] AbsPyramid: Benchmarking the Abstraction Ability of Language Models with a Unified Entailment Graph. 2023.11
[arxiv] A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model's Accuracy for Question Answering on Enterprise SQL Databases. 2023.11
[arxiv] Towards Verifiable Generation: A Benchmark for Knowledge-aware Language Model Attribution. 2023.10
[EMNLP 2023] MarkQA: A large scale KBQA dataset with numerical reasoning. 2023.10
[CIKM 2023] Ontology Enrichment from Texts: A Biomedical Dataset for Concept Discovery and Placement. 2023.06
[arxiv] Xiezhi: An Ever-Updating Benchmark for Holistic Domain Knowledge Evaluation. 2023.06
[AACL 2023 System Demonstrations] LambdaKG: A Library for Pre-trained Language Model-Based Knowledge Graph Embeddings 2023.03 [Repo]
[arxiv] Construction of Paired Knowledge Graph-Text Datasets Informed by Cyclic Evaluation. 2023.09
[arxiv] From Large Language Models to Knowledge Graphs for Biomarker Discovery in Cancer. 2023.10
[ISWC 2023] Text2KGBench: A Benchmark for Ontology-Driven Knowledge Graph Generation from Text. 2023.08

Contribution

👥 Contributors

🎉 Contributing ( welcome ! )

✨ Add a new paper or update an existing KG-related LLM paper.
🧐 Use the same format as existing entries to describe the work.
😄 A very brief explanation why you think a paper should be added or updated is recommended (Not Neccessary) via Adding Issues or Pull Requests.

Don't worry if you put something wrong, they will be fixed for you. Just feel free to contribute and promote your awesome work here! 🤩 We'll get back to you in time ~ 😉

🤝 Cite:

If this Repo is helpful to you, please consider citing our paper. We would greatly appreciate it :)

@article{DBLP:journals/corr/abs-2311-06503,
  author       = {Yichi Zhang and
                  Zhuo Chen and
                  Yin Fang and
                  Lei Cheng and
                  Yanxi Lu and
                  Fangming Li and
                  Wen Zhang and
                  Huajun Chen},
  title        = {Knowledgeable Preference Alignment for LLMs in Domain-specific Question
                  Answering},
  journal      = {CoRR},
  volume       = {abs/2311.06503},
  year         = {2023}
}

@article{DBLP:journals/corr/abs-2310-06671,
  author       = {Yichi Zhang and
                  Zhuo Chen and
                  Wen Zhang and
                  Huajun Chen},
  title        = {Making Large Language Models Perform Better in Knowledge Graph Completion},
  journal      = {CoRR},
  volume       = {abs/2310.06671},
  year         = {2023}
}

For Tasks:

Click tags to check more tools for each tasks

analyze knowledge graphs improve llm performance conduct surveys enhance question answering explore graph reasoning

For Jobs:

research scientist data scientist machine learning engineer ai researcher knowledge engineer

Alternative AI tools for KG-LLM-Papers

Similar Open Source Tools

KG-LLM-Papers

github

: 1.2k

openlrc

Open-Lyrics is a Python library that transcribes voice files using faster-whisper and translates/polishes the resulting text into `.lrc` files in the desired language using LLM, e.g. OpenAI-GPT, Anthropic-Claude. It offers well preprocessed audio to reduce hallucination and context-aware translation to improve translation quality. Users can install the library from PyPI or GitHub and follow the installation steps to set up the environment. The tool supports GUI usage and provides Python code examples for transcription and translation tasks. It also includes features like utilizing context and glossary for translation enhancement, pricing information for different models, and a list of todo tasks for future improvements.

github

: 381

ai00_server

AI00 RWKV Server is an inference API server for the RWKV language model based upon the web-rwkv inference engine. It supports VULKAN parallel and concurrent batched inference and can run on all GPUs that support VULKAN. No need for Nvidia cards!!! AMD cards and even integrated graphics can be accelerated!!! No need for bulky pytorch, CUDA and other runtime environments, it's compact and ready to use out of the box! Compatible with OpenAI's ChatGPT API interface. 100% open source and commercially usable, under the MIT license. If you are looking for a fast, efficient, and easy-to-use LLM API server, then AI00 RWKV Server is your best choice. It can be used for various tasks, including chatbots, text generation, translation, and Q&A.

github

: 452

LLMTSCS

LLMLight is a novel framework that employs Large Language Models (LLMs) as decision-making agents for Traffic Signal Control (TSC). The framework leverages the advanced generalization capabilities of LLMs to engage in a reasoning and decision-making process akin to human intuition for effective traffic control. LLMLight has been demonstrated to be remarkably effective, generalizable, and interpretable against various transportation-based and RL-based baselines on nine real-world and synthetic datasets.

github

: 158

mergoo

Mergoo is a library for easily merging multiple LLM experts and efficiently training the merged LLM. With Mergoo, you can efficiently integrate the knowledge of different generic or domain-based LLM experts. Mergoo supports several merging methods, including Mixture-of-Experts, Mixture-of-Adapters, and Layer-wise merging. It also supports various base models, including LLaMa, Mistral, and BERT, and trainers, including Hugging Face Trainer, SFTrainer, and PEFT. Mergoo provides flexible merging for each layer and supports training choices such as only routing MoE layers or fully fine-tuning the merged LLM.

github

: 329

llama-api-server

This project aims to create a RESTful API server compatible with the OpenAI API using open-source backends like llama/llama2. With this project, various GPT tools/frameworks can be compatible with your own model. Key features include: - **Compatibility with OpenAI API**: The API server follows the OpenAI API structure, allowing seamless integration with existing tools and frameworks. - **Support for Multiple Backends**: The server supports both llama.cpp and pyllama backends, providing flexibility in model selection. - **Customization Options**: Users can configure model parameters such as temperature, top_p, and top_k to fine-tune the model's behavior. - **Batch Processing**: The API supports batch processing for embeddings, enabling efficient handling of multiple inputs. - **Token Authentication**: The server utilizes token authentication to secure access to the API. This tool is particularly useful for developers and researchers who want to integrate large language models into their applications or explore custom models without relying on proprietary APIs.

github

: 176

morgana-form

MorGana Form is a full-stack form builder project developed using Next.js, React, TypeScript, Ant Design, PostgreSQL, and other technologies. It allows users to quickly create and collect data through survey forms. The project structure includes components, hooks, utilities, pages, constants, Redux store, themes, types, server-side code, and component packages. Environment variables are required for database settings, NextAuth login configuration, and file upload services. Additionally, the project integrates an AI model for form generation using the Ali Qianwen model API.

github

: 204

lobe-chat-agents

github

: 443

chrome-ai

Chrome AI is a Vercel AI provider for Chrome's built-in model (Gemini Nano). It allows users to create language models using Chrome's AI capabilities. The tool is under development and may contain errors and frequent changes. Users can install the ChromeAI provider module and use it to generate text, stream text, and generate objects. To enable AI in Chrome, users need to have Chrome version 127 or greater and turn on specific flags. The tool is designed for developers and researchers interested in experimenting with Chrome's built-in AI features.

github

: 245

langcorn

LangCorn is an API server that enables you to serve LangChain models and pipelines with ease, leveraging the power of FastAPI for a robust and efficient experience. It offers features such as easy deployment of LangChain models and pipelines, ready-to-use authentication functionality, high-performance FastAPI framework for serving requests, scalability and robustness for language processing applications, support for custom pipelines and processing, well-documented RESTful API endpoints, and asynchronous processing for faster response times.

github

: 821

scrape-it-now

Scrape It Now is a versatile tool for scraping websites with features like decoupled architecture, CLI functionality, idempotent operations, and content storage options. The tool includes a scraper component for efficient scraping, ad blocking, link detection, markdown extraction, dynamic content loading, and anonymity features. It also offers an indexer component for creating AI search indexes, chunking content, embedding chunks, and enabling semantic search. The tool supports various configurations for Azure services and local storage, providing flexibility and scalability for web scraping and indexing tasks.

github

: 452

LLM-Tuning

LLM-Tuning is a collection of tools and resources for fine-tuning large language models (LLMs). It includes a library of pre-trained LoRA models, a set of tutorials and examples, and a community forum for discussion and support. LLM-Tuning makes it easy to fine-tune LLMs for a variety of tasks, including text classification, question answering, and dialogue generation. With LLM-Tuning, you can quickly and easily improve the performance of your LLMs on downstream tasks.

github

: 897

LongLLaVA

LongLLaVA is a tool for scaling multi-modal LLMs to 1000 images efficiently via hybrid architecture. It includes stages for single-image alignment, instruction-tuning, and multi-image instruction-tuning, with evaluation through a command line interface and model inference. The tool aims to achieve GPT-4V level capabilities and beyond, providing reproducibility of results and benchmarks for efficiency and performance.

github

: 158

ollama4j

Ollama4j is a Java library that serves as a wrapper or binding for the Ollama server. It facilitates communication with the Ollama server and provides models for deployment. The tool requires Java 11 or higher and can be installed locally or via Docker. Users can integrate Ollama4j into Maven projects by adding the specified dependency. The tool offers API specifications and supports various development tasks such as building, running unit tests, and integration tests. Releases are automated through GitHub Actions CI workflow. Areas of improvement include adhering to Java naming conventions, updating deprecated code, implementing logging, using lombok, and enhancing request body creation. Contributions to the project are encouraged, whether reporting bugs, suggesting enhancements, or contributing code.

github

: 162

langcheck

LangCheck is a Python library that provides a suite of metrics and tools for evaluating the quality of text generated by large language models (LLMs). It includes metrics for evaluating text fluency, sentiment, toxicity, factual consistency, and more. LangCheck also provides tools for visualizing metrics, augmenting data, and writing unit tests for LLM applications. With LangCheck, you can quickly and easily assess the quality of LLM-generated text and identify areas for improvement.

github

: 184

BricksLLM

BricksLLM is a cloud native AI gateway written in Go. Currently, it provides native support for OpenAI, Anthropic, Azure OpenAI and vLLM. BricksLLM aims to provide enterprise level infrastructure that can power any LLM production use cases. Here are some use cases for BricksLLM: * Set LLM usage limits for users on different pricing tiers * Track LLM usage on a per user and per organization basis * Block or redact requests containing PIIs * Improve LLM reliability with failovers, retries and caching * Distribute API keys with rate limits and cost limits for internal development/production use cases * Distribute API keys with rate limits and cost limits for students

github

: 897

For similar tasks

KG-LLM-Papers

github

: 1.2k

For similar jobs

weave

Weave is a toolkit for developing Generative AI applications, built by Weights & Biases. With Weave, you can log and debug language model inputs, outputs, and traces; build rigorous, apples-to-apples evaluations for language model use cases; and organize all the information generated across the LLM workflow, from experimentation to evaluations to production. Weave aims to bring rigor, best-practices, and composability to the inherently experimental process of developing Generative AI software, without introducing cognitive overhead.

github

: 685

LLMStack

LLMStack is a no-code platform for building generative AI agents, workflows, and chatbots. It allows users to connect their own data, internal tools, and GPT-powered models without any coding experience. LLMStack can be deployed to the cloud or on-premise and can be accessed via HTTP API or triggered from Slack or Discord.

github

: 1.5k

VisionCraft

The VisionCraft API is a free API for using over 100 different AI models. From images to sound.

github

: 94

kaito

Kaito is an operator that automates the AI/ML inference model deployment in a Kubernetes cluster. It manages large model files using container images, avoids tuning deployment parameters to fit GPU hardware by providing preset configurations, auto-provisions GPU nodes based on model requirements, and hosts large model images in the public Microsoft Container Registry (MCR) if the license allows. Using Kaito, the workflow of onboarding large AI inference models in Kubernetes is largely simplified.

github

: 405

PyRIT

PyRIT is an open access automation framework designed to empower security professionals and ML engineers to red team foundation models and their applications. It automates AI Red Teaming tasks to allow operators to focus on more complicated and time-consuming tasks and can also identify security harms such as misuse (e.g., malware generation, jailbreaking), and privacy harms (e.g., identity theft). The goal is to allow researchers to have a baseline of how well their model and entire inference pipeline is doing against different harm categories and to be able to compare that baseline to future iterations of their model. This allows them to have empirical data on how well their model is doing today, and detect any degradation of performance based on future improvements.

github

: 1.8k

tabby

Tabby is a self-hosted AI coding assistant, offering an open-source and on-premises alternative to GitHub Copilot. It boasts several key features: * Self-contained, with no need for a DBMS or cloud service. * OpenAPI interface, easy to integrate with existing infrastructure (e.g Cloud IDE). * Supports consumer-grade GPUs.

github

: 21.5k

spear

SPEAR (Simulator for Photorealistic Embodied AI Research) is a powerful tool for training embodied agents. It features 300 unique virtual indoor environments with 2,566 unique rooms and 17,234 unique objects that can be manipulated individually. Each environment is designed by a professional artist and features detailed geometry, photorealistic materials, and a unique floor plan and object layout. SPEAR is implemented as Unreal Engine assets and provides an OpenAI Gym interface for interacting with the environments via Python.

github

: 224

Magick

Magick is a groundbreaking visual AIDE (Artificial Intelligence Development Environment) for no-code data pipelines and multimodal agents. Magick can connect to other services and comes with nodes and templates well-suited for intelligent agents, chatbots, complex reasoning systems and realistic characters.

github

: 675