Awesome-Graph-LLM
A collection of AWESOME things about Graph-Related LLMs.
Stars: 1700
Awesome-Graph-LLM is a curated collection of research papers exploring the intersection of graph-based techniques with Large Language Models (LLMs). The repository aims to bridge the gap between LLMs and graph structures prevalent in real-world applications by providing a comprehensive list of papers covering various aspects of graph reasoning, node classification, graph classification/regression, knowledge graphs, multimodal models, applications, and tools. It serves as a valuable resource for researchers and practitioners interested in leveraging LLMs for graph-related tasks.
README:
A collection of AWESOME things about Graph-Related Large Language Models (LLMs).
Large Language Models (LLMs) have shown remarkable progress in natural language processing tasks. However, their integration with graph structures, which are prevalent in real-world applications, remains relatively unexplored. This repository aims to bridge that gap by providing a curated list of research papers that explore the intersection of graph-based techniques with LLMs.
- (NAACL'21) Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training [paper][code]
- (NeurIPS'23) Can Language Models Solve Graph Problems in Natural Language? [paper][code]
- (IEEE Intelligent Systems 2023) Integrating Graphs with Large Language Models: Methods and Prospects [paper]
- (ICLR'24) Talk like a Graph: Encoding Graphs for Large Language Models [paper]
- (KDD'24) LLM4DyG: Can Large Language Models Solve Problems on Dynamic Graphs? [paper][code]
- (arXiv 2023.05) GPT4Graph: Can Large Language Models Understand Graph Structured Data? An Empirical Evaluation and Benchmarking [paper][code]
- (arXiv 2023.08) Graph Meets LLMs: Towards Large Graph Models [paper]
- (arXiv 2023.10) Towards Graph Foundation Models: A Survey and Beyond [paper]
- (arXiv 2023.11) Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey [paper]
- (arXiv 2023.11) A Survey of Graph Meets Large Language Model: Progress and Future Directions [paper][code]
- (arXiv 2023.12) Large Language Models on Graphs: A Comprehensive Survey [paper][code]
- (arXiv 2024.02) Towards Versatile Graph Learning Approach: from the Perspective of Large Language Models [paper]
- (arXiv 2024.04) Graph Machine Learning in the Era of Large Language Models (LLMs) [paper]
- (arXiv 2024.05) A Survey of Large Language Models for Graphs [paper][code]
- (arXiv 2024.07) GLBench: A Comprehensive Benchmark for Graph with Large Language Models [paper][code]
- (arXiv 2024.07) Learning on Graphs with Large Language Models(LLMs): A Deep Dive into Model Robustness [paper][code]
- (arXiv 2024.09) LLMs hallucinate graphs too: a structural perspective [paper]
- (EMNLP'23) StructGPT: A General Framework for Large Language Model to Reason over Structured Data [paper][code]
- (AAAI'24) Graph of Thoughts: Solving Elaborate Problems with Large Language Models [paper][code]
- (arXiv 2023.05) PiVe: Prompting with Iterative Verification Improving Graph-based Generative Capability of LLMs [paper][code]
- (arXiv 2023.08) Boosting Logical Reasoning in Large Language Models through a New Framework: The Graph of Thought [paper]
- (arxiv 2023.10) Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Models [paper]
- (arxiv 2024.01) Topologies of Reasoning: Demystifying Chains, Trees, and Graphs of Thoughts [paper]
- (arxiv 2024.04) Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs [paper)
- (ICLR'24) One for All: Towards Training One Graph Model for All Classification Tasks [paper][code]
- (WWW'24) GraphTranslator: Aligning Graph Model to Large Language Model for Open-ended Tasks [paper][code]
- (arXiv 2023.08) Natural Language is All a Graph Needs [paper][code]
- (arXiv 2023.10) GraphGPT: Graph Instruction Tuning for Large Language Models [paper][code][blog in Chinese]
- (arXiv 2023.10) Graph Agent: Explicit Reasoning Agent for Graphs [paper]
- (arXiv 2024.02) Let Your Graph Do the Talking: Encoding Structured Data for LLMs [paper]
- (NeurIPS'24) G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering [paper][code][blog]
- (arXiv 2024.02) InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignment [paper][code]
- (arXiv 2024.02) LLaGA: Large Language and Graph Assistant [paper][code]
- (arXiv 2024.02) HiGPT: Heterogeneous Graph Language Model [paper][code]
- (arXiv 2024.02) UniGraph: Learning a Cross-Domain Graph Foundation Model From Natural Language [paper]
- (arXiv 2024.06) UniGLM: Training One Unified Language Model for Text-Attributed Graphs [paper][code]
- (arXiv 2024.07) GOFA: A Generative One-For-All Model for Joint Graph Language Modeling [paper][code]
- (NeurIPS'23) GraphAdapter: Tuning Vision-Language Models With Dual Knowledge Graph [paper][code]
- (arXiv 2023.10) Multimodal Graph Learning for Generative Tasks [paper][code]
- (arXiv 2024.02) Rendering Graphs for Graph Reasoning in Multimodal Large Language Models [paper]
- (ACL 2024) Graph Language Models [paper][code]
- (KDD'24) GraphWiz: An Instruction-Following Language Model for Graph Problems [paper][code][project]
- (arXiv 2023.04) Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPT [paper][code]
- (arXiv 2023.10) GraphText: Graph Reasoning in Text Space [paper]
- (arXiv 2023.10) GraphLLM: Boosting Graph Reasoning Ability of Large Language Model [paper][code]
- (ICLR'24) Explanations as Features: LLM-Based Features for Text-Attributed Graphs [paper][code]
- (ICLR'24) Label-free Node Classification on Graphs with Large Language Models (LLMS) [paper]
- (WWW'24) Can GNN be Good Adapter for LLMs? [paper][code]
- (CIKM'24) Distilling Large Language Models for Text-Attributed Graph Learning [paper]
- (arXiv 2023.07) Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs [paper][code]
- (arXiv 2023.09) Can LLMs Effectively Leverage Structural Information for Graph Learning: When and Why [paper][code]
- (arXiv 2023.10) Empower Text-Attributed Graphs Learning with Large Language Models (LLMs) [paper]
- (arXiv 2023.10) Disentangled Representation Learning with Large Language Models for Text-Attributed Graphs [paper]
- (arXiv 2023.11) Large Language Models as Topological Structure Enhancers for Text-Attributed Graphs [paper]
- (arXiv 2024.01) Efficient Tuning and Inference for Large Language Models on Textual Graphs [paper][code]
- (arXiv 2024.02) Similarity-based Neighbor Selection for Graph LLMs [paper] [code]
- (arXiv 2024.02) Distilling Large Language Models for Text-Attributed Graph Learning [paper]
- (arXiv 2024.02) GraphEdit: Large Language Models for Graph Structure Learning [paper][code]
- (arXiv 2024.05) LOGIN: A Large Language Model Consulted Graph Neural Network Training Framework [paper][code]
- (arXiv 2024.06) GAugLLM: Improving Graph Contrastive Learning for Text-Attributed Graphs with Large Language Models [paper][code]
- (arXiv 2024.07) Enhancing Data-Limited Graph Neural Networks by Actively Distilling Knowledge from Large Language Models [paper]
- (arXiv 2024.07) All Against Some: Efficient Integration of Large Language Models for Message Passing in Graph Neural Networks [paper]
- (arXiv 2023.06) GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learning [paper][code]
- (arXiv 2023.07) Can Large Language Models Empower Molecular Property Prediction? [paper][code]
- (AAAI'22) Enhanced Story Comprehension for Large Language Models through Dynamic Document-Based Knowledge Graphs [paper]
- (EMNLP'22) Language Models of Code are Few-Shot Commonsense Learners [paper][code]
- (SIGIR'23) Schema-aware Reference as Prompt Improves Data-Efficient Knowledge Graph Construction [paper][code]
- (TKDE‘23) AutoAlign: Fully Automatic and Effective Knowledge Graph Alignment enabled by Large Language Models [paper][code]
- (AAAI'24) Graph Neural Prompting with Large Language Models [paper][code]
- (NAACL'24) zrLLM: Zero-Shot Relational Learning on Temporal Knowledge Graphs with Large Language Models [paper]
- (ICLR'24) Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph [paper][code]
- (arXiv 2023.04) CodeKGC: Code Language Model for Generative Knowledge Graph Construction [paper][code]
- (arXiv 2023.05) Knowledge Graph Completion Models are Few-shot Learners: An Empirical Study of Relation Labeling in E-commerce with LLMs [paper]
- (arXiv 2023.08) MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Models [paper][code]
- (arXiv 2023.10) Faithful Path Language Modelling for Explainable Recommendation over Knowledge Graph [paper]
- (arXiv 2023.10) Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning [paper][code]
- (arXiv 2023.11) Zero-Shot Relational Learning on Temporal Knowledge Graphs with Large Language Models [paper]
- (arXiv 2023.12) KGLens: A Parameterized Knowledge Graph Solution to Assess What an LLM Does and Doesn’t Know [paper]
- (arXiv 2024.02) Large Language Model Meets Graph Neural Network in Knowledge Distillation [paper]
- (arXiv 2024.02) Large Language Models Can Learn Temporal Reasoning [paper][code]
- (arXiv 2024.02) Knowledge Graph Large Language Model (KG-LLM) for Link Prediction [paper]
- (arXiv 2024.03) Call Me When Necessary: LLMs can Efficiently and Faithfully Reason over Structured Environments [paper]
- (arXiv 2024.04) Evaluating the Factuality of Large Language Models using Large-Scale Knowledge Graphs [paper][code]
- (arXiv 2024.04) Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction [paper]
- (arXiv 2024.05) FiDeLiS: Faithful Reasoning in Large Language Model for Knowledge Graph Question Answering [paper]
- (arXiv 2024.06) Explore then Determine: A GNN-LLM Synergy Framework for Reasoning over Knowledge Graph [paper]
- (ACL 2024) Graph Language Models [paper][code]
- (arXiv 2024.06) MolecularGPT: Open Large Language Model (LLM) for Few-Shot Molecular Property Prediction [paper][code]
- (arXiv 2024.06) HIGHT: Hierarchical Graph Tokenization for Graph-Language Alignment [paper][project]
- (arXiv 2024.06) MolX: Enhancing Large Language Models for Molecular Learning with A Multi-Modal Extension [paper]
- (arXiv 2024.06) LLM and GNN are Complementary: Distilling LLM for Multimodal Graph Learning [paper]
- (arXiv 2024.05) Intruding with Words: Towards Understanding Graph Injection Attacks at the Text Level [paper]
- (arXiv 2024.08) Can Large Language Models Improve the Adversarial Robustness of Graph Neural Networks? [paper]
- (WSDM'24) LLMRec: Large Language Models with Graph Augmentation for Recommendation [paper][code][blog in Chinese].
- (arXiv 2023.03) Ask and You Shall Receive (a Graph Drawing): Testing ChatGPT’s Potential to Apply Graph Layout Algorithms [paper]
- (arXiv 2023.05) Graph Meets LLM: A Novel Approach to Collaborative Filtering for Robust Conversational Understanding [paper]
- (arXiv 2023.05) ChatGPT Informed Graph Neural Network for Stock Movement Prediction [paper][code]
- (arXiv 2023.10) Graph Neural Architecture Search with GPT-4 [paper]
- (arXiv 2023.11) Biomedical knowledge graph-enhanced prompt generation for large language models [paper][code]
- (arXiv 2023.11) Graph-Guided Reasoning for Multi-Hop Question Answering in Large Language Models [paper]
- (arXiv 2024.02) Microstructures and Accuracy of Graph Recall by Large Language Models [paper]
- (arXiv 2024.02) Causal Graph Discovery with Retrieval-Augmented Generation based Large Language Models [paper]
- (arXiv 2024.02) Graph-enhanced Large Language Models in Asynchronous Plan Reasoning [paper][code]
- (arXiv 2024.02) Efficient Causal Graph Discovery Using Large Language Models [paper]
- (arXiv 2024.03) Exploring the Potential of Large Language Models in Graph Generation [paper]
- (arXiv 2024.05) Don't Forget to Connect! Improving RAG with Graph-based Reranking [paper]
- (NeurIPS'24) Can Graph Learning Improve Planning in LLM-based Agents? [paper][code]
- (arXiv 2024.06) GNN-RAG: Graph Neural Retrieval for Large Language Modeling Reasoning [paper][code]
- (arXiv 2024.07) LLMExplainer: Large Language Model based Bayesian Inference for Graph Explanation Generation [paper]
- (arXiv 2024.08) CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases [paper][code][project]
- GraphGPT: Extrapolating knowledge graphs from unstructured text using GPT-3
- GraphML: Graph markup language. An XML-based file format for graphs.
- GML: Graph modelling language. Read graphs in GML format.
👍 Contributions to this repository are welcome!
If you have come across relevant resources, feel free to open an issue or submit a pull request.
- (*conference|journal*) paper_name [[pdf](link)][[code](link)]
For Tasks:
Click tags to check more tools for each tasksFor Jobs:
Alternative AI tools for Awesome-Graph-LLM
Similar Open Source Tools
Awesome-Graph-LLM
Awesome-Graph-LLM is a curated collection of research papers exploring the intersection of graph-based techniques with Large Language Models (LLMs). The repository aims to bridge the gap between LLMs and graph structures prevalent in real-world applications by providing a comprehensive list of papers covering various aspects of graph reasoning, node classification, graph classification/regression, knowledge graphs, multimodal models, applications, and tools. It serves as a valuable resource for researchers and practitioners interested in leveraging LLMs for graph-related tasks.
Awesome-LLMs-in-Graph-tasks
This repository is a collection of papers on leveraging Large Language Models (LLMs) in Graph Tasks. It provides a comprehensive overview of how LLMs can enhance graph-related tasks by combining them with traditional Graph Neural Networks (GNNs). The integration of LLMs with GNNs allows for capturing both structural and contextual aspects of nodes in graph data, leading to more powerful graph learning. The repository includes summaries of various models that leverage LLMs to assist in graph-related tasks, along with links to papers and code repositories for further exploration.
Awesome-GenAI-Unlearning
This repository is a collection of papers on Generative AI Machine Unlearning, categorized based on modality and applications. It includes datasets, benchmarks, and surveys related to unlearning scenarios in generative AI. The repository aims to provide a comprehensive overview of research in the field of machine unlearning for generative models.
Awesome-LLM4Graph-Papers
A collection of papers and resources about Large Language Models (LLM) for Graph Learning (Graph). Integrating LLMs with graph learning techniques to enhance performance in graph learning tasks. Categorizes approaches based on four primary paradigms and nine secondary-level categories. Valuable for research or practice in self-supervised learning for recommendation systems.
llm-continual-learning-survey
This repository is an updating survey for Continual Learning of Large Language Models (CL-LLMs), providing a comprehensive overview of various aspects related to the continual learning of large language models. It covers topics such as continual pre-training, domain-adaptive pre-training, continual fine-tuning, model refinement, model alignment, multimodal LLMs, and miscellaneous aspects. The survey includes a collection of relevant papers, each focusing on different areas within the field of continual learning of large language models.
Awesome-TimeSeries-SpatioTemporal-LM-LLM
Awesome-TimeSeries-SpatioTemporal-LM-LLM is a curated list of Large (Language) Models and Foundation Models for Temporal Data, including Time Series, Spatio-temporal, and Event Data. The repository aims to summarize recent advances in Large Models and Foundation Models for Time Series and Spatio-Temporal Data with resources such as papers, code, and data. It covers various applications like General Time Series Analysis, Transportation, Finance, Healthcare, Event Analysis, Climate, Video Data, and more. The repository also includes related resources, surveys, and papers on Large Language Models, Foundation Models, and their applications in AIOps.
LLM-for-misinformation-research
LLM-for-misinformation-research is a curated paper list of misinformation research using large language models (LLMs). The repository covers methods for detection and verification, tools for fact-checking complex claims, decision-making and explanation, claim matching, post-hoc explanation generation, and other tasks related to combating misinformation. It includes papers on fake news detection, rumor detection, fact verification, and more, showcasing the application of LLMs in various aspects of misinformation research.
aim
Aim is an open-source, self-hosted ML experiment tracking tool designed to handle 10,000s of training runs. Aim provides a performant and beautiful UI for exploring and comparing training runs. Additionally, its SDK enables programmatic access to tracked metadata — perfect for automations and Jupyter Notebook analysis. **Aim's mission is to democratize AI dev tools 🎯**
prompt-in-context-learning
An Open-Source Engineering Guide for Prompt-in-context-learning from EgoAlpha Lab. 📝 Papers | ⚡️ Playground | 🛠 Prompt Engineering | 🌍 ChatGPT Prompt | ⛳ LLMs Usage Guide > **⭐️ Shining ⭐️:** This is fresh, daily-updated resources for in-context learning and prompt engineering. As Artificial General Intelligence (AGI) is approaching, let’s take action and become a super learner so as to position ourselves at the forefront of this exciting era and strive for personal and professional greatness. The resources include: _🎉Papers🎉_: The latest papers about _In-Context Learning_ , _Prompt Engineering_ , _Agent_ , and _Foundation Models_. _🎉Playground🎉_: Large language models(LLMs)that enable prompt experimentation. _🎉Prompt Engineering🎉_: Prompt techniques for leveraging large language models. _🎉ChatGPT Prompt🎉_: Prompt examples that can be applied in our work and daily lives. _🎉LLMs Usage Guide🎉_: The method for quickly getting started with large language models by using LangChain. In the future, there will likely be two types of people on Earth (perhaps even on Mars, but that's a question for Musk): - Those who enhance their abilities through the use of AIGC; - Those whose jobs are replaced by AI automation. 💎EgoAlpha: Hello! human👤, are you ready?
Awesome-explainable-AI
This repository contains frontier research on explainable AI (XAI), a hot topic in the field of artificial intelligence. It includes trends, use cases, survey papers, books, open courses, papers, and Python libraries related to XAI. The repository aims to organize and categorize publications on XAI, provide evaluation methods, and list various Python libraries for explainable AI.
awesome-LLM-AIOps
The 'awesome-LLM-AIOps' repository is a curated list of academic research and industrial materials related to Large Language Models (LLM) and Artificial Intelligence for IT Operations (AIOps). It covers various topics such as incident management, log analysis, root cause analysis, incident mitigation, and incident postmortem analysis. The repository provides a comprehensive collection of papers, projects, and tools related to the application of LLM and AI in IT operations, offering valuable insights and resources for researchers and practitioners in the field.
awesome-object-detection-datasets
This repository is a curated list of awesome public object detection and recognition datasets. It includes a wide range of datasets related to object detection and recognition tasks, such as general detection and recognition datasets, autonomous driving datasets, adverse weather datasets, person detection datasets, anti-UAV datasets, optical aerial imagery datasets, low-light image datasets, infrared image datasets, SAR image datasets, multispectral image datasets, 3D object detection datasets, vehicle-to-everything field datasets, super-resolution field datasets, and face detection and recognition datasets. The repository also provides information on tools for data annotation, data augmentation, and data management related to object detection tasks.
Awesome-Efficient-AIGC
This repository, Awesome Efficient AIGC, collects efficient approaches for AI-generated content (AIGC) to cope with its huge demand for computing resources. It includes efficient Large Language Models (LLMs), Diffusion Models (DMs), and more. The repository is continuously improving and welcomes contributions of works like papers and repositories that are missed by the collection.
EvalAI
EvalAI is an open-source platform for evaluating and comparing machine learning (ML) and artificial intelligence (AI) algorithms at scale. It provides a central leaderboard and submission interface, making it easier for researchers to reproduce results mentioned in papers and perform reliable & accurate quantitative analysis. EvalAI also offers features such as custom evaluation protocols and phases, remote evaluation, evaluation inside environments, CLI support, portability, and faster evaluation.
For similar tasks
Awesome-LLM4Graph-Papers
A collection of papers and resources about Large Language Models (LLM) for Graph Learning (Graph). Integrating LLMs with graph learning techniques to enhance performance in graph learning tasks. Categorizes approaches based on four primary paradigms and nine secondary-level categories. Valuable for research or practice in self-supervised learning for recommendation systems.
Graph-CoT
This repository contains the source code and datasets for Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs accepted to ACL 2024. It proposes a framework called Graph Chain-of-thought (Graph-CoT) to enable Language Models to traverse graphs step-by-step for reasoning, interaction, and execution. The motivation is to alleviate hallucination issues in Language Models by augmenting them with structured knowledge sources represented as graphs.
Awesome-Graph-LLM
Awesome-Graph-LLM is a curated collection of research papers exploring the intersection of graph-based techniques with Large Language Models (LLMs). The repository aims to bridge the gap between LLMs and graph structures prevalent in real-world applications by providing a comprehensive list of papers covering various aspects of graph reasoning, node classification, graph classification/regression, knowledge graphs, multimodal models, applications, and tools. It serves as a valuable resource for researchers and practitioners interested in leveraging LLMs for graph-related tasks.
automatic-KG-creation-with-LLM
This repository presents a (semi-)automatic pipeline for Ontology and Knowledge Graph Construction using Large Language Models (LLMs) such as Mixtral 8x22B Instruct v0.1, GPT-4o, GPT-3.5, and Gemini. It explores the generation of Knowledge Graphs by formulating competency questions, developing ontologies, constructing KGs, and evaluating the results with minimal human involvement. The project showcases the creation of a KG on deep learning methodologies from scholarly publications. It includes components for data preprocessing, prompts for LLMs, datasets, and results from the selected LLMs.
Sarvadnya
Sarvadnya is a repository focused on interfacing custom data using Large Language Models (LLMs) through Proof-of-Concepts (PoCs) like Retrieval Augmented Generation (RAG) and Fine-Tuning. It aims to enable domain adaptation for LLMs to answer on user-specific corpora. The repository also covers topics such as Indic-languages models, 3D World Simulations, Knowledge Graphs Generation, Signal Processing, Drones, UAV Image Processing, and Floor Plan Segmentation. It provides insights into building chatbots of various modalities, preparing videos, and creating content for different platforms like Medium, LinkedIn, and YouTube. The tech stacks involved range from enterprise solutions like Google Doc AI and Microsoft Azure Language AI Services to open-source tools like Langchain and HuggingFace.
For similar jobs
weave
Weave is a toolkit for developing Generative AI applications, built by Weights & Biases. With Weave, you can log and debug language model inputs, outputs, and traces; build rigorous, apples-to-apples evaluations for language model use cases; and organize all the information generated across the LLM workflow, from experimentation to evaluations to production. Weave aims to bring rigor, best-practices, and composability to the inherently experimental process of developing Generative AI software, without introducing cognitive overhead.
LLMStack
LLMStack is a no-code platform for building generative AI agents, workflows, and chatbots. It allows users to connect their own data, internal tools, and GPT-powered models without any coding experience. LLMStack can be deployed to the cloud or on-premise and can be accessed via HTTP API or triggered from Slack or Discord.
VisionCraft
The VisionCraft API is a free API for using over 100 different AI models. From images to sound.
kaito
Kaito is an operator that automates the AI/ML inference model deployment in a Kubernetes cluster. It manages large model files using container images, avoids tuning deployment parameters to fit GPU hardware by providing preset configurations, auto-provisions GPU nodes based on model requirements, and hosts large model images in the public Microsoft Container Registry (MCR) if the license allows. Using Kaito, the workflow of onboarding large AI inference models in Kubernetes is largely simplified.
PyRIT
PyRIT is an open access automation framework designed to empower security professionals and ML engineers to red team foundation models and their applications. It automates AI Red Teaming tasks to allow operators to focus on more complicated and time-consuming tasks and can also identify security harms such as misuse (e.g., malware generation, jailbreaking), and privacy harms (e.g., identity theft). The goal is to allow researchers to have a baseline of how well their model and entire inference pipeline is doing against different harm categories and to be able to compare that baseline to future iterations of their model. This allows them to have empirical data on how well their model is doing today, and detect any degradation of performance based on future improvements.
tabby
Tabby is a self-hosted AI coding assistant, offering an open-source and on-premises alternative to GitHub Copilot. It boasts several key features: * Self-contained, with no need for a DBMS or cloud service. * OpenAPI interface, easy to integrate with existing infrastructure (e.g Cloud IDE). * Supports consumer-grade GPUs.
spear
SPEAR (Simulator for Photorealistic Embodied AI Research) is a powerful tool for training embodied agents. It features 300 unique virtual indoor environments with 2,566 unique rooms and 17,234 unique objects that can be manipulated individually. Each environment is designed by a professional artist and features detailed geometry, photorealistic materials, and a unique floor plan and object layout. SPEAR is implemented as Unreal Engine assets and provides an OpenAI Gym interface for interacting with the environments via Python.
Magick
Magick is a groundbreaking visual AIDE (Artificial Intelligence Development Environment) for no-code data pipelines and multimodal agents. Magick can connect to other services and comes with nodes and templates well-suited for intelligent agents, chatbots, complex reasoning systems and realistic characters.