Awesome-Knowledge-Distillation-of-LLMs

Awesome-Knowledge-Distillation-of-LLMs

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vertical Distillation of LLMs.

Stars: 890

Visit
screenshot

A collection of papers related to knowledge distillation of large language models (LLMs). The repository focuses on techniques to transfer advanced capabilities from proprietary LLMs to smaller models, compress open-source LLMs, and refine their performance. It covers various aspects of knowledge distillation, including algorithms, skill distillation, verticalization distillation in fields like law, medical & healthcare, finance, science, and miscellaneous domains. The repository provides a comprehensive overview of the research in the area of knowledge distillation of LLMs.

README:

Awesome Knowledge Distillation of LLM Papers

Awesome PDF

A Survey on Knowledge Distillation of Large Language Models

Xiaohan Xu1&nbsp&nbsp Ming Li2&nbsp&nbsp Chongyang Tao3&nbsp&nbsp Tao Shen4&nbsp&nbsp Reynold Cheng1&nbsp&nbsp Jinyang Li1&nbsp&nbsp Can Xu5&nbsp&nbsp Dacheng Tao6&nbsp&nbsp Tianyi Zhou2&nbsp&nbsp

1 The University of Hong Kong &nbsp&nbsp 2 University of Maryland &nbsp&nbsp 3 Microsoft &nbsp&nbsp 4 University of Technology Sydney &nbsp&nbsp 5 Peking University &nbsp&nbsp 6 The University of Sydney



A collection of papers related to knowledge distillation of large language models (LLMs). If you want to use LLMs for benefitting your own smaller models training, or use self-generated knowledge to achieve the self-improvement, just take a look at this collection.

We will update this collection every week. Welcome to star ⭐️ this repo to keep track of the updates.

❗️Legal Consideration: It's crucial to note the legal implications of utilizing LLM outputs, such as those from ChatGPT (Restrictions), Llama (License), etc. We strongly advise users to adhere to the terms of use specified by the model providers, such as the restrictions on developing competitive products, and so on.

💡 News

Contributing to This Collection

Feel free to open an issue/PR or e-mail [email protected], [email protected], [email protected] and [email protected] if you find any missing taxonomies or papers. We will keep updating this collection and survey.

📝 Introduction

KD of LLMs: This survey delves into knowledge distillation (KD) techniques in Large Language Models (LLMs), highlighting KD's crucial role in transferring advanced capabilities from proprietary LLMs like GPT-4 to open-source counterparts such as LLaMA and Mistral. We also explore how KD enables the compression and self-improvement of open-source LLMs by using them as teachers.

KD and Data Augmentation: Crucially, the survey navigates the intricate interplay between data augmentation (DA) and KD, illustrating how DA emerges as a powerful paradigm within the KD framework to bolster LLMs' performance. By leveraging DA to generate context-rich, skill-specific training data, KD transcends traditional boundaries, enabling open-source models to approximate the contextual adeptness, ethical alignment, and deep semantic insights characteristic of their proprietary counterparts.

Taxonomy: Our analysis is meticulously structured around three foundational pillars: algorithm, skill, and verticalization -- providing a comprehensive examination of KD mechanisms, the enhancement of specific cognitive abilities, and their practical implications across diverse fields.

KD Algorithms: For KD algorithms, we categorize it into two principal steps: "Knowledge Elicitation" focusing on eliciting knowledge from teacher LLMs, and "Distillation Algorithms" centered on injecting this knowledge into student models.


Figure: An illustration of different knowledge elicitation methods from teacher LLMs.

Skill Distillation: We delve into the enhancement of specific cognitive abilities, such as context following, alignment, agent, NLP task specialization, and multi-modality.

Verticalization Distillation: We explore the practical implications of KD across diverse fields, including law, medical & healthcare, finance, science, and miscellaneous domains.

Note that both Skill Distillation and Verticalization Distillation employ Knowledge Elicitation and Distillation Algorithms in KD Algorithms to achieve their KD. Thus, there are overlaps between them. However, this could also provide different perspectives for the papers.

Why KD of LLMs?

In the era of LLMs, KD of LLMs plays the following crucial roles:



Role Description Trend
① Advancing SLMs Transferring advanced capabilities from proprietary LLMs to smaller SLMs, such as open source LLMs or other smaller models. Most common
② Compression Compressing LLMs to make them more efficient and practical. More popular with the prosperity of open-source LLMs
③ Self-Improvement Refining open-source LLMs' performance by leveraging their own knowledge, i.e. self-knowledge. New trend to make open-source LLMs more competitive

📒 Table of Contents

KD Algorithms

Knowledge Elicitation

Labeling

Title Venue Date Code Data
Evidence-Focused Fact Summarization for Knowledge-Augmented Zero-Shot Question Answering arXiv 2024-03
Aligning Large and Small Language Models via Chain-of-Thought Reasoning EACL 2024-03 Github
Divide-or-Conquer? Which Part Should You Distill Your LLM? arXiv 2024-02
Miko: Multimodal Intention Knowledge Distillation from Large Language Models for Social-Media Commonsense Discovery arXiv 2024-02
KnowTuning: Knowledge-aware Fine-tuning for Large Language Models arXiv 2024-02 Github
TinyLLM: Learning a Small Student from Multiple Large Language Models arXiv 2024-02
Mixed Distillation Helps Smaller Language Model Better Reasoning arXiv 2023-12
Tailoring Self-Rationalizers with Multi-Reward Distillation arXiv 2023-11 Github Data
Orca 2: Teaching Small Language Models How to Reason arXiv 2023-11
Mammoth: Building Math Generalist Models through Hybrid Instruction Tuning arXiv 2023-09 Github Data
PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization arXiv 2023-06 Github Data
Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Step ACL 2023-06
Orca: Progressive Learning from Complex Explanation Traces of GPT-4 arXiv 2023-06
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes ACL 2023-05 Github Data
Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasing arXiv 2023-05 Github
Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data EMNLP 2023-04 Github Data
ChatGPT outperforms crowd workers for text-annotation tasks arXiv 2023-03
Annollm: Making large language models to be better crowdsourced annotators arXiv 2023-03
GPT-4All: Training an Assistant-Style Chatbot with Large Scale Data Distillation from GPT-3.5-Turbo - 2023-03 Github
Specializing Smaller Language Models towards Multi-Step Reasoning arXiv 2023-01
Is GPT-3 a Good Data Annotator? ACL 2022-12 Github
Large Language Models Are Reasoning Teachers ACL 2022-12 Github Data
Teaching Small Language Models to Reason ACL 2022-12
Explanations from Large Language Models Make Small Reasoners Better arXiv 2022-10
Want To Reduce Labeling Cost? GPT-3 Can Help Findings of EMNLP 2021-08

Expansion

Title Venue Date Code Data
Instruction Fusion: Advancing Prompt Evolution through Hybridization arXiv 2023-12
An Empirical Study of Instruction-tuning Large Language Models in Chinese EMNLP 2023-10 Github Data
PromptMix: A Class Boundary Augmentation Method for Large Language Model Distillation EMNLP 2023-10 Github
Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct arXiv 2023-08 Github
Code Llama: Open Foundation Models for Code arXiv 2023-08 Github
WizardCoder: Empowering Code Large Language Models with Evol-Instruct ICLR 2023-06 Github
Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision NeurIPS 2023-05 Github Data
Targeted Data Generation: Finding and Fixing Model Weaknesses ACL 2023-05 Github
Wizardlm: Empowering large language models to follow complex instructions ICLR 2023-04 Github Data
Data
LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions arXiv 2023-04 Github Data
Alpaca: Aligning Language Model with Human Preferences - 2023-03 Github Data
Code Alpaca: An Instruction-following LLaMA model for code generation - 2023-03 Github Data
Exploring the Impact of Instruction Data Scaling on Large Language Models: An Empirical Study on Real-World Use Cases arXiv 2023-03 Github Data
AugGPT: Leveraging ChatGPT for Text Data Augmentation arXiv 2023-02 Github
Self-instruct: Aligning language model with self generated instructions ACL 2022-12 Github Data
Symbolic Knowledge Distillation: from General Language Models to Commonsense Models NAACL 2021-10 Github Data

Curation

Title Venue Date Code Data
Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models arXiv 2024-02
Phi-2: The surprising power of small language models - 2023-12
WaveCoder: Widespread And Versatile Enhanced Instruction Tuning with Refined Data Generation arXiv 2023-12
Magicoder: Source Code Is All You Need arXiv 2023-12 Github Data
Data
MFTCoder: Boosting Code LLMs with Multitask Fine-Tuning arXiv 2023-11 Github Data
Data
Textbooks Are All You Need II: Phi-1.5 Technical Report arXiv 2023-09
Neural Machine Translation Data Generation and Augmentation using ChatGPT arXiv 2023-07
Textbooks Are All You Need: A Large-Scale Instructional Text Data Set for Language Models arXiv 2023-06
Enhancing Chat Language Models by Scaling High-quality Instructional Conversations arXiv 2023-05 Github Data
AugTriever: Unsupervised Dense Retrieval by Scalable Data Augmentation arXiv 2022-12 Github
SunGen: Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learning ICLR 2022-05 Github
ZeroGen: Efficient Zero-shot Learning via Dataset Generation EMNLP 2022-02 Github
InPars: Data Augmentation for Information Retrieval using Large Language Models arXiv 2022-02 Github Data
Towards Zero-Label Language Learning arXiv 2021-09

Feature

Title Venue Date Code Data
PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuning EMNLP Findings 2024-02 Github Data
Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models arXiv 2024-04
Direct Alignment of Draft Model for Speculative Decoding with Chat-Fine-Tuned LLMs arXiv 2024-03
DB-LLM: Accurate Dual-Binarization for Efficient LLMs arXiv 2024-02
BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillation arXiv 2024-02 Github
DISTILLM: Towards Streamlined Distillation for Large Language Models arXiv 2024-02 Github
Towards Cross-Tokenizer Distillation: the Universal Logit Distillation Loss for LLMs arXiv 2024-02 Github Data
Revisiting Knowledge Distillation for Autoregressive Language Models arXiv 2024-02
Knowledge Fusion of Large Language Models ICLR 2024-01 Github
Improving In-context Learning via Bidirectional Alignment arXiv 2023-12
Towards the Fundamental Limits of Knowledge Transfer over Finite Domains NeurIPS 2023-10
Baby Llama: Knowledge Distillation from an Ensemble of Teachers Trained on a Small Dataset with No Performance Penalty CoNLL 2023-08 Github Data
f-Divergence Minimization for Sequence-Level Knowledge Distillation ACL 2023-07 Github Data
MiniLLM: Knowledge Distillation of Large Language Models ICLR 2023-06 Github Data
On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes ICLR 2023-06
LLM-QAT: Data-Free Quantization Aware Training for Large Language Models arXiv 2023-05 Github Data
Less is more: Task-aware layer-wise distillation for language model compression PMLR 2022-10 Github

Feedback

Title Venue Date Code Data
PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuning EMNLP Findings 2024-02 Github Data
Evidence-Focused Fact Summarization for Knowledge-Augmented Zero-Shot Question Answering arXiv 2024-03
Evolving Knowledge Distillation with Large Language Models and Active Learning arXiv 2024-03
Direct Language Model Alignment from Online AI Feedback arXiv 2024-02
DISTILLM: Towards Streamlined Distillation for Large Language Models arXiv 2024-02 Github
Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraint arXiv 2024-01 Github
Beyond Imitation: Leveraging Fine-grained Quality Signals for Alignment arXiv 2023-11
Can Language Models Teach Weaker Agents? Teacher Explanations Improve Students via Personalization ICLR 2023-10 Github
Motif: Intrinsic Motivation from Artificial Intelligence Feedback ICLR 2023-10 Github
Ultrafeedback: Boosting language models with high-quality feedback arXiv 2023-10 Github Data
Personalised Distillation: Empowering Open-Sourced LLMs with Adaptive Learning for Code Generation EMNLP 2023-10 Github
CycleAlign: Iterative Distillation from Black-box LLM to White-box Models for Better Human Alignment arXiv 2023-10
Rlaif: Scaling Reinforcement Learning from Human Feedback with AI Feedback arXiv 2023-09
Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct arXiv 2023-08 Github
On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes ICLR 2023-06
MiniLLM: Knowledge Distillation of Large Language Models ICLR 2023-06 Github Data
Language to Rewards for Robotic Skill Synthesis arXiv 2023-06 Github
Lion: Adversarial Distillation of Closed-Source Large Language Model EMNLP 2023-05 Github
SelFee: Iterative Self-Revising LLM Empowered by Self-Feedback Generation arXiv 2023-05
LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions arXiv 2023-04 Github Data
Reward Design with Language Models ICLR 2023-03 Github
Consitutional AI: Harmlessness from AI Feedback arXiv 2022-12

Self-Knowledge

Title Venue Date Code Data
V-STaR: Training Verifiers for Self-Taught Reasoners arXiv 2024-02
Self-Rewarding Language Models arXiv 2024-01 Github
Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models arXiv 2024-01 Github Data
Kun: Answer Polishment for Chinese Self-Alignment with Instruction Back-Translation arXiv 2024-01 Github Data
APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference arXiv 2024-01
GRATH: Gradual Self-Truthifying for Large Language Models arXiv 2024-01
Beyond human data: Scaling self-training for problem-solving with language models arXiv 2023-12
Self-Knowledge Guided Retrieval Augmentation for Large Language Models EMNLP Findings 2023-10 Github
RAIN: Your Language Models Can Align Themselves without Finetuning arXiv 2023-09 Github
Reinforced Self-Training (ReST) for Language Modeling arXiv 2023-08
Humback: Self-Alignment with Instruction Backtranslation ICLR 2023-08 Github
Self-Alignment of Large Language Models via Reinforcement Learning from Contrast Distillation ICLR 2023-07 Github
Self-Improvement of Large Language Models via Reinforcement Learning from Human Feedback EMNLP 2023-06
Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision NeurIPS 2023-05 Github Data
Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasing arXiv 2023-05 Github
Language Model Self-improvement by Reinforcement Learning Contemplation arXiv 2023-05
Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data EMNLP 2023-04 Github Data
Self-instruct: Aligning language model with self generated instructions ACL 2022-12 Github Data
Large Language Models Can Self-Improve EMNLP 2022-10
STaR: Bootstrapping Reasoning With Reasoning NeurIPS 2022-03 Github

Distillation Algorithms

Supervised Fine-Tuning

Due to the large number of works applying supervised fine-tuning, we only list the most representative ones here.

Title Venue Date Code Data
Evidence-Focused Fact Summarization for Knowledge-Augmented Zero-Shot Question Answering arXiv 2024-03
Aligning Large and Small Language Models via Chain-of-Thought Reasoning EACL 2024-03 Github
Divide-or-Conquer? Which Part Should You Distill Your LLM? arXiv 2024-02
Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models arXiv 2024-02
Orca 2: Teaching Small Language Models How to Reason arXiv 2023-11
TinyLLM: Learning a Small Student from Multiple Large Language Models arXiv 2024-02
Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct arXiv 2023-08 Github
Orca: Progressive Learning from Complex Explanation Traces of GPT-4 arXiv 2023-06
LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions arXiv 2023-04 Github Data
Wizardlm: Empowering large language models to follow complex instructions ICLR 2023-04 Github Data
Data
Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data EMNLP 2023-04 Github Data
Alpaca: Aligning Language Model with Human Preferences - 2023-03 Github Data
Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90% ChatGPT Quality* - 2023-03 Github Data
Self-instruct: Aligning language model with self generated instructions ACL 2022-12 Github Data
Large Language Models Can Self-Improve EMNLP 2022-10
STaR: Bootstrapping Reasoning With Reasoning NeurIPS 2022-03 Github

Divergence and Similarity

Title Venue Date Code Data
PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuning EMNLP Findings 2024-02 Github Data
Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models arXiv 2024-04
Weight-Inherited Distillation for Task-Agnostic BERT Compression NAACL 2024-03 Github
BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillation arXiv 2024-02 Github
DISTILLM: Towards Streamlined Distillation for Large Language Models arXiv 2024-02 Github
Towards Cross-Tokenizer Distillation: the Universal Logit Distillation Loss for LLMs arXiv 2024-02 Github Data
Revisiting Knowledge Distillation for Autoregressive Language Models arXiv 2024-02
Knowledge Distillation for Closed-Source Language Models arXiv 2024-01
Knowledge Fusion of Large Language Models ICLR 2024-01 Github
Improving In-context Learning via Bidirectional Alignment arXiv 2023-12
Towards the Fundamental Limits of Knowledge Transfer over Finite Domains NeurIPS 2023-10
Baby Llama: Knowledge Distillation from an Ensemble of Teachers Trained on a Small Dataset with No Performance Penalty CoNLL 2023-08 Github Data
f-Divergence Minimization for Sequence-Level Knowledge Distillation ACL 2023-07 Github Data
f-Divergence Minimization for Sequence-Level Knowledge Distillation ACL 2023-07 Github Data
MiniLLM: Knowledge Distillation of Large Language Models ICLR 2023-06 Github Data
On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes ICLR 2023-06
LLM-QAT: Data-Free Quantization Aware Training for Large Language Models arXiv 2023-05 Github Data
Less is more: Task-aware layer-wise distillation for language model compression PMLR 2022-10 Github
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter NeurIPS 2019-10

Reinforcement Learning

Title Venue Date Code Data
Direct Language Model Alignment from Online AI Feedback arXiv 2024-02
Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraint arXiv 2024-01 Github
Accelerating Reinforcement Learning of Robotic Manipulations via Feedback from Large Language Models CoRL 2023-11
Motif: Intrinsic Motivation from Artificial Intelligence Feedback ICLR 2023-10 Github
Ultrafeedback: Boosting language models with high-quality feedback arXiv 2023-10 Github Data
Eureka: Human-Level Reward Design via Coding Large Language Models arXiv 2023-10 Github
Rlaif: Scaling Reinforcement Learning from Human Feedback with AI Feedback arXiv 2023-09
Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct arXiv 2023-08 Github
On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes ICLR 2023-06
Aligning Large Language Models through Synthetic Feedback EMNLP 2023-05 Github Data
Language Model Self-improvement by Reinforcement Learning Contemplation arXiv 2023-05
Consitutional AI: Harmlessness from AI Feedback arXiv 2022-12

Rank Optimization

Title Venue Date Code Data
Evidence-Focused Fact Summarization for Knowledge-Augmented Zero-Shot Question Answering arXiv 2024-03
KnowTuning: Knowledge-aware Fine-tuning for Large Language Models arXiv 2024-02 Github
Self-Rewarding Language Models arXiv 2024-01 Github
Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models arXiv 2024-01 Github Data
Zephyr: Direct Distillation of Language Model Alignment arXiv 2023-10 Github Data
CycleAlign: Iterative Distillation from Black-box LLM to White-box Models for Better Human Alignment arXiv 2023-10

Skill Distillation

Context Following

Instruction Following

Title Venue Date Code Data
Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models arXiv 2024-02
Revisiting Knowledge Distillation for Autoregressive Language Models arXiv 2024-02
Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning arXiv 2024-02 Github Data
Phi-2: The surprising power of small language models - 2023-12
What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning ICLR 2023-12 Github Data
MUFFIN: Curating Multi-Faceted Instructions for Improving Instruction-Following arXiv 2023-12 Github Data
Instruction Fusion: Advancing Prompt Evolution through Hybridization arXiv 2023-12
Orca 2: Teaching Small Language Models How to Reason arXiv 2023-11
Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning NIPS Workshop 2023-10 Github Data
Textbooks Are All You Need II: Phi-1.5 Technical Report arXiv 2023-09
Orca: Progressive Learning from Complex Explanation Traces of GPT-4 arXiv 2023-06
Textbooks Are All You Need: A Large-Scale Instructional Text Data Set for Language Models arXiv 2023-06
SelFee: Iterative Self-Revising LLM Empowered by Self-Feedback Generation arXiv 2023-05
ExpertPrompting: Instructing Large Language Models to be Distinguished Experts arXiv 2023-05 Github Data
LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions arXiv 2023-04 Github Data
Wizardlm: Empowering large language models to follow complex instructions ICLR 2023-04 Github Data
Data
Koala: A Dialogue Model for Academic Research - 2023-04 Github Data
Alpaca: Aligning Language Model with Human Preferences - 2023-03 Github Data
Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90% ChatGPT Quality* - 2023-03 Github Data
Self-instruct: Aligning language model with self generated instructions ACL 2022-12 Github Data

Multi-turn Dialogue

Title Venue Date Code Data
Zephyr: Direct Distillation of LM Alignment arXiv 2023-10 Github Data
OPENCHAT: ADVANCING OPEN-SOURCE LANGUAGE MODELS WITH MIXED-QUALITY DATA ICLR 2023-09 Github Data
Enhancing Chat Language Models by Scaling High-quality Instructional Conversations arXiv 2023-05 Github Data
Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data EMNLP 2023-04 Github Data
Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90% ChatGPT Quality* - 2023-03 Github Data

RAG Capability

Title Venue Date Code Data
Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection NIPS 2023-10 Github Data
SAIL: Search-Augmented Instruction Learning arXiv 2023-05 Github Data
Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks NIPS 2023-05 Github Data

Alignment

Thinking Pattern

Title Venue Date Code Data
Aligning Large and Small Language Models via Chain-of-Thought Reasoning EACL 2024-03 Github
Divide-or-Conquer? Which Part Should You Distill Your LLM? arXiv 2024-02
Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning arXiv 2024-02 Github Data
Can LLMs Speak For Diverse People? Tuning LLMs via Debate to Generate Controllable Controversial Statements arXiv 2024-02 Github Data
Knowledgeable Preference Alignment for LLMs in Domain-specific Question Answering arXiv 2023-11 Github
Orca 2: Teaching Small Language Models How to Reason arXiv 2023-11
Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning NIPS Workshop 2023-10 Github Data
Orca: Progressive Learning from Complex Explanation Traces of GPT-4 arXiv 2023-06
SelFee: Iterative Self-Revising LLM Empowered by Self-Feedback Generation arXiv 2023-05

Preference

Title Venue Date Code Data
Ultrafeedback: Boosting language models with high-quality feedback arXiv 2023-10 Github Data
Zephyr: Direct Distillation of LM Alignment arXiv 2023-10 Github Data
Rlaif: Scaling Reinforcement Learning from Human Feedback with AI Feedback arXiv 2023-09
OPENCHAT: ADVANCING OPEN-SOURCE LANGUAGE MODELS WITH MIXED-QUALITY DATA ICLR 2023-09 Github Data
RLCD: Reinforcement Learning from Contrast Distillation for Language Model Alignment arXiv 2023-07 Github
Aligning Large Language Models through Synthetic Feedbacks EMNLP 2023-05 Github Data
Reward Design with Language Models ICLR 2023-03 Github
Training Language Models with Language Feedback at Scale arXiv 2023-03
Constitutional AI: Harmlessness from AI Feedback arXiv 2022-12

Value

Title Venue Date Code Data
Ultrafeedback: Boosting language models with high-quality feedback arXiv 2023-10 Github Data
RLCD: Reinforcement Learning from Contrast Distillation for Language Model Alignment arXiv 2023-07 Github
Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision NeurIPS 2023-05 Github Data
Training Socially Aligned Language Models on Simulated Social Interactions arXiv 2023-05
Constitutional AI: Harmlessness from AI Feedback arXiv 2022-12

Agent

Tool Using

Title Venue Date Code Data
Toolformer: Language Models Can Teach Themselves to Use Tools arXiv 2023-02
Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPT arXiv 2023-04 Github Data
Gorilla: Large Language Model Connected with Massive APIs arXiv 2023-05 Github Data
GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction arXiv 2023-05 Github Data
ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases arXiv 2023-06 Github Data
ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs arXiv 2023-07 Github Data
Confucius: Iterative Tool Learning from Introspection Feedback by Easy-to-Difficult Curriculum arXiv 2023-08 Github
CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsets arXiv 2023-09 Github
MLLM-Tool: A Multimodal Large Language Model For Tool Agent Learning arXiv 2024-01 Github Data
Small LLMs Are Weak Tool Learners: A Multi-LLM Agent arXiv 2024-01 Github
EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction arXiv 2024-01 Github

Planning

Title Venue Date Code Data
AUTOACT: Automatic Agent Learning from Scratch via Self-Planning arXiv 2024-01 Github
Lumos: Learning Agents with Unified Data, Modular Design, and Open-Source LLMs arXiv 2023-11 Github Data
TPTU-v2: Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Systems arXiv 2023-11
Embodied Multi-Modal Agent trained by an LLM from a Parallel TextWorld arXiv 2023-11
Accelerating Reinforcement Learning of Robotic Manipulations via Feedback from Large Language Models CoRL 2023-11
Motif: Intrinsic Motivation from Artificial Intelligence Feedback ICLR 2023-10 Github
FireAct: Toward Language Agent Fine-tuning arXiv 2023-10 Github Data
AgentTuning: Enabling Generalized Agent Abilities for LLMs arXiv 2023-10 Github
Eureka: Human-Level Reward Design via Coding Large Language Models arXiv 2023-10 Github
Language Instructed Reinforcement Learning for Human-AI Coordination PMLR 2023-04
Guiding Pretraining in Reinforcement Learning with Large Language Models PMLR 2023-02
Distilling Internet-Scale Vision-Language Models into Embodied Agents ICML 2023-01

NLP Task Specialization

NLU

Title Venue Date Code Data
LLM vs Small Model? Large Language Model Based Text Augmentation Enhanced Personality Detection Model arXiv 2024-03
Evolving Knowledge Distillation with Large Language Models and Active Learning arXiv 2024-03
Mixed Distillation Helps Smaller Language Model Better Reasoning arXiv 2023-12
PromptMix: A Class Boundary Augmentation Method for Large Language Model Distillation EMNLP 2023-10 Github
TinyLLM: Learning a Small Student from Multiple Large Language Models arXiv 2024-02
Targeted Data Generation: Finding and Fixing Model Weaknesses ACL 2023-05 Github
Distilling ChatGPT for Explainable Automated Student Answer Assessment arXiv 2023-05 Github
ChatGPT outperforms crowd workers for text-annotation tasks arXiv 2023-03
Annollm: Making large language models to be better crowdsourced annotators arXiv 2023-03
AugGPT: Leveraging ChatGPT for Text Data Augmentation arXiv 2023-02 Github
Is GPT-3 a Good Data Annotator? ACL 2022-12 Github
SunGen: Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learning ICLR 2022-05 Github
ZeroGen: Efficient Zero-shot Learning via Dataset Generation EMNLP 2022-02 Github
Generating Training Data with Language Models: Towards Zero-Shot Language Understanding NeurIPS 2022-02 Github
Towards Zero-Label Language Learning arXiv 2021-09
Generate, Annotate, and Learn: NLP with Synthetic Text TACL 2021-06

NLG

Title Venue Date Code Data
Tailoring Self-Rationalizers with Multi-Reward Distillation arXiv 2023-11 Github Data
RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation arXiv 2023-10 Github
Neural Machine Translation Data Generation and Augmentation using ChatGPT arXiv 2023-07
On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes ICLR 2023-06
Can LLMs generate high-quality synthetic note-oriented doctor-patient conversations? arXiv 2023-06 Github Data
InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPT EMNLP 2023-05
Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasing arXiv 2023-05 Github
Data Augmentation for Radiology Report Simplification Findings of EACL 2023-04 Github
Want To Reduce Labeling Cost? GPT-3 Can Help Findings of EMNLP 2021-08

Information Retrieval

Title Venue Date Code Data
InstructDistill: Instruction Distillation Makes Large Language Models Efficient Zero-shot Rankers arXiv 2023-11 Github Data
Soft prompt tuning for augmenting dense retrieval with large language models arXiv 2023-07 Github
Query Rewriting in Retrieval-Augmented Large Language Models EMNLP 2023-05
Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents EMNLP 2023-04 Github Data
AugTriever: Unsupervised Dense Retrieval by Scalable Data Augmentation arXiv 2022-12 Github
QUILL: Query Intent with Large Language Models using Retrieval Augmentation and Multi-stage Distillation EMNLP 2022-10
Promptagator: Few-shot Dense Retrieval From 8 Examples ICLR 2022-09
Questions Are All You Need to Train a Dense Passage Retrieval TACL 2022-06 Github
Improving Passage Retrieval with Zero-Shot Question Generation EMNLP 2022-04 Github Data
InPars: Data Augmentation for Information Retrieval using Large Language Models arXiv 2022-02 Github Data
Generating Datasets with Pretrained Language Models EMNLP 2021-04 Github

Recommendation

Title Venue Date Code Data
Can Small Language Models be Good Reasoners for Sequential Recommendation? arXiv 2024-03
Large Language Model Augmented Narrative Driven Recommendations arXiv 2023-06
Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach arXiv 2023-05
ONCE: Boosting Content-based Recommendation with Both Open- and Closed-source Large Language Models WSDM 2023-05 Github Data

Text Generation Evaluation

Title Venue Date Code Data
Prometheus: Inducing Fine-grained Evaluation Capability in Language Models ICLR 2023-10 Github Data
TIGERScore: Towards Building Explainable Metric for All Text Generation Tasks arXiv 2023-10 Github Data
Generative Judge for Evaluating Alignment ICLR 2023-10 Github Data
PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization arXiv 2023-06 Github Data
INSTRUCTSCORE: Explainable Text Generation Evaluation with Fine-grained Feedback EMNLP 2023-05 Github Data

Code

Title Venue Date Code Data
Magicoder: Source Code Is All You Need arXiv 2023-12 Github Data
Data
WaveCoder: Widespread And Versatile Enhanced Instruction Tuning with Refined Data Generation arXiv 2023-12
Instruction Fusion: Advancing Prompt Evolution through Hybridization arXiv 2023-12
MFTCoder: Boosting Code LLMs with Multitask Fine-Tuning arXiv 2023-11 Github Data
Data
LLM-Assisted Code Cleaning For Training Accurate Code Generators arXiv 2023-11
Personalised Distillation: Empowering Open-Sourced LLMs with Adaptive Learning for Code Generation EMNLP 2023-10 Github
Code Llama: Open Foundation Models for Code arXiv 2023-08 Github
Distilled GPT for Source Code Summarization arXiv 2023-08 Github Data
Textbooks Are All You Need: A Large-Scale Instructional Text Data Set for Language Models arXiv 2023-06
Code Alpaca: An Instruction-following LLaMA model for code generation - 2023-03 Github Data

Multi-Modality

Title Venue Date Code Data
Miko: Multimodal Intention Knowledge Distillation from Large Language Models for Social-Media Commonsense Discovery arXiv 2024-02
Localizing Visual Commonsense Knowledge in Large Language Models NeurIPS 2023-12 Github Data
To See is to Believe: Prompting GPT-4V for Better Visual Instruction Tuning arXiv 2023-11 Github Data
ILuvUI: Instruction-tuned LangUage-Vision modeling of UIs from Machine Conversations arXiv 2023-10
NExT-GPT: Any-to-Any Multimodal LLM arXiv 2023-09 Github Data
StableLLaVA: Enhanced Visual Instruction Tuning with Synthesized Image-Dialogue Data arXiv 2023-08 Github Data
PointLLM: Empowering Large Language Models to Understand Point Clouds arXiv 2023-08 Github Data
SVIT: Scaling up Visual Instruction Tuning arXiv 2023-07 Github Data
ChatSpot: Bootstrapping Multimodal LLMs via Precise Referring Instruction Tuning arXiv 2023-07
Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic arXiv 2023-06 Github Data
Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning ICLR 2023-06 Github Data
Valley: Video Assistant with Large Language model Enhanced abilitY arXiv 2023-06 Github Data
DetGPT: Detect What You Need via Reasoning EMNLP 2023-05 Github
Visual Instruction Tuning: A Comprehensive Study of Visual Instruction Tuning for Large Language Models NeurIPS 2023-04 Github Data

Summary Table


Figure: A summary of representative works about skill distillation.

Verticalization Distillation

Law

Title Venue Date Code Data
Fuzi - 2023-08 Github
ChatLaw: Open-Source Legal Large Language Model with Integrated External Knowledge Bases arXiv 2023-06 Github
Lawyer LLaMA Technical Report arXiv 2023-05 Github Data

Medical & Healthcare

Title Venue Date Code Data
HuatuoGPT-II, One-stage Training for Medical Adaption of LLMs arXiv 2023-11 Github Data
AlpaCare: Instruction-tuned large language models for medical application arXiv 2023-10 Github Data
DISC-MedLLM: Bridging General Large Language Models and Real-World Medical Consultation arXiv 2023-08 Github Data
HuatuoGPT: Taming Language Model to Be a Doctor EMNLP 2023-05 Github Data
DoctorGLM: Fine-tuning your Chinese doctor is not a herculean task arXiv 2023-04 Github Data
Huatuo: Tuning LLM with Chinese Medical Knowledge arXiv 2023-04 Github
MedAlpaca: An Open-Source Collection of Medical Conversational AI Models and Training Data arXiv 2023-04 Github Data
PMC-LLaMA: Further Finetuning LLaMA on Medical Papers arXiv 2023-04 Github Data
ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge arXiv 2023-03 Github

Finance

Title Venue Date Code Data
XuanYuan 2.0: A Large Chinese Financial Chat Model with Hundreds of Billions Parameters CIKM 2023-05

Science

Title Venue Date Code Data
MuseGraph: Graph-oriented Instruction Tuning of Large Language Models for Generic Graph Mining arXiv 2024-03
SciGLM: Training Scientific Language Models with Self-Reflective Instruction Annotation and Tuning arXiv 2024-01 Github
AstroLLaMA-Chat: Scaling AstroLLaMA with Conversational and Diverse Datasets arXiv 2024-01
GeoGalactica: A Scientific Large Language Model in Geoscience arXiv 2024-01 Github Data
InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discovery arXiv 2023-11 Github
LLM-Prop: Predicting Physical And Electronic Properties Of Crystalline Solids From Their Text Descriptions arXiv 2023-10 Github
OceanGPT: A Large Language Model for Ocean Science Tasks arXiv 2023-10 Github Data
MarineGPT: Unlocking Secrets of Ocean to the Public arXiv 2023-10 Github
Mammoth: Building Math Generalist Models through Hybrid Instruction Tuning arXiv 2023-09 Github Data
ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving ICLR 2023-09 Github
DARWIN Series: Domain Specific Large Language Models for Natural Science arXiv 2023-08 Github
Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct arXiv 2023-08 Github
Biomedgpt: Open Multimodal Generative Pre-trained Transformer for Biomedicine arXiv 2023-08 Github Data
Prot2Text: Multimodal Protein’s Function Generation with GNNs and Transformers NeurIPS 2023-07
xTrimoPGLM: Unified 100B-Scale Pre-trained Transformer for Deciphering the Language of Protein bioRxiv 2023-07
GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learning NeurIPS 2023-06 Github Data
K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization arXiv 2023-06 Github
Visual Instruction Tuning: A Comprehensive Study of Visual Instruction Tuning for Large Language Models NeurIPS 2023-04 Github Data

Misc.

Title Venue Date Code Data
OWL: A Large Language Model for IT Operations arXiv 2023-09 Github Data
EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Education arXiv 2023-08 Github Data

Encoder-based KD

Note: Our survey mainly focuses on generative LLMs, and thus the encoder-based KD is not included in the survey. However, we are also interested in this topic and continue to update the latest works in this area.

Title Venue Date Code Data
Masked Latent Semantic Modeling: an Efficient Pre-training Alternative to Masked Language Modeling Findings of ACL 2023-08
Better Together: Jointly Using Masked Latent Semantic Modeling and Masked Language Modeling for Sample Efficient Pre-training CoNLL 2023-08

TODO List

  • [ ] Add works about O1-like distillation. Stay tuned!

Citation

If you find this repository helpful, please consider citing the following paper:

@misc{xu2024survey,
      title={A Survey on Knowledge Distillation of Large Language Models}, 
      author={Xiaohan Xu and Ming Li and Chongyang Tao and Tao Shen and Reynold Cheng and Jinyang Li and Can Xu and Dacheng Tao and Tianyi Zhou},
      year={2024},
      eprint={2402.13116},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

Star History

Star History Chart

For Tasks:

Click tags to check more tools for each tasks
improve smaller models compress models enhance cognitive abilities align models generate synthetic data

For Jobs:

data scientist machine learning engineer research scientist ai engineer nlp specialist

Alternative AI tools for Awesome-Knowledge-Distillation-of-LLMs

Similar Open Source Tools

Awesome-Knowledge-Distillation-of-LLMs Screenshot
Awesome-Knowledge-Distillation-of-LLMs

A collection of papers related to knowledge distillation of large language models (LLMs). The repository focuses on techniques to transfer advanced capabilities from proprietary LLMs to smaller models, compress open-source LLMs, and refine their performance. It covers various aspects of knowledge distillation, including algorithms, skill distillation, verticalization distillation in fields like law, medical & healthcare, finance, science, and miscellaneous domains. The repository provides a comprehensive overview of the research in the area of knowledge distillation of LLMs.

github
: 890
Awesome-LLMs-for-Video-Understanding Screenshot
Awesome-LLMs-for-Video-Understanding

Awesome-LLMs-for-Video-Understanding is a repository dedicated to exploring Video Understanding with Large Language Models. It provides a comprehensive survey of the field, covering models, pretraining, instruction tuning, and hybrid methods. The repository also includes information on tasks, datasets, and benchmarks related to video understanding. Contributors are encouraged to add new papers, projects, and materials to enhance the repository.

github
: 1.8k
Awesome-Interpretability-in-Large-Language-Models Screenshot
Awesome-Interpretability-in-Large-Language-Models

This repository is a collection of resources focused on interpretability in large language models (LLMs). It aims to help beginners get started in the area and keep researchers updated on the latest progress. It includes libraries, blogs, tutorials, forums, tools, programs, papers, and more related to interpretability in LLMs.

github
: 230
Awesome-Model-Merging-Methods-Theories-Applications Screenshot
Awesome-Model-Merging-Methods-Theories-Applications

A comprehensive repository focusing on 'Model Merging in LLMs, MLLMs, and Beyond', providing an exhaustive overview of model merging methods, theories, applications, and future research directions. The repository covers various advanced methods, applications in foundation models, different machine learning subfields, and tasks like pre-merging methods, architecture transformation, weight alignment, basic merging methods, and more.

github
: 519
Awesome-Jailbreak-on-LLMs Screenshot
Awesome-Jailbreak-on-LLMs

Awesome-Jailbreak-on-LLMs is a collection of state-of-the-art, novel, and exciting jailbreak methods on Large Language Models (LLMs). The repository contains papers, codes, datasets, evaluations, and analyses related to jailbreak attacks on LLMs. It serves as a comprehensive resource for researchers and practitioners interested in exploring various jailbreak techniques and defenses in the context of LLMs. Contributions such as additional jailbreak-related content, pull requests, and issue reports are welcome, and contributors are acknowledged. For any inquiries or issues, contact [email protected]. If you find this repository useful for your research or work, consider starring it to show appreciation.

github
: 507
MOSS-TTS Screenshot
MOSS-TTS

MOSS-TTS Family is an open-source speech and sound generation model family designed for high-fidelity, high-expressiveness, and complex real-world scenarios. It includes five production-ready models: MOSS-TTS, MOSS-TTSD, MOSS-VoiceGenerator, MOSS-TTS-Realtime, and MOSS-SoundEffect, each serving specific purposes in speech generation, dialogue, voice design, real-time interactions, and sound effect generation. The models offer features like long-speech generation, fine-grained control over phonemes and duration, multilingual synthesis, voice cloning, and real-time voice agents.

github
: 256
DataFlow Screenshot
DataFlow

DataFlow is a data preparation and training system designed to parse, generate, process, and evaluate high-quality data from noisy sources, improving the performance of large language models in specific domains. It constructs diverse operators and pipelines, validated to enhance domain-oriented LLM's performance in fields like healthcare, finance, and law. DataFlow also features an intelligent DataFlow-agent capable of dynamically assembling new pipelines by recombining existing operators on demand.

github
: 2.9k
Awesome-Tabular-LLMs Screenshot
Awesome-Tabular-LLMs

This repository is a collection of papers on Tabular Large Language Models (LLMs) specialized for processing tabular data. It includes surveys, models, and applications related to table understanding tasks such as Table Question Answering, Table-to-Text, Text-to-SQL, and more. The repository categorizes the papers based on key ideas and provides insights into the advancements in using LLMs for processing diverse tables and fulfilling various tabular tasks based on natural language instructions.

github
: 151
MMOS Screenshot
MMOS

MMOS (Mix of Minimal Optimal Sets) is a dataset designed for math reasoning tasks, offering higher performance and lower construction costs. It includes various models and data subsets for tasks like arithmetic reasoning and math word problem solving. The dataset is used to identify minimal optimal sets through reasoning paths and statistical analysis, with a focus on QA-pairs generated from open-source datasets. MMOS also provides an auto problem generator for testing model robustness and scripts for training and inference.

github
: 61
VoiceBench Screenshot
VoiceBench

VoiceBench is a repository containing code and data for benchmarking LLM-Based Voice Assistants. It includes a leaderboard with rankings of various voice assistant models based on different evaluation metrics. The repository provides setup instructions, datasets, evaluation procedures, and a curated list of awesome voice assistants. Users can submit new voice assistant results through the issue tracker for updates on the ranking list.

github
: 119
Awesome-LLM-Constrained-Decoding Screenshot
Awesome-LLM-Constrained-Decoding

Awesome-LLM-Constrained-Decoding is a curated list of papers, code, and resources related to constrained decoding of Large Language Models (LLMs). The repository aims to facilitate reliable, controllable, and efficient generation with LLMs by providing a comprehensive collection of materials in this domain.

github
: 180
AI-Competition-Collections Screenshot
AI-Competition-Collections

AI-Competition-Collections is a repository that collects and curates various experiences and tips from AI competitions. It includes posts on competition experiences in computer vision, NLP, speech, and other AI-related fields. The repository aims to provide valuable insights and techniques for individuals participating in AI competitions, covering topics such as image classification, object detection, OCR, adversarial attacks, and more.

github
: 365
ZhiLight Screenshot
ZhiLight

ZhiLight is a highly optimized large language model (LLM) inference engine developed by Zhihu and ModelBest Inc. It accelerates the inference of models like Llama and its variants, especially on PCIe-based GPUs. ZhiLight offers significant performance advantages compared to mainstream open-source inference engines. It supports various features such as custom defined tensor and unified global memory management, optimized fused kernels, support for dynamic batch, flash attention prefill, prefix cache, and different quantization techniques like INT8, SmoothQuant, FP8, AWQ, and GPTQ. ZhiLight is compatible with OpenAI interface and provides high performance on mainstream NVIDIA GPUs with different model sizes and precisions.

github
: 832
CogVLM2 Screenshot
CogVLM2

CogVLM2 is a new generation of open source models that offer significant improvements in benchmarks such as TextVQA and DocVQA. It supports 8K content length, image resolution up to 1344 * 1344, and both Chinese and English languages. The project provides basic calling methods, fine-tuning examples, and OpenAI API format calling examples to help developers quickly get started with the model.

github
: 83
END-TO-END-GENERATIVE-AI-PROJECTS Screenshot
END-TO-END-GENERATIVE-AI-PROJECTS

The 'END TO END GENERATIVE AI PROJECTS' repository is a collection of awesome industry projects utilizing Large Language Models (LLM) for various tasks such as chat applications with PDFs, image to speech generation, video transcribing and summarizing, resume tracking, text to SQL conversion, invoice extraction, medical chatbot, financial stock analysis, and more. The projects showcase the deployment of LLM models like Google Gemini Pro, HuggingFace Models, OpenAI GPT, and technologies such as Langchain, Streamlit, LLaMA2, LLaMAindex, and more. The repository aims to provide end-to-end solutions for different AI applications.

github
: 145
Prompt-Engineering-Holy-Grail Screenshot
Prompt-Engineering-Holy-Grail

The Prompt Engineering Holy Grail repository is a curated resource for prompt engineering enthusiasts, providing essential resources, tools, templates, and best practices to support learning and working in prompt engineering. It covers a wide range of topics related to prompt engineering, from beginner fundamentals to advanced techniques, and includes sections on learning resources, online courses, books, prompt generation tools, prompt management platforms, prompt testing and experimentation, prompt crafting libraries, prompt libraries and datasets, prompt engineering communities, freelance and job opportunities, contributing guidelines, code of conduct, support for the project, and contact information.

github
: 366

For similar tasks

Awesome-Knowledge-Distillation-of-LLMs Screenshot
Awesome-Knowledge-Distillation-of-LLMs

A collection of papers related to knowledge distillation of large language models (LLMs). The repository focuses on techniques to transfer advanced capabilities from proprietary LLMs to smaller models, compress open-source LLMs, and refine their performance. It covers various aspects of knowledge distillation, including algorithms, skill distillation, verticalization distillation in fields like law, medical & healthcare, finance, science, and miscellaneous domains. The repository provides a comprehensive overview of the research in the area of knowledge distillation of LLMs.

github
: 890
rlhf_thinking_model Screenshot
rlhf_thinking_model

This repository is a collection of research notes and resources focusing on training large language models (LLMs) and Reinforcement Learning from Human Feedback (RLHF). It includes methodologies, techniques, and state-of-the-art approaches for optimizing preferences and model alignment in LLM training. The purpose is to serve as a reference for researchers and engineers interested in reinforcement learning, large language models, model alignment, and alternative RL-based methods.

github
: 76
aimet Screenshot
aimet

AIMET is a library that provides advanced model quantization and compression techniques for trained neural network models. It provides features that have been proven to improve run-time performance of deep learning neural network models with lower compute and memory requirements and minimal impact to task accuracy. AIMET is designed to work with PyTorch, TensorFlow and ONNX models. We also host the AIMET Model Zoo - a collection of popular neural network models optimized for 8-bit inference. We also provide recipes for users to quantize floating point models using AIMET.

github
: 2.5k
hqq Screenshot
hqq

HQQ is a fast and accurate model quantizer that skips the need for calibration data. It's super simple to implement (just a few lines of code for the optimizer). It can crunch through quantizing the Llama2-70B model in only 4 minutes! 🚀

github
: 879
llm-resource Screenshot
llm-resource

llm-resource is a comprehensive collection of high-quality resources for Large Language Models (LLM). It covers various aspects of LLM including algorithms, training, fine-tuning, alignment, inference, data engineering, compression, evaluation, prompt engineering, AI frameworks, AI basics, AI infrastructure, AI compilers, LLM application development, LLM operations, AI systems, and practical implementations. The repository aims to gather and share valuable resources related to LLM for the community to benefit from.

github
: 309
llmc Screenshot
llmc

llmc is an off-the-shell tool designed for compressing LLM, leveraging state-of-the-art compression algorithms to enhance efficiency and reduce model size without compromising performance. It provides users with the ability to quantize LLMs, choose from various compression algorithms, export transformed models for further optimization, and directly infer compressed models with a shallow memory footprint. The tool supports a range of model types and quantization algorithms, with ongoing development to include pruning techniques. Users can design their configurations for quantization and evaluation, with documentation and examples planned for future updates. llmc is a valuable resource for researchers working on post-training quantization of large language models.

github
: 430
Awesome-Efficient-LLM Screenshot
Awesome-Efficient-LLM

Awesome-Efficient-LLM is a curated list focusing on efficient large language models. It includes topics such as knowledge distillation, network pruning, quantization, inference acceleration, efficient MOE, efficient architecture of LLM, KV cache compression, text compression, low-rank decomposition, hardware/system, tuning, and survey. The repository provides a collection of papers and projects related to improving the efficiency of large language models through various techniques like sparsity, quantization, and compression.

github
: 1.6k
TensorRT-Model-Optimizer Screenshot
TensorRT-Model-Optimizer

The NVIDIA TensorRT Model Optimizer is a library designed to quantize and compress deep learning models for optimized inference on GPUs. It offers state-of-the-art model optimization techniques including quantization and sparsity to reduce inference costs for generative AI models. Users can easily stack different optimization techniques to produce quantized checkpoints from torch or ONNX models. The quantized checkpoints are ready for deployment in inference frameworks like TensorRT-LLM or TensorRT, with planned integrations for NVIDIA NeMo and Megatron-LM. The tool also supports 8-bit quantization with Stable Diffusion for enterprise users on NVIDIA NIM. Model Optimizer is available for free on NVIDIA PyPI, and this repository serves as a platform for sharing examples, GPU-optimized recipes, and collecting community feedback.

github
: 1.4k

For similar jobs

sweep Screenshot
sweep

Sweep is an AI junior developer that turns bugs and feature requests into code changes. It automatically handles developer experience improvements like adding type hints and improving test coverage.

github
: 7.1k
teams-ai Screenshot
teams-ai

The Teams AI Library is a software development kit (SDK) that helps developers create bots that can interact with Teams and Microsoft 365 applications. It is built on top of the Bot Framework SDK and simplifies the process of developing bots that interact with Teams' artificial intelligence capabilities. The SDK is available for JavaScript/TypeScript, .NET, and Python.

github
: 502
ai-guide Screenshot
ai-guide

This guide is dedicated to Large Language Models (LLMs) that you can run on your home computer. It assumes your PC is a lower-end, non-gaming setup.

github
: 159
classifai Screenshot
classifai

Supercharge WordPress Content Workflows and Engagement with Artificial Intelligence. Tap into leading cloud-based services like OpenAI, Microsoft Azure AI, Google Gemini and IBM Watson to augment your WordPress-powered websites. Publish content faster while improving SEO performance and increasing audience engagement. ClassifAI integrates Artificial Intelligence and Machine Learning technologies to lighten your workload and eliminate tedious tasks, giving you more time to create original content that matters.

github
: 697
chatbot-ui Screenshot
chatbot-ui

Chatbot UI is an open-source AI chat app that allows users to create and deploy their own AI chatbots. It is easy to use and can be customized to fit any need. Chatbot UI is perfect for businesses, developers, and anyone who wants to create a chatbot.

github
: 27.7k
BricksLLM Screenshot
BricksLLM

BricksLLM is a cloud native AI gateway written in Go. Currently, it provides native support for OpenAI, Anthropic, Azure OpenAI and vLLM. BricksLLM aims to provide enterprise level infrastructure that can power any LLM production use cases. Here are some use cases for BricksLLM: * Set LLM usage limits for users on different pricing tiers * Track LLM usage on a per user and per organization basis * Block or redact requests containing PIIs * Improve LLM reliability with failovers, retries and caching * Distribute API keys with rate limits and cost limits for internal development/production use cases * Distribute API keys with rate limits and cost limits for students

github
: 953
uAgents Screenshot
uAgents

uAgents is a Python library developed by Fetch.ai that allows for the creation of autonomous AI agents. These agents can perform various tasks on a schedule or take action on various events. uAgents are easy to create and manage, and they are connected to a fast-growing network of other uAgents. They are also secure, with cryptographically secured messages and wallets.

github
: 1.3k
griptape Screenshot
griptape

Griptape is a modular Python framework for building AI-powered applications that securely connect to your enterprise data and APIs. It offers developers the ability to maintain control and flexibility at every step. Griptape's core components include Structures (Agents, Pipelines, and Workflows), Tasks, Tools, Memory (Conversation Memory, Task Memory, and Meta Memory), Drivers (Prompt and Embedding Drivers, Vector Store Drivers, Image Generation Drivers, Image Query Drivers, SQL Drivers, Web Scraper Drivers, and Conversation Memory Drivers), Engines (Query Engines, Extraction Engines, Summary Engines, Image Generation Engines, and Image Query Engines), and additional components (Rulesets, Loaders, Artifacts, Chunkers, and Tokenizers). Griptape enables developers to create AI-powered applications with ease and efficiency.

github
: 2.2k