Awesome-Model-Merging-Methods-Theories-Applications
Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities. arXiv:2408.07666.
Stars: 158
A comprehensive repository focusing on 'Model Merging in LLMs, MLLMs, and Beyond', providing an exhaustive overview of model merging methods, theories, applications, and future research directions. The repository covers various advanced methods, applications in foundation models, different machine learning subfields, and tasks like pre-merging methods, architecture transformation, weight alignment, basic merging methods, and more.
README:
A comprehensive list of papers about 'Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities. Arxiv, 2024.'.
[!IMPORTANT] If you have a relevant paper not included in the library, or have any clarification about the content of the paper, please contact us!
- 🔥🔥🔥 We marked the papers that used model size $\geq$ 7B in experiments.
Model merging is an efficient empowerment technique in the machine learning community that does not require the collection of raw training data and does not require expensive computation. As model merging becomes increasingly prevalent across various fields, it is crucial to understand the available model merging techniques comprehensively. However, there is a significant gap in the literature regarding a systematic and thorough review of these techniques. To address this gap, this survey provides a comprehensive overview of model merging methods and theories, their applications in various domains and settings, and future research directions. Specifically, we first propose a new taxonomic approach that exhaustively discusses existing model merging methods. Secondly, we discuss the application of model merging techniques in large language models, multimodal large language models, and 10+ machine learning subfields, including continual learning, multi-task learning, few-shot learning, etc. Finally, we highlight the remaining challenges of model merging and discuss future research directions.
If you find our paper or this resource helpful, please consider cite:
@article{Survery_ModelMerging_2024,
title={Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities},
author={Yang, Enneng and Shen, Li and Guo, Guibing and Wang, Xingwei and Cao, Xiaochun and Zhang, Jie and Tao, Dacheng},
journal={arXiv preprint arXiv:2408.07666},
year={2024}
}
Thanks!
-
Awesome-Model-Merging-Methods-Theories-Applications
- Survey
- Benchmark/Evaluation
- Advanced Methods
- Application of Model Merging in Foundation Models
- Application of Model Merging in Different Machine Learning Subfields
- Other Applications
| Paper Title | Year | Conference/Journal |
|---|---|---|
| Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities | 2024 | Arxiv |
| A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning | 2024 | Arxiv |
| Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models | 2024 | Arxiv |
| Learn From Model Beyond Fine-Tuning: A Survey | 2023 | Arxiv |
| Deep Model Fusion: A Survey | 2023 | Arxiv |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| Model-GLUE: Democratized LLM Scaling for A Large Model Zoo in the Wild | 2024 | NeurIPS Track on Datasets and Benchmarks | Synthia-7B-v1.2, Llama-2-7b-evolcodealpaca, OpenHermes-7B, pygmalion-2-7b, Llama-2-7b-chat-hf, BeingWell_llama2_7b, MetaMath-7B-V1.0, vicuna-7b-v1.5, Platypus2-7B, GOAT-7B-Community, Llama-2-7b-WikiChat-fused, dolphin-llama2-7b, MetaMath-Llemma-7B, CodeLlama-7b-Instruct-hf, Magicoder-S-CL-7B , CrystalChat |
| What Matters for Model Merging at Scale? | 2024 | Arxiv | PaLM-2 (1B, 8B, 24B, 64B), PaLM-2-IT (1B, 8B, 24B, 64B) |
| Realistic Evaluation of Model Merging for Compositional Generalization | 2024 | Arxiv | |
| Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilities | 2024 | Arxiv | Llama-3.1-8B, Mistral-7B-v0.3 |
| FusionBench: A Comprehensive Benchmark of Deep Model Fusion | 2024 | Arxiv | |
| Arcee's MergeKit: A Toolkit for Merging Large Language Models | 2024 | Arxiv | Llama2-7B-Chat, Meditron-7B |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| Fine-Tuning Linear Layers Only Is a Simple yet Effective Way for Task Arithmetic | 2024 | Arxiv | |
| Tangent Transformers for Composition,Privacy and Removal | 2024 | ICLR | |
| Parameter Efficient Multi-task Model Fusion with Partial Linearization | 2024 | ICLR | |
| Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models | 2023 | NeurIPS |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| Knowledge fusion of large language models | 2024 | ICLR | Llama-2 7B, OpenLLaMA 7B, MPT 7B |
| Knowledge Fusion of Chat LLMs: A Preliminary Technical Report | 2024 | Arxiv | NH2-Mixtral-8x7B, NH2-Solar-10.7B, and OpenChat-3.5-7B |
| On Cross-Layer Alignment for Model Fusion of Heterogeneous Neural Networks | 2023 | ICASSP | |
| GAN Cocktail: mixing GANs without dataset access | 2022 | ECCV |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| Composing parameter-efficient modules with arithmetic operation | 2023 | NeurIPS | |
| Editing models with task arithmetic | 2023 | ICLR | |
| Model fusion via optimal transport | 2020 | NeurIPS | |
| Weight averaging for neural networks and local resampling schemes | 1996 | AAAI Workshop |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| Merging Multi-Task Models via Weight-Ensembling Mixture of Experts | 2024 | ICML | |
| Learning to Route Among Specialized Experts for Zero-Shot Generalization | 2024 | ICML | |
| Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy | 2024 | ICLR | |
| Soft merging of experts with adaptive routing | 2024 | TMLR | |
| SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models | 2024 | Arxiv | Mistral-7B-v0.1, MetaMath-Mistral-7B, dolphin-2.1-mistral-7b, speechless-code-mistral-7b-v1.0 |
| Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging | 2024 | Arxiv | Qwen-14B |
| Self-MoE: Towards Compositional Large Language Models with Self-Specialized Experts | 2024 | Arxiv | Gemma-7B, LLaMA-2 7B & 13B, Mistral 7B, LLaMA-3 8B |
| Towards Efficient Pareto Set Approximation via Mixture of Experts Based Model Fusion | 2024 | Arxiv | |
| Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints | 2023 | ICLR |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| Representation Surgery for Multi-Task Model Merging | 2024 | ICML |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| HM3: Hierarchical Multi-Objective Model Merging for Pretrained Models | 2024 | Arxiv | Llama-2-7B-Chat, WizardMath-7B, CodeLlama-7B |
| Weight Scope Alignment: A Frustratingly Easy Method for Model Merging | 2024 | Arxiv | |
| It’s Morphing Time: Unleashing the Potential of Multiple LLMs via Multi-objective Optimization | 2024 | Arxiv | Qwen1.5-7B-Chat, Liberated-Qwen1.5-7B, firefly-qwen1.5-en-7B |
| SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling | 2023 | Arxiv | SOLAR 10.7B, SOLAR 10.7B-Instruct |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| Separate the Wheat from the Chaff: Model Deficiency Unlearning via Parameter-Efficient Module Operation | 2024 | AAAI | LLaMA-7B |
| Mitigating Social Biases in Language Models through Unlearning | 2024 | Arxiv | LLaMA-2 7B |
| Fine-Grained Detoxification via Instance-Level Prefixes for Large Language Models | 2024 | Arxiv | Llama-2-7B, Llama-2-chat-7B, Vicuna-7B, Llama-2-13B |
| Composing Parameter-Efficient Modules with Arithmetic Operation | 2023 | NeurIPS | |
| Editing models with task arithmetic | 2023 | ICLR | |
| Elastic Weight Removal for Faithful and Abstractive Dialogue Generation | 2023 | Arxiv |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| NegMerge: Consensual Weight Negation for Strong Machine Unlearning | 2024 | Arxiv | |
| Strong Copyright Protection for Language Models via Adaptive Model Fusion | 2024 | ICML | LLaMa2 7B, StarCoder 7B |
| Avoiding Copyright Infringement via Machine Unlearning | 2024 | Arxiv | Llama3-8B |
| Towards Safer Large Language Models through Machine Unlearning | 2024 | ACL | LLAMA2-7B, LLAMA2-13B |
| Editing models with task arithmetic | 2023 | ICLR | |
| Forgetting before Learning: Utilizing Parametric Arithmetic for Knowledge Updating in Large Language Model | 2023 | Arxiv | LLAMA2-7B, LLAMA-7B, BLOOM-7B |
| Fuse to Forget: Bias Reduction and Selective Memorization through Model Fusion | 2023 | Arxiv |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| DEM: Distribution Edited Model for Training with Mixed Data Distributions | 2024 | Arxiv | OpenLLaMA 7B and 13B |
| Checkpoint Merging via Bayesian Optimization in LLM Pretraining | 2024 | Arxiv | Baichuan2-220B, Baichuan2-440B, Baichuan2-660B, Baichuan2-1540B, Baichuan2-1760B, Baichuan2-1980B, Baichuan2-2200B, Baichuan2-2420B, DeepSeek-1400B, DeepSeek-1600B, DeepSeek-1800B, DeepSeek-2000B |
| ColD Fusion: Collaborative Descent for Distributed Multitask Finetuning | 2023 | ACL | |
| Early Weight Averaging meets High Learning Rates for LLM Pre-training | 2023 | NeurIPS Workshop | |
| Stop wasting my time! saving days of imagenet and bert training with latest weight averaging | 2022 | NeurIPS Workshop | |
| Fusing finetuned models for better pretraining | 2022 | Arxiv |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| Jointly training large autoregressive multimodal models | 2024 | ICLR | |
| Model Composition for Multimodal Large Language Models | 2024 | ACL | Vicuna-7B-v1.5 |
| π-Tuning: Transferring Multimodal Foundation Models with Optimal Multi-task Interpolation | 2023 | ICML | |
| An Empirical Study of Multimodal Model Merging | 2023 | EMNLP | |
| UnIVAL: Unified Model for Image, Video, Audio and Language Tasks | 2023 | TMLR |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| Multimodal Attention Merging for Improved Speech Recognition and Audio Event Classification | 2024 | ICASSP Workshop |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| Diffusion Soup: Model Merging for Text-to-Image Diffusion Models | 2024 | Arxiv | |
| MaxFusion: Plug&Play Multi-Modal Generation in Text-to-Image Diffusion Models | 2024 | Arxiv | |
| MoLE: Mixture of LoRA Experts | 2024 | ICLR | |
| LoRA-Composer: Leveraging Low-Rank Adaptation for Multi-Concept Customization in Training-Free Diffusion Models | 2024 | Arxiv | |
| Multi-LoRA Composition for Image Generation | 2024 | Arxiv | |
| Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models | 2023 | NeurIPS | |
| Merging loras | 2023 | (github) | |
| ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAs | 2023 | Arxiv | |
| GAN Cocktail: mixing GANs without dataset access | 2022 | ECCV |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| Linear Combination of Saved Checkpoints Makes Consistency and Diffusion Models Better | 2024 | Arxiv | |
| A Unified Module for Accelerating STABLE-DIFFUSION: LCM-LORA | 2024 | Arxiv |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| Decouple-Then-Merge: Towards Better Training for Diffusion Models | 2024 | Arxiv | |
| SELMA: Learning and Merging Skill-Specific Text-to-Image Experts with Auto-Generated Data | 2024 | Arxiv |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| You Only Merge Once: Learning the Pareto Set of Preference-Aware Model Merging | 2024 | Arxiv | |
| Towards Efficient Pareto Set Approximation via Mixture of Experts Based Model Fusion | 2024 | Arxiv | |
| MAP: Low-compute Model Merging with Amortized Pareto Fronts via Quadratic Approximation | 2024 | Arxiv | Llama3-8B |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| DEM: Distribution Edited Model for Training with Mixed Data Distributions | 2024 | Arxiv | OpenLLaMA-7B, OpenLLaMA-13B |
| Merging Vision Transformers from Different Tasks and Domains | 2023 | Arxiv |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learning | 2023 | NeurIPS |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| Realistic Evaluation of Model Merging for Compositional Generalization | 2024 | Arxiv | |
| Layer-wise Model Merging for Unsupervised Domain Adaptation in Segmentation Tasks | 2024 | Arxiv | |
| Training-Free Model Merging for Multi-target Domain Adaptation | 2024 | Arxiv | |
| Domain Adaptation of Llama3-70B-Instruct through Continual Pre-Training and Model Merging: A Comprehensive Evaluation | 2024 | Arxiv | Llama3-70B |
| Ensemble of averages: Improving model selection and boosting performance in domain generalization | 2022 | NeurIPS | |
| Swad: Domain generalization by seeking flat minima | 2021 | NeurIPS |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| LoRA-Flow: Dynamic LoRA Fusion for Large Language Models in Generative Tasks | 2024 | ACL | Llama-2- 7B |
| LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition | 2024 | COLM | Llama-2-7B, Llama-2-13B |
| LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wild | 2024 | ACL | |
| Does Combining Parameter-efficient Modules Improve Few-shot Transfer Accuracy? | 2024 | Arxiv | |
| MerA: Merging pretrained adapters for few-shot learning | 2023 | Arxiv |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| BadMerging: Backdoor Attacks Against Model Merging | 2024 | CCS | |
| LoRA-as-an-Attack! Piercing LLM Safety Under The Share-and-Play Scenario | 2024 | ACL | Llama-2-7B |
| Paper Title | Year | Conference/Journal | Remark |
|---|---|---|---|
| MergePrint: Robust Fingerprinting against Merging Large Language Models | 2024 | Arxiv | LLaMA-2-7B, WizardMath-7B-V1.0, LLaMA-2-7B-CHAT |
| Here’s a Free Lunch: Sanitizing Backdoored Models with Model Merge | 2024 | ACL | |
| Merging Improves Self-Critique Against Jailbreak Attacks | 2024 | Arxiv | Mistral-7B, Mixtral-8x7B |
| Have You Merged My Model? On The Robustness of Large Language Model IP Protection Methods Against Model Merging | 2024 | Arxiv | LLaMA-2-7B, LLaMA-2-7B-CHAT, WizardMath-7B-V1.0 |
| Revisiting adapters with adversarial training | 2023 | ICLR | |
| Seasoning model soups for robustness to adversarial and natural distribution shifts | 2023 | CVPR |
Star History
We welcome all researchers to contribute to this repository 'model merging in foundation models or machine learning'.
If you have a related paper that was not added to the library, please contact us.
Email: [email protected] / [email protected]
For Tasks:
Click tags to check more tools for each tasksFor Jobs:
Alternative AI tools for Awesome-Model-Merging-Methods-Theories-Applications
Similar Open Source Tools
Awesome-Model-Merging-Methods-Theories-Applications
A comprehensive repository focusing on 'Model Merging in LLMs, MLLMs, and Beyond', providing an exhaustive overview of model merging methods, theories, applications, and future research directions. The repository covers various advanced methods, applications in foundation models, different machine learning subfields, and tasks like pre-merging methods, architecture transformation, weight alignment, basic merging methods, and more.
Awesome-LLMs-for-Video-Understanding
Awesome-LLMs-for-Video-Understanding is a repository dedicated to exploring Video Understanding with Large Language Models. It provides a comprehensive survey of the field, covering models, pretraining, instruction tuning, and hybrid methods. The repository also includes information on tasks, datasets, and benchmarks related to video understanding. Contributors are encouraged to add new papers, projects, and materials to enhance the repository.
Awesome-Knowledge-Distillation-of-LLMs
A collection of papers related to knowledge distillation of large language models (LLMs). The repository focuses on techniques to transfer advanced capabilities from proprietary LLMs to smaller models, compress open-source LLMs, and refine their performance. It covers various aspects of knowledge distillation, including algorithms, skill distillation, verticalization distillation in fields like law, medical & healthcare, finance, science, and miscellaneous domains. The repository provides a comprehensive overview of the research in the area of knowledge distillation of LLMs.
Awesome-Tabular-LLMs
This repository is a collection of papers on Tabular Large Language Models (LLMs) specialized for processing tabular data. It includes surveys, models, and applications related to table understanding tasks such as Table Question Answering, Table-to-Text, Text-to-SQL, and more. The repository categorizes the papers based on key ideas and provides insights into the advancements in using LLMs for processing diverse tables and fulfilling various tabular tasks based on natural language instructions.
CogVLM2
CogVLM2 is a new generation of open source models that offer significant improvements in benchmarks such as TextVQA and DocVQA. It supports 8K content length, image resolution up to 1344 * 1344, and both Chinese and English languages. The project provides basic calling methods, fine-tuning examples, and OpenAI API format calling examples to help developers quickly get started with the model.
LLM4Opt
LLM4Opt is a collection of references and papers focusing on applying Large Language Models (LLMs) for diverse optimization tasks. The repository includes research papers, tutorials, workshops, competitions, and related collections related to LLMs in optimization. It covers a wide range of topics such as algorithm search, code generation, machine learning, science, industry, and more. The goal is to provide a comprehensive resource for researchers and practitioners interested in leveraging LLMs for optimization tasks.
AudioLLM
AudioLLMs is a curated collection of research papers focusing on developing, implementing, and evaluating language models for audio data. The repository aims to provide researchers and practitioners with a comprehensive resource to explore the latest advancements in AudioLLMs. It includes models for speech interaction, speech recognition, speech translation, audio generation, and more. Additionally, it covers methodologies like multitask audioLLMs and segment-level Q-Former, as well as evaluation benchmarks like AudioBench and AIR-Bench. Adversarial attacks such as VoiceJailbreak are also discussed.
Awesome-Jailbreak-on-LLMs
Awesome-Jailbreak-on-LLMs is a collection of state-of-the-art, novel, and exciting jailbreak methods on Large Language Models (LLMs). The repository contains papers, codes, datasets, evaluations, and analyses related to jailbreak attacks on LLMs. It serves as a comprehensive resource for researchers and practitioners interested in exploring various jailbreak techniques and defenses in the context of LLMs. Contributions such as additional jailbreak-related content, pull requests, and issue reports are welcome, and contributors are acknowledged. For any inquiries or issues, contact [email protected]. If you find this repository useful for your research or work, consider starring it to show appreciation.
gpupixel
GPUPixel is a real-time, high-performance image and video filter library written in C++11 and based on OpenGL/ES. It incorporates a built-in beauty face filter that achieves commercial-grade beauty effects. The library is extremely easy to compile and integrate with a small size, supporting platforms including iOS, Android, Mac, Windows, and Linux. GPUPixel provides various filters like skin smoothing, whitening, face slimming, big eyes, lipstick, and blush. It supports input formats like YUV420P, RGBA, JPEG, PNG, and output formats like RGBA and YUV420P. The library's performance on devices like iPhone and Android is optimized, with low CPU usage and fast processing times. GPUPixel's lib size is compact, making it suitable for mobile and desktop applications.
speech-trident
Speech Trident is a repository focusing on speech/audio large language models, covering representation learning, neural codec, and language models. It explores speech representation models, speech neural codec models, and speech large language models. The repository includes contributions from various researchers and provides a comprehensive list of speech/audio language models, representation models, and codec models.
Awesome-LLM-Safety
Welcome to our Awesome-llm-safety repository! We've curated a collection of the latest, most comprehensive, and most valuable resources on large language model safety (llm-safety). But we don't stop there; included are also relevant talks, tutorials, conferences, news, and articles. Our repository is constantly updated to ensure you have the most current information at your fingertips.
LLamaTuner
LLamaTuner is a repository for the Efficient Finetuning of Quantized LLMs project, focusing on building and sharing instruction-following Chinese baichuan-7b/LLaMA/Pythia/GLM model tuning methods. The project enables training on a single Nvidia RTX-2080TI and RTX-3090 for multi-round chatbot training. It utilizes bitsandbytes for quantization and is integrated with Huggingface's PEFT and transformers libraries. The repository supports various models, training approaches, and datasets for supervised fine-tuning, LoRA, QLoRA, and more. It also provides tools for data preprocessing and offers models in the Hugging Face model hub for inference and finetuning. The project is licensed under Apache 2.0 and acknowledges contributions from various open-source contributors.
Awesome-LLM-Eval
Awesome-LLM-Eval: a curated list of tools, benchmarks, demos, papers for Large Language Models (like ChatGPT, LLaMA, GLM, Baichuan, etc) Evaluation on Language capabilities, Knowledge, Reasoning, Fairness and Safety.
Model-References
The 'Model-References' repository contains examples for training and inference using Intel Gaudi AI Accelerator. It includes models for computer vision, natural language processing, audio, generative models, MLPerf™ training, and MLPerf™ inference. The repository provides performance data and model validation information for various frameworks like PyTorch. Users can find examples of popular models like ResNet, BERT, and Stable Diffusion optimized for Intel Gaudi AI accelerator.
SpinQuant
SpinQuant is a tool designed for LLM quantization with learned rotations. It focuses on optimizing rotation matrices to enhance the performance of quantized models, narrowing the accuracy gap to full precision models. The tool implements rotation optimization and PTQ evaluation with optimized rotation, providing arguments for model name, batch sizes, quantization bits, and rotation options. SpinQuant is based on the findings that rotation helps in removing outliers and improving quantization, with specific enhancements achieved through learning rotation with Cayley optimization.
For similar tasks
Awesome-Model-Merging-Methods-Theories-Applications
A comprehensive repository focusing on 'Model Merging in LLMs, MLLMs, and Beyond', providing an exhaustive overview of model merging methods, theories, applications, and future research directions. The repository covers various advanced methods, applications in foundation models, different machine learning subfields, and tasks like pre-merging methods, architecture transformation, weight alignment, basic merging methods, and more.
optillm
optillm is an OpenAI API compatible optimizing inference proxy implementing state-of-the-art techniques to enhance accuracy and performance of LLMs, focusing on reasoning over coding, logical, and mathematical queries. By leveraging additional compute at inference time, it surpasses frontier models across diverse tasks.
llm-structured-output
This repository contains a library for constraining LLM generation to structured output, enforcing a JSON schema for precise data types and property names. It includes an acceptor/state machine framework, JSON acceptor, and JSON schema acceptor for guiding decoding in LLMs. The library provides reference implementations using Apple's MLX library and examples for function calling tasks. The tool aims to improve LLM output quality by ensuring adherence to a schema, reducing unnecessary output, and enhancing performance through pre-emptive decoding. Evaluations show performance benchmarks and comparisons with and without schema constraints.
AirLine
AirLine is a learnable edge-based line detection algorithm designed for various robotic tasks such as scene recognition, 3D reconstruction, and SLAM. It offers a novel approach to extracting line segments directly from edges, enhancing generalization ability for unseen environments. The algorithm balances efficiency and accuracy through a region-grow algorithm and local edge voting scheme for line parameterization. AirLine demonstrates state-of-the-art precision with significant runtime acceleration compared to other learning-based methods, making it ideal for low-power robots.
LongRecipe
LongRecipe is a tool designed for efficient long context generalization in large language models. It provides a recipe for extending the context window of language models while maintaining their original capabilities. The tool includes data preprocessing steps, model training stages, and a process for merging fine-tuned models to enhance foundational capabilities. Users can follow the provided commands and scripts to preprocess data, train models in multiple stages, and merge models effectively.
ai-on-gke
This repository contains assets related to AI/ML workloads on Google Kubernetes Engine (GKE). Run optimized AI/ML workloads with Google Kubernetes Engine (GKE) platform orchestration capabilities. A robust AI/ML platform considers the following layers: Infrastructure orchestration that support GPUs and TPUs for training and serving workloads at scale Flexible integration with distributed computing and data processing frameworks Support for multiple teams on the same infrastructure to maximize utilization of resources
ray
Ray is a unified framework for scaling AI and Python applications. It consists of a core distributed runtime and a set of AI libraries for simplifying ML compute, including Data, Train, Tune, RLlib, and Serve. Ray runs on any machine, cluster, cloud provider, and Kubernetes, and features a growing ecosystem of community integrations. With Ray, you can seamlessly scale the same code from a laptop to a cluster, making it easy to meet the compute-intensive demands of modern ML workloads.
labelbox-python
Labelbox is a data-centric AI platform for enterprises to develop, optimize, and use AI to solve problems and power new products and services. Enterprises use Labelbox to curate data, generate high-quality human feedback data for computer vision and LLMs, evaluate model performance, and automate tasks by combining AI and human-centric workflows. The academic & research community uses Labelbox for cutting-edge AI research.
For similar jobs
sweep
Sweep is an AI junior developer that turns bugs and feature requests into code changes. It automatically handles developer experience improvements like adding type hints and improving test coverage.
teams-ai
The Teams AI Library is a software development kit (SDK) that helps developers create bots that can interact with Teams and Microsoft 365 applications. It is built on top of the Bot Framework SDK and simplifies the process of developing bots that interact with Teams' artificial intelligence capabilities. The SDK is available for JavaScript/TypeScript, .NET, and Python.
ai-guide
This guide is dedicated to Large Language Models (LLMs) that you can run on your home computer. It assumes your PC is a lower-end, non-gaming setup.
classifai
Supercharge WordPress Content Workflows and Engagement with Artificial Intelligence. Tap into leading cloud-based services like OpenAI, Microsoft Azure AI, Google Gemini and IBM Watson to augment your WordPress-powered websites. Publish content faster while improving SEO performance and increasing audience engagement. ClassifAI integrates Artificial Intelligence and Machine Learning technologies to lighten your workload and eliminate tedious tasks, giving you more time to create original content that matters.
chatbot-ui
Chatbot UI is an open-source AI chat app that allows users to create and deploy their own AI chatbots. It is easy to use and can be customized to fit any need. Chatbot UI is perfect for businesses, developers, and anyone who wants to create a chatbot.
BricksLLM
BricksLLM is a cloud native AI gateway written in Go. Currently, it provides native support for OpenAI, Anthropic, Azure OpenAI and vLLM. BricksLLM aims to provide enterprise level infrastructure that can power any LLM production use cases. Here are some use cases for BricksLLM: * Set LLM usage limits for users on different pricing tiers * Track LLM usage on a per user and per organization basis * Block or redact requests containing PIIs * Improve LLM reliability with failovers, retries and caching * Distribute API keys with rate limits and cost limits for internal development/production use cases * Distribute API keys with rate limits and cost limits for students
uAgents
uAgents is a Python library developed by Fetch.ai that allows for the creation of autonomous AI agents. These agents can perform various tasks on a schedule or take action on various events. uAgents are easy to create and manage, and they are connected to a fast-growing network of other uAgents. They are also secure, with cryptographically secured messages and wallets.
griptape
Griptape is a modular Python framework for building AI-powered applications that securely connect to your enterprise data and APIs. It offers developers the ability to maintain control and flexibility at every step. Griptape's core components include Structures (Agents, Pipelines, and Workflows), Tasks, Tools, Memory (Conversation Memory, Task Memory, and Meta Memory), Drivers (Prompt and Embedding Drivers, Vector Store Drivers, Image Generation Drivers, Image Query Drivers, SQL Drivers, Web Scraper Drivers, and Conversation Memory Drivers), Engines (Query Engines, Extraction Engines, Summary Engines, Image Generation Engines, and Image Query Engines), and additional components (Rulesets, Loaders, Artifacts, Chunkers, and Tokenizers). Griptape enables developers to create AI-powered applications with ease and efficiency.