
llm.nvim
Free large language model (LLM) support for Neovim, provides commands to interact with LLM (like ChatGPT, ChatGLM, kimi, deepseek, openrouter and local llms). Support Github models.
Stars: 255

llm.nvim is a universal plugin for a large language model (LLM) designed to enable users to interact with LLM within neovim. Users can customize various LLMs such as gpt, glm, kimi, and local LLM. The plugin provides tools for optimizing code, comparing code, translating text, and more. It also supports integration with free models from Cloudflare, Github models, siliconflow, and others. Users can customize tools, chat with LLM, quickly translate text, and explain code snippets. The plugin offers a flexible window interface for easy interaction and customization.
README:
[!IMPORTANT] A free large language model(LLM) plugin that allows you to interact with LLM in Neovim.
- Supports any LLM, such as GPT, GLM, Kimi, deepseek or local LLMs (such as ollama).
- Allows you to define your own AI tools, with different tools able to use different models.
- Most importantly, you can use free models provided by any platform (such as
Cloudflare
,GitHub models
,SiliconFlow
,openrouter
or other platforms).
[!NOTE] The configurations of different LLMs (such as ollama, deepseek), UI configurations, and AI tools (including code completion) should be checked in the examples first. Here you will find most of the information you want to know. Additionally, before using the plugin, you should ensure that your
LLM_KEY
is valid and that the environment variable is in effect.
- Screenshots
- Installation
- Configuration
- Default Keyboard Shortcuts
- TODO List
- Author's configuration
- Acknowledgments
-
Q&A
- The format of curl usage in Windows is different from Linux, and the default request format of llm.nvim may cause issues under Windows.
- Switching between multiple LLMs and frequently changing the value of LLM_KEY is troublesome, and I don't want to expose my key in Neovim's configuration file.
- Priority of different parse/streaming functions
- How can the AI-generated git commit message feature be integrated with lazygit
- virtual text
- blink.cmp or nvim-cmp
One-time, no history retained.
You can configure inline_assistant to decide whether to display diffs (default: show by pressing 'd').
You can configure inline_assistant to decide whether to display diffs (default: show by pressing 'd').
- Display side by side
- Display in the form of a diff
curl
-
Register on the official website and obtain your API Key (Cloudflare needs to obtain an additional account).
-
Set the
LLM_KEY
(Cloudflare needs to set an additionalACCOUNT
) environment variable in yourzshrc
orbashrc
.
export LLM_KEY=<Your API_KEY>
export ACCOUNT=<Your ACCOUNT> # just for cloudflare
Platform | Link to obtain api key | Note |
---|---|---|
Cloudflare | https://dash.cloudflare.com/ | You can see all of Cloudflare's models here, with the ones marked as beta being free models. |
ChatGLM(智谱清言) | https://open.bigmodel.cn/ | |
Kimi(月之暗面) | Moonshot AI 开放平台 | |
Github Models | Github Token | |
siliconflow (硅基流动) | siliconflow | You can see all models on Siliconflow here, and select 'Only Free' to see all free models. |
Deepseek | https://platform.deepseek.com/api_keys | |
Openrouter | https://openrouter.ai/ | |
Chatanywhere | https://api.chatanywhere.org/v1/oauth/free/render | 200 free calls to GPT-4o-mini are available every day. |
For local llms, Set LLM_KEY
to NONE
in your zshrc
or bashrc
.
- lazy.nvim
{
"Kurama622/llm.nvim",
dependencies = { "nvim-lua/plenary.nvim", "MunifTanjim/nui.nvim"},
cmd = { "LLMSessionToggle", "LLMSelectedTextHandler", "LLMAppHandler" },
keys = {
{ "<leader>ac", mode = "n", "<cmd>LLMSessionToggle<cr>" },
},
}
Some commands you should know about
-
LLMSessionToggle
: open/hide the Chat UI. -
LLMSelectedTextHandler
: Handles the selected text, the way it is processed depends on the prompt words you input. -
LLMAppHandler
: call AI tools.
If the URL is not configured, the default is to use Cloudflare.
For more details or examples, please refer to Chat Configuration.
Click here to see meanings of some configuration options
-
prompt
: Model prompt. -
prefix
: Dialog role indicator. -
style
: Style of the Chat UI (float means floating window, others are split windows). -
url
: Model api url. -
model
: Model name. -
api_type
: The parsing format of the model output:openai
,zhipu
,ollama
,workers-ai
. Theopenai
format is compatible with most models, butChatGLM
can only be parsed using thezhipu
format, andcloudflare
can only be parsed using theworkers-ai
format. If you use ollama to run the model, you can useollama
. -
fetch_key
: If you need to use models from different platforms simultaneously, you can configurefetch_key
to ensure that different models use different API Keys. The usage is as follows:fetch_key = function() return "<your api key>" end
-
max_tokens
: Maximum output length of the model. -
save_session
: Whether to save session history. -
max_history
: Maximum number of saved sessions. -
history_path
: Path for saving session history. -
temperature
: The temperature of the model, controlling the randomness of the model's output. -
temperature
: The top_p of the model, controlling the randomness of the model's output. -
spinner
: The waiting animation of the model output (effective when non-streaming output). -
display
-
diff
: Display style of diff (effective when optimizing code and showing diff, the style in the screenshot is mini_diff, which requires installation of mini.diff).
-
-
keys
: Shortcut key settings for different windows, default values can be found in Default Shortcuts-
floating style
- input window
-
Input:Cancel
: Cancel dialog response. -
Input:Submit
: Submit your question. -
Input:Resend
: Rerespond to the dialog. -
Input:HistoryNext
: Select the next session history. -
Input:HistoryPrev
: Select the previous session history.
-
- Chat UI
-
Session:Toggle
: open/hide the Chat UI. -
Session:Close
: close the Chat UI.
-
- input window
-
split style
- output window
-
Output:Ask
: Open input window. -
Output:Cancel
: Cancel diaglog response. -
Output:Resend
: Rerespond to the dialog. -
Session:History
: open session history.
-
- Chat UI
-
Session:Toggle
: open/hide the Chat UI. -
Session:Close
: close the Chat UI.
-
- output window
-
floating style
If you use a local LLM (but not one running on ollama), you may need to define the streaming_handler (required), as well as the parse_handler (optional, used by only a few AI tools), for details see Local LLM Configuration.
If you want to further configure the style of the conversation interface, you can configure chat_ui_opts
and popwin_opts
separately.
Click here to see how to configure the window style
Their configuration options are the same:
-
relative
:-
editor
: The floating window relative to the current editor window. -
cursor
: The floating window relative to the current cursor position. -
win
: The floating window relative to the current window.
-
-
position
: The position of the window. -
size
: The size of the window. -
enter
: Whether the window automatically gains focus. -
focusable
: Whether the window can gain focus. -
zindex
: The layer of the window. -
border
-
style
: The style of the window border. -
text
: The text of the window border.
-
-
win_options
: The options of the window.-
winblend
: The transparency of the window. -
winhighlight
: The highlight of the window.
-
More information can be found in nui/popup.
For more details or examples, please refer to UI Configuration.
Currently, llm.nvim provides some templates for AI tools, making it convenient for everyone to customize their own AI tools.
All AI tools need to be defined in app_handler
, presented in the form of a pair of key-value
(key
is the tool name and value
is the configuration information of the tool).
For more details or examples, please refer to AI Tools Configuration.
Click here to see how to configure AI tools
For all AI tools, their configuration options are similar:
-
handler
: Which template to use-
side_by_side_handler
: Display results in two windows side by side -
action_handler
: Display results in the source file in the form of a diff-
Y
/y
: Accept LLM suggested code -
N
/n
: Reject LLM suggested code -
<ESC>
: Exit directly -
I
/i
: Input additional optimization conditions -
<C-r>
: Optimize again directly
-
-
qa_handler
: AI for single-round dialogue -
flexi_handler
: Results will be displayed in a flexible window (window size is automatically calculated based on the amount of output text) - You can also customize functions
-
-
prompt
: Prompt words for the AI tool -
opts
-
spell
: Whether to have spell check -
number
: Whether to display line numbers -
wrap
: Whether to automatically wrap lines -
linebreak
: Whether to allow line breaks in the middle of words -
url
,model
: The LLM used by this AI tool -
api_type
: The type of parsing output by this AI tool -
streaming_handler
: This AI tool uses a custom streaming parsing function -
parse_handler
: This AI tool uses a custom parsing function -
border
: Floating window border style -
accept
-
mapping
: The key mapping for accepting the output-
mode
: Vim mode (Default mode:n
) -
keys
: Your key mappings. (Default keys:Y
/y
)
-
-
action
: The action for accepting the output, which is executed when accepting the output. (Default action: Copy the output)
-
-
reject
-
mapping
: The key mapping for rejecting the output-
mode
: Vim mode (Default mode:n
) -
keys
: Your key mappings. (Default keys:N
/n
)
-
-
action
: The action for rejecting the output, which is executed when rejecting the output. (Default action: None or close the window)
-
-
close
-
mapping
: The key mapping for closing the AI tool-
mode
: Vim mode (Default mode:n
) -
keys
: Your key mappings. (Default keys:<ESC>
)
-
-
action
: The action for closing the AI tool. (Default action: Reject all output and close the window)
-
-
Different templates also have some exclusive configuration items of their own.
-
You can also define in the
opts
ofqa_handler
:-
component_width
: the width of the component -
component_height
: the height of the component -
query
-
title
: the title of the component, which will be displayed in the center above the component -
hl
: the highlight of the title
-
-
input_box_opts
: the window options for the input box (size
,win_options
) -
preview_box_opts
: the window options for the preview box (size
,win_options
)
-
-
You can also define in the
opts
ofaction_handler
:-
language
: The language used for the output result (English
/Chinese
/Japanese
etc.) -
input
-
relative
: The relative position of the split window (editor
/win
) -
position
: The position of the split window (top
/left
/right
/bottom
) -
size
: The proportion of the split window (default is 25%) -
enter
: Whether to automatically enter the window
-
-
output
-
relative
: Same asinput
-
position
: Same asinput
-
size
: Same asinput
-
enter
: Same asinput
-
-
-
In the
opts
ofside_by_side_handler
, you can also define:-
left
Left window-
title
: The title of the window -
focusable
: Whether the window can gain focus border
win_options
-
-
right
Right window-
title
: The title of the window -
focusable
: Whether the window can gain focus border
win_options
-
-
-
In the
opts
offlexi_handler
, you can also define:-
exit_on_move
: Whether to close the flexible window when the cursor moves -
enter_flexible_window
: Whether to automatically enter the window when the flexible window pops up -
apply_visual_selection
: Whether to append the selected text content after theprompt
-
My some AI tool configurations:
{
"Kurama622/llm.nvim",
dependencies = { "nvim-lua/plenary.nvim", "MunifTanjim/nui.nvim" },
cmd = { "LLMSessionToggle", "LLMSelectedTextHandler", "LLMAppHandler" },
config = function()
local tools = require("llm.common.tools")
require("llm").setup({
app_handler = {
OptimizeCode = {
handler = tools.side_by_side_handler,
-- opts = {
-- streaming_handler = local_llm_streaming_handler,
-- },
},
TestCode = {
handler = tools.side_by_side_handler,
prompt = [[ Write some test cases for the following code, only return the test cases.
Give the code content directly, do not use code blocks or other tags to wrap it. ]],
opts = {
right = {
title = " Test Cases ",
},
},
},
OptimCompare = {
handler = tools.action_handler,
opts = {
fetch_key = function()
return vim.env.GITHUB_TOKEN
end,
url = "https://models.inference.ai.azure.com/chat/completions",
model = "gpt-4o",
api_type = "openai",
},
},
Translate = {
handler = tools.qa_handler,
opts = {
fetch_key = function()
return vim.env.GLM_KEY
end,
url = "https://open.bigmodel.cn/api/paas/v4/chat/completions",
model = "glm-4-flash",
api_type = "zhipu",
component_width = "60%",
component_height = "50%",
query = {
title = " Trans ",
hl = { link = "Define" },
},
input_box_opts = {
size = "15%",
win_options = {
winhighlight = "Normal:Normal,FloatBorder:FloatBorder",
},
},
preview_box_opts = {
size = "85%",
win_options = {
winhighlight = "Normal:Normal,FloatBorder:FloatBorder",
},
},
},
},
-- check siliconflow's balance
UserInfo = {
handler = function()
local key = os.getenv("LLM_KEY")
local res = tools.curl_request_handler(
"https://api.siliconflow.cn/v1/user/info",
{ "GET", "-H", string.format("'Authorization: Bearer %s'", key) }
)
if res ~= nil then
print("balance: " .. res.data.balance)
end
end,
},
WordTranslate = {
handler = tools.flexi_handler,
prompt = "Translate the following text to Chinese, please only return the translation",
opts = {
fetch_key = function()
return vim.env.GLM_KEY
end,
url = "https://open.bigmodel.cn/api/paas/v4/chat/completions",
model = "glm-4-flash",
api_type = "zhipu",
args = [[return {url, "-N", "-X", "POST", "-H", "Content-Type: application/json", "-H", authorization, "-d", vim.fn.json_encode(body)}]],
exit_on_move = true,
enter_flexible_window = false,
},
},
CodeExplain = {
handler = tools.flexi_handler,
prompt = "Explain the following code, please only return the explanation, and answer in Chinese",
opts = {
fetch_key = function()
return vim.env.GLM_KEY
end,
url = "https://open.bigmodel.cn/api/paas/v4/chat/completions",
model = "glm-4-flash",
api_type = "zhipu",
enter_flexible_window = true,
},
},
CommitMsg = {
handler = tools.flexi_handler,
prompt = function()
-- Source: https://andrewian.dev/blog/ai-git-commits
return string.format([[You are an expert at following the Conventional Commit specification. Given the git diff listed below, please generate a commit message for me:
1. First line: conventional commit format (type: concise description) (remember to use semantic types like feat, fix, docs, style, refactor, perf, test, chore, etc.)
2. Optional bullet points if more context helps:
- Keep the second line blank
- Keep them short and direct
- Focus on what changed
- Always be terse
- Don't overly explain
- Drop any fluffy or formal language
Return ONLY the commit message - no introduction, no explanation, no quotes around it.
Examples:
feat: add user auth system
- Add JWT tokens for API auth
- Handle token refresh for long sessions
fix: resolve memory leak in worker pool
- Clean up idle connections
- Add timeout for stale workers
Simple change example:
fix: typo in README.md
Very important: Do not respond with any of the examples. Your message must be based off the diff that is about to be provided, with a little bit of styling informed by the recent commits you're about to see.
Based on this format, generate appropriate commit messages. Respond with message only. DO NOT format the message in Markdown code blocks, DO NOT use backticks:
```diff
%s
```
]],
vim.fn.system("git diff --no-ext-diff --staged")
)
end,
opts = {
fetch_key = function()
return vim.env.GLM_KEY
end,
url = "https://open.bigmodel.cn/api/paas/v4/chat/completions",
model = "glm-4-flash",
api_type = "zhipu",
enter_flexible_window = true,
apply_visual_selection = false,
},
},
},
})
end,
keys = {
{ "<leader>ac", mode = "n", "<cmd>LLMSessionToggle<cr>" },
{ "<leader>ts", mode = "x", "<cmd>LLMAppHandler WordTranslate<cr>" },
{ "<leader>ae", mode = "v", "<cmd>LLMAppHandler CodeExplain<cr>" },
{ "<leader>at", mode = "n", "<cmd>LLMAppHandler Translate<cr>" },
{ "<leader>tc", mode = "x", "<cmd>LLMAppHandler TestCode<cr>" },
{ "<leader>ao", mode = "x", "<cmd>LLMAppHandler OptimCompare<cr>" },
{ "<leader>au", mode = "n", "<cmd>LLMAppHandler UserInfo<cr>" },
{ "<leader>ag", mode = "n", "<cmd>LLMAppHandler CommitMsg<cr>" },
-- { "<leader>ao", mode = "x", "<cmd>LLMAppHandler OptimizeCode<cr>" },
},
},
Local LLMs require custom parsing functions; for streaming output, we use our custom streaming_handler
; for AI tools that return output results in one go, we use our custom parse_handler
.
Below is an example of ollama
running llama3.2:1b
.
Expand the code.
local function local_llm_streaming_handler(chunk, ctx, F)
if not chunk then
return ctx.assistant_output
end
local tail = chunk:sub(-1, -1)
if tail:sub(1, 1) ~= "}" then
ctx.line = ctx.line .. chunk
else
ctx.line = ctx.line .. chunk
local status, data = pcall(vim.fn.json_decode, ctx.line)
if not status or not data.message.content then
return ctx.assistant_output
end
ctx.assistant_output = ctx.assistant_output .. data.message.content
F.WriteContent(ctx.bufnr, ctx.winid, data.message.content)
ctx.line = ""
end
return ctx.assistant_output
end
local function local_llm_parse_handler(chunk)
local assistant_output = chunk.message.content
return assistant_output
end
return {
{
"Kurama622/llm.nvim",
dependencies = { "nvim-lua/plenary.nvim", "MunifTanjim/nui.nvim" },
cmd = { "LLMSessionToggle", "LLMSelectedTextHandler" },
config = function()
require("llm").setup({
url = "http://localhost:11434/api/chat", -- your url
model = "llama3.2:1b",
streaming_handler = local_llm_streaming_handler,
app_handler = {
WordTranslate = {
handler = tools.flexi_handler,
prompt = "Translate the following text to Chinese, please only return the translation",
opts = {
parse_handler = local_llm_parse_handler,
exit_on_move = true,
enter_flexible_window = false,
},
},
}
})
end,
keys = {
{ "<leader>ac", mode = "n", "<cmd>LLMSessionToggle<cr>" },
},
}
}
- floating window
window | key | mode | desc |
---|---|---|---|
Input | ctrl+g |
i |
Submit your question |
Input | ctrl+c |
i |
Cancel dialog response |
Input | ctrl+r |
i |
Rerespond to the dialog |
Input | ctrl+j |
i |
Select the next session history |
Input | ctrl+k |
i |
Select the previous session history |
Output+Input | <leader>ac |
n |
Toggle session |
Output+Input | <esc> |
n |
Close session |
You can use
<C-w><C-w>
to switch windows, and if you find it ungraceful, you can also set your own shortcut key for window switching. (This feature has not set a default shortcut key)
-- Switch from the output window to the input window.
["Focus:Input"] = { mode = "n", key = {"i", "<C-w>"} },
-- Switch from the input window to the output window.
["Focus:Output"] = { mode = { "n", "i" }, key = "<C-w>" },
- split window
window | key | mode | desc |
---|---|---|---|
Input | <cr> |
n |
Submit your question |
Output | i |
n |
Open the input box |
Output | ctrl+c |
n |
Cancel dialog response |
Output | ctrl+r |
n |
Rerespond to the dialog |
Output | ctrl+h |
n |
Open the history window |
Output+Input | <leader>ac |
n |
Toggle session |
Output+Input | <esc> |
n |
Close session |
History | j |
n |
Preview the next session history |
History | k |
n |
Preview the previous session history |
History | <cr> |
n |
Enter the selected session |
History | <esc> |
n |
Close the history window |
We would like to express our heartfelt gratitude to the contributors of the following open-source projects, whose code has provided invaluable inspiration and reference for the development of llm.nvim:
- olimorris/codecompanion.nvim: Diff style and prompt.
- SmiteshP/nvim-navbuddy: UI.
- milanglacier/minuet-ai.nvim: Code completions.
The format of curl usage in Windows is different from Linux, and the default request format of llm.nvim may cause issues under Windows.
Use a custom request format
-
Basic Chat and some AI tools (using streaming output) with customized request format
Define the
args
parameter at the same level as theprompt
.--[[ custom request args ]] args = [[return {url, "-N", "-X", "POST", "-H", "Content-Type: application/json", "-H", authorization, "-d", vim.fn.json_encode(body)}]],
-
AI tools (using non-streaming output) custom request format
Define args in
opts
WordTranslate = { handler = tools.flexi_handler, prompt = "Translate the following text to Chinese, please only return the translation", opts = { fetch_key = function() return vim.env.GLM_KEY end, url = "https://open.bigmodel.cn/api/paas/v4/chat/completions", model = "glm-4-flash", api_type = "zhipu", args = [[return {url, "-N", "-X", "POST", "-H", "Content-Type: application/json", "-H", authorization, "-d", vim.fn.json_encode(body)}]], exit_on_move = true, enter_flexible_window = false, }, },
[!NOTE] You need to modify the args according to your actual situation.
Switching between multiple LLMs and frequently changing the value of LLM_KEY is troublesome, and I don't want to expose my key in Neovim's configuration file.
- Create a
.env
file specifically to store your various keys. Note: Do not upload this file to GitHub.
export GITHUB_TOKEN=xxxxxxx
export DEEPSEEK_TOKEN=xxxxxxx
export SILICONFLOW_TOKEN=xxxxxxx
-
Load the
.env
file inzshrc
orbashrc
source ~/.config/zsh/.env # Default to using the LLM provided by Github Models. export LLM_KEY=$GITHUB_TOKEN
-
Finally, switching keys is completed through
fetch_key
.fetch_key = function() return vim.env.DEEPSEEK_TOKEN end,
AI tool configuration's streaming_handler
or parse_handler
> AI tool configuration's api_type
> Main configuration's streaming_handler
or parse_handler
> Main configuration's api_type
{
"kdheepak/lazygit.nvim",
lazy = true,
cmd = {
"LazyGit",
"LazyGitConfig",
"LazyGitCurrentFile",
"LazyGitFilter",
"LazyGitFilterCurrentFile",
},
-- optional for floating window border decoration
dependencies = {
"nvim-lua/plenary.nvim",
},
config = function()
vim.keymap.set("t", "<C-c>", function()
vim.api.nvim_win_close(vim.api.nvim_get_current_win(), true)
vim.api.nvim_command("LLMAppHandler CommitMsg")
end, { desc = "AI Commit Msg" })
end,
}
For Tasks:
Click tags to check more tools for each tasksFor Jobs:
Alternative AI tools for llm.nvim
Similar Open Source Tools

llm.nvim
llm.nvim is a universal plugin for a large language model (LLM) designed to enable users to interact with LLM within neovim. Users can customize various LLMs such as gpt, glm, kimi, and local LLM. The plugin provides tools for optimizing code, comparing code, translating text, and more. It also supports integration with free models from Cloudflare, Github models, siliconflow, and others. Users can customize tools, chat with LLM, quickly translate text, and explain code snippets. The plugin offers a flexible window interface for easy interaction and customization.

avante.nvim
avante.nvim is a Neovim plugin that emulates the behavior of the Cursor AI IDE, providing AI-driven code suggestions and enabling users to apply recommendations to their source files effortlessly. It offers AI-powered code assistance and one-click application of suggested changes, streamlining the editing process and saving time. The plugin is still in early development, with functionalities like setting API keys, querying AI about code, reviewing suggestions, and applying changes. Key bindings are available for various actions, and the roadmap includes enhancing AI interactions, stability improvements, and introducing new features for coding tasks.

ax
Ax is a Typescript library that allows users to build intelligent agents inspired by agentic workflows and the Stanford DSP paper. It seamlessly integrates with multiple Large Language Models (LLMs) and VectorDBs to create RAG pipelines or collaborative agents capable of solving complex problems. The library offers advanced features such as streaming validation, multi-modal DSP, and automatic prompt tuning using optimizers. Users can easily convert documents of any format to text, perform smart chunking, embedding, and querying, and ensure output validation while streaming. Ax is production-ready, written in Typescript, and has zero dependencies.

python-tgpt
Python-tgpt is a Python package that enables seamless interaction with over 45 free LLM providers without requiring an API key. It also provides image generation capabilities. The name _python-tgpt_ draws inspiration from its parent project tgpt, which operates on Golang. Through this Python adaptation, users can effortlessly engage with a number of free LLMs available, fostering a smoother AI interaction experience.

receipt-scanner
The receipt-scanner repository is an AI-Powered Receipt and Invoice Scanner for Laravel that allows users to easily extract structured receipt data from images, PDFs, and emails within their Laravel application using OpenAI. It provides a light wrapper around OpenAI Chat and Completion endpoints, supports various input formats, and integrates with Textract for OCR functionality. Users can install the package via composer, publish configuration files, and use it to extract data from plain text, PDFs, images, Word documents, and web content. The scanned receipt data is parsed into a DTO structure with main classes like Receipt, Merchant, and LineItem.

mistral-inference
Mistral Inference repository contains minimal code to run 7B, 8x7B, and 8x22B models. It provides model download links, installation instructions, and usage guidelines for running models via CLI or Python. The repository also includes information on guardrailing, model platforms, deployment, and references. Users can interact with models through commands like mistral-demo, mistral-chat, and mistral-common. Mistral AI models support function calling and chat interactions for tasks like testing models, chatting with models, and using Codestral as a coding assistant. The repository offers detailed documentation and links to blogs for further information.

langchainrb
Langchain.rb is a Ruby library that makes it easy to build LLM-powered applications. It provides a unified interface to a variety of LLMs, vector search databases, and other tools, making it easy to build and deploy RAG (Retrieval Augmented Generation) systems and assistants. Langchain.rb is open source and available under the MIT License.

nano-graphrag
nano-GraphRAG is a simple, easy-to-hack implementation of GraphRAG that provides a smaller, faster, and cleaner version of the official implementation. It is about 800 lines of code, small yet scalable, asynchronous, and fully typed. The tool supports incremental insert, async methods, and various parameters for customization. Users can replace storage components and LLM functions as needed. It also allows for embedding function replacement and comes with pre-defined prompts for entity extraction and community reports. However, some features like covariates and global search implementation differ from the original GraphRAG. Future versions aim to address issues related to data source ID, community description truncation, and add new components.

magentic
Easily integrate Large Language Models into your Python code. Simply use the `@prompt` and `@chatprompt` decorators to create functions that return structured output from the LLM. Mix LLM queries and function calling with regular Python code to create complex logic.

Webscout
WebScout is a versatile tool that allows users to search for anything using Google, DuckDuckGo, and phind.com. It contains AI models, can transcribe YouTube videos, generate temporary email and phone numbers, has TTS support, webai (terminal GPT and open interpreter), and offline LLMs. It also supports features like weather forecasting, YT video downloading, temp mail and number generation, text-to-speech, advanced web searches, and more.

ai00_server
AI00 RWKV Server is an inference API server for the RWKV language model based upon the web-rwkv inference engine. It supports VULKAN parallel and concurrent batched inference and can run on all GPUs that support VULKAN. No need for Nvidia cards!!! AMD cards and even integrated graphics can be accelerated!!! No need for bulky pytorch, CUDA and other runtime environments, it's compact and ready to use out of the box! Compatible with OpenAI's ChatGPT API interface. 100% open source and commercially usable, under the MIT license. If you are looking for a fast, efficient, and easy-to-use LLM API server, then AI00 RWKV Server is your best choice. It can be used for various tasks, including chatbots, text generation, translation, and Q&A.

monacopilot
Monacopilot is a powerful and customizable AI auto-completion plugin for the Monaco Editor. It supports multiple AI providers such as Anthropic, OpenAI, Groq, and Google, providing real-time code completions with an efficient caching system. The plugin offers context-aware suggestions, customizable completion behavior, and framework agnostic features. Users can also customize the model support and trigger completions manually. Monacopilot is designed to enhance coding productivity by providing accurate and contextually appropriate completions in daily spoken language.

deepgram-js-sdk
Deepgram JavaScript SDK. Power your apps with world-class speech and Language AI models.

nuxt-llms
Nuxt LLMs automatically generates llms.txt markdown documentation for Nuxt applications. It provides runtime hooks to collect data from various sources and generate structured documentation. The tool allows customization of sections directly from nuxt.config.ts and integrates with Nuxt modules via the runtime hooks system. It generates two documentation formats: llms.txt for concise structured documentation and llms_full.txt for detailed documentation. Users can extend documentation using hooks to add sections, links, and metadata. The tool is suitable for developers looking to automate documentation generation for their Nuxt applications.

client-python
The Mistral Python Client is a tool inspired by cohere-python that allows users to interact with the Mistral AI API. It provides functionalities to access and utilize the AI capabilities offered by Mistral. Users can easily install the client using pip and manage dependencies using poetry. The client includes examples demonstrating how to use the API for various tasks, such as chat interactions. To get started, users need to obtain a Mistral API Key and set it as an environment variable. Overall, the Mistral Python Client simplifies the integration of Mistral AI services into Python applications.
For similar tasks

llm.nvim
llm.nvim is a universal plugin for a large language model (LLM) designed to enable users to interact with LLM within neovim. Users can customize various LLMs such as gpt, glm, kimi, and local LLM. The plugin provides tools for optimizing code, comparing code, translating text, and more. It also supports integration with free models from Cloudflare, Github models, siliconflow, and others. Users can customize tools, chat with LLM, quickly translate text, and explain code snippets. The plugin offers a flexible window interface for easy interaction and customization.

gpt_academic
GPT Academic is a powerful tool that leverages the capabilities of large language models (LLMs) to enhance academic research and writing. It provides a user-friendly interface that allows researchers, students, and professionals to interact with LLMs and utilize their abilities for various academic tasks. With GPT Academic, users can access a wide range of features and functionalities, including: * **Summarization and Paraphrasing:** GPT Academic can summarize complex texts, articles, and research papers into concise and informative summaries. It can also paraphrase text to improve clarity and readability. * **Question Answering:** Users can ask GPT Academic questions related to their research or studies, and the tool will provide comprehensive and well-informed answers based on its knowledge and understanding of the relevant literature. * **Code Generation and Explanation:** GPT Academic can generate code snippets and provide explanations for complex coding concepts. It can also help debug code and suggest improvements. * **Translation:** GPT Academic supports translation of text between multiple languages, making it a valuable tool for researchers working with international collaborations or accessing resources in different languages. * **Citation and Reference Management:** GPT Academic can help users manage their citations and references by automatically generating citations in various formats and providing suggestions for relevant references based on the user's research topic. * **Collaboration and Note-Taking:** GPT Academic allows users to collaborate on projects and take notes within the tool. They can share their work with others and access a shared workspace for real-time collaboration. * **Customizable Interface:** GPT Academic offers a customizable interface that allows users to tailor the tool to their specific needs and preferences. They can choose from a variety of themes, adjust the layout, and add or remove features to create a personalized workspace. Overall, GPT Academic is a versatile and powerful tool that can significantly enhance the productivity and efficiency of academic research and writing. It empowers users to leverage the capabilities of LLMs and unlock new possibilities for academic exploration and knowledge creation.

shell-ask
Shell Ask is a command-line tool that enables users to interact with various language models through a simple interface. It supports multiple LLMs such as OpenAI, Anthropic, Ollama, and Google Gemini. Users can ask questions, provide context through command output, select models interactively, and define reusable AI commands. The tool allows piping the output of other programs for enhanced functionality. With AI command presets and configuration options, Shell Ask provides a versatile and efficient way to leverage language models for various tasks.

DevoxxGenieIDEAPlugin
Devoxx Genie is a Java-based IntelliJ IDEA plugin that integrates with local and cloud-based LLM providers to aid in reviewing, testing, and explaining project code. It supports features like code highlighting, chat conversations, and adding files/code snippets to context. Users can modify REST endpoints and LLM parameters in settings, including support for cloud-based LLMs. The plugin requires IntelliJ version 2023.3.4 and JDK 17. Building and publishing the plugin is done using Gradle tasks. Users can select an LLM provider, choose code, and use commands like review, explain, or generate unit tests for code analysis.

neural
Neural is a Vim and Neovim plugin that integrates various machine learning tools to assist users in writing code, generating text, and explaining code or paragraphs. It supports multiple machine learning models, focuses on privacy, and is compatible with Vim 8.0+ and Neovim 0.8+. Users can easily configure Neural to interact with third-party machine learning tools, such as OpenAI, to enhance code generation and completion. The plugin also provides commands like `:NeuralExplain` to explain code or text and `:NeuralStop` to stop Neural from working. Neural is maintained by the Dense Analysis team and comes with a disclaimer about sending input data to third-party servers for machine learning queries.

fittencode.nvim
Fitten Code AI Programming Assistant for Neovim provides fast completion using AI, asynchronous I/O, and support for various actions like document code, edit code, explain code, find bugs, generate unit test, implement features, optimize code, refactor code, start chat, and more. It offers features like accepting suggestions with Tab, accepting line with Ctrl + Down, accepting word with Ctrl + Right, undoing accepted text, automatic scrolling, and multiple HTTP/REST backends. It can run as a coc.nvim source or nvim-cmp source.

chatgpt
The ChatGPT R package provides a set of features to assist in R coding. It includes addins like Ask ChatGPT, Comment selected code, Complete selected code, Create unit tests, Create variable name, Document code, Explain selected code, Find issues in the selected code, Optimize selected code, and Refactor selected code. Users can interact with ChatGPT to get code suggestions, explanations, and optimizations. The package helps in improving coding efficiency and quality by providing AI-powered assistance within the RStudio environment.

scalene
Scalene is a high-performance CPU, GPU, and memory profiler for Python that provides detailed information and runs faster than many other profilers. It incorporates AI-powered proposed optimizations, allowing users to generate optimization suggestions by clicking on specific lines or regions of code. Scalene separates time spent in Python from native code, highlights hotspots, and identifies memory usage per line. It supports GPU profiling on NVIDIA-based systems and detects memory leaks. Users can generate reduced profiles, profile specific functions using decorators, and suspend/resume profiling for background processes. Scalene is available as a pip or conda package and works on various platforms. It offers features like profiling at the line level, memory trends, copy volume reporting, and leak detection.
For similar jobs

sweep
Sweep is an AI junior developer that turns bugs and feature requests into code changes. It automatically handles developer experience improvements like adding type hints and improving test coverage.

teams-ai
The Teams AI Library is a software development kit (SDK) that helps developers create bots that can interact with Teams and Microsoft 365 applications. It is built on top of the Bot Framework SDK and simplifies the process of developing bots that interact with Teams' artificial intelligence capabilities. The SDK is available for JavaScript/TypeScript, .NET, and Python.

ai-guide
This guide is dedicated to Large Language Models (LLMs) that you can run on your home computer. It assumes your PC is a lower-end, non-gaming setup.

classifai
Supercharge WordPress Content Workflows and Engagement with Artificial Intelligence. Tap into leading cloud-based services like OpenAI, Microsoft Azure AI, Google Gemini and IBM Watson to augment your WordPress-powered websites. Publish content faster while improving SEO performance and increasing audience engagement. ClassifAI integrates Artificial Intelligence and Machine Learning technologies to lighten your workload and eliminate tedious tasks, giving you more time to create original content that matters.

chatbot-ui
Chatbot UI is an open-source AI chat app that allows users to create and deploy their own AI chatbots. It is easy to use and can be customized to fit any need. Chatbot UI is perfect for businesses, developers, and anyone who wants to create a chatbot.

BricksLLM
BricksLLM is a cloud native AI gateway written in Go. Currently, it provides native support for OpenAI, Anthropic, Azure OpenAI and vLLM. BricksLLM aims to provide enterprise level infrastructure that can power any LLM production use cases. Here are some use cases for BricksLLM: * Set LLM usage limits for users on different pricing tiers * Track LLM usage on a per user and per organization basis * Block or redact requests containing PIIs * Improve LLM reliability with failovers, retries and caching * Distribute API keys with rate limits and cost limits for internal development/production use cases * Distribute API keys with rate limits and cost limits for students

uAgents
uAgents is a Python library developed by Fetch.ai that allows for the creation of autonomous AI agents. These agents can perform various tasks on a schedule or take action on various events. uAgents are easy to create and manage, and they are connected to a fast-growing network of other uAgents. They are also secure, with cryptographically secured messages and wallets.

griptape
Griptape is a modular Python framework for building AI-powered applications that securely connect to your enterprise data and APIs. It offers developers the ability to maintain control and flexibility at every step. Griptape's core components include Structures (Agents, Pipelines, and Workflows), Tasks, Tools, Memory (Conversation Memory, Task Memory, and Meta Memory), Drivers (Prompt and Embedding Drivers, Vector Store Drivers, Image Generation Drivers, Image Query Drivers, SQL Drivers, Web Scraper Drivers, and Conversation Memory Drivers), Engines (Query Engines, Extraction Engines, Summary Engines, Image Generation Engines, and Image Query Engines), and additional components (Rulesets, Loaders, Artifacts, Chunkers, and Tokenizers). Griptape enables developers to create AI-powered applications with ease and efficiency.