tokencost
Easy token price estimates for 400+ LLMs. TokenOps.
Stars: 1392
Tokencost is a clientside tool for calculating the USD cost of using major Large Language Model (LLMs) APIs by estimating the cost of prompts and completions. It helps track the latest price changes of major LLM providers, accurately count prompt tokens before sending OpenAI requests, and easily integrate to get the cost of a prompt or completion with a single function. Users can calculate prompt and completion costs using OpenAI requests, count tokens in prompts formatted as message lists or string prompts, and refer to a cost table with updated prices for various LLM models. The tool also supports callback handlers for LLM wrapper/framework libraries like LlamaIndex and Langchain.
README:
Clientside token counting + price estimation for LLM apps and AI agents.
🐦 Twitter • 📢 Discord • 🖇️ AgentOps
Tokencost helps calculate the USD cost of using major Large Language Model (LLMs) APIs by calculating the estimated cost of prompts and completions.
Building AI agents? Check out AgentOps
- LLM Price Tracking Major LLM providers frequently add new models and update pricing. This repo helps track the latest price changes
- Token counting Accurately count prompt tokens before sending OpenAI requests
- Easy integration Get the cost of a prompt or completion with a single function
from tokencost import calculate_prompt_cost, calculate_completion_cost
model = "gpt-3.5-turbo"
prompt = [{ "role": "user", "content": "Hello world"}]
completion = "How may I assist you today?"
prompt_cost = calculate_prompt_cost(prompt, model)
completion_cost = calculate_completion_cost(completion, model)
print(f"{prompt_cost} + {completion_cost} = {prompt_cost + completion_cost}")
# 0.0000135 + 0.000014 = 0.0000275Recommended: PyPI:
pip install tokencostCalculating the cost of prompts and completions from OpenAI requests
from openai import OpenAI
client = OpenAI()
model = "gpt-3.5-turbo"
prompt = [{ "role": "user", "content": "Say this is a test"}]
chat_completion = client.chat.completions.create(
messages=prompt, model=model
)
completion = chat_completion.choices[0].message.content
# "This is a test."
prompt_cost = calculate_prompt_cost(prompt, model)
completion_cost = calculate_completion_cost(completion, model)
print(f"{prompt_cost} + {completion_cost} = {prompt_cost + completion_cost}")
# 0.0000180 + 0.000010 = 0.0000280Calculating cost using string prompts instead of messages:
from tokencost import calculate_prompt_cost
prompt_string = "Hello world"
response = "How may I assist you today?"
model= "gpt-3.5-turbo"
prompt_cost = calculate_prompt_cost(prompt_string, model)
print(f"Cost: ${prompt_cost}")
# Cost: $3e-06Counting tokens
from tokencost import count_message_tokens, count_string_tokens
message_prompt = [{ "role": "user", "content": "Hello world"}]
# Counting tokens in prompts formatted as message lists
print(count_message_tokens(message_prompt, model="gpt-3.5-turbo"))
# 9
# Alternatively, counting tokens in string prompts
print(count_string_tokens(prompt="Hello world", model="gpt-3.5-turbo"))
# 2Units denominated in USD. All prices can be located in model_prices.json.
- Prices last updated Jan 30, 2024 from LiteLLM's cost dictionary
| Model Name | Prompt Cost (USD) per 1M tokens | Completion Cost (USD) per 1M tokens | Max Prompt Tokens | Max Output Tokens |
|---|---|---|---|---|
| gpt-4 | $30.00 | $60.00 | 8,192 | 4096 |
| gpt-4o | $ 5.00 | $15.00 | 128,000 | 4096 |
| gpt-4o-mini | $0.15 | $0.6 | 128,000 | 4096 |
| gpt-4o-mini-2024-07-18 | $0.15 | $0.6 | 128,000 | 4096 |
| gpt-4o-2024-05-13 | $ 5.00 | $15.00 | 128,000 | 4096 |
| gpt-4-turbo-preview | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-4-0314 | $30.00 | $60.00 | 8,192 | 4096 |
| gpt-4-0613 | $30.00 | $60.00 | 8,192 | 4096 |
| gpt-4-32k | $60.00 | $120.00 | 32,768 | 4096 |
| gpt-4-32k-0314 | $60.00 | $120.00 | 32,768 | 4096 |
| gpt-4-32k-0613 | $60.00 | $120.00 | 32,768 | 4096 |
| gpt-4-turbo | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-4-turbo-2024-04-09 | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-4-1106-preview | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-4-0125-preview | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-4-vision-preview | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-4-1106-vision-preview | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-3.5-turbo | $1.5 | $ 2.00 | 16,385 | 4096 |
| gpt-3.5-turbo-0301 | $1.5 | $ 2.00 | 4,097 | 4096 |
| gpt-3.5-turbo-0613 | $1.5 | $ 2.00 | 4,097 | 4096 |
| gpt-3.5-turbo-1106 | $ 1.00 | $ 2.00 | 16,385 | 4096 |
| gpt-3.5-turbo-0125 | $0.5 | $1.5 | 16,385 | 4096 |
| gpt-3.5-turbo-16k | $ 3.00 | $ 4.00 | 16,385 | 4096 |
| gpt-3.5-turbo-16k-0613 | $ 3.00 | $ 4.00 | 16,385 | 4096 |
| ft:gpt-3.5-turbo | $ 3.00 | $ 6.00 | 4,097 | 4096 |
| ft:gpt-4-0613 | $30.00 | $60.00 | 8,192 | 4096 |
| ft:gpt-4o-2024-05-13 | $ 5.00 | $15.00 | 128,000 | 4096 |
| ft:davinci-002 | $ 2.00 | $ 2.00 | 16,384 | 4096 |
| ft:babbage-002 | $0.4 | $0.4 | 16,384 | 4096 |
| text-embedding-3-large | $0.13 | $ 0.00 | 8,191 | nan |
| text-embedding-3-small | $0.02 | $ 0.00 | 8,191 | nan |
| text-embedding-ada-002 | $0.1 | $ 0.00 | 8,191 | nan |
| text-embedding-ada-002-v2 | $0.1 | $ 0.00 | 8,191 | nan |
| text-moderation-stable | $ 0.00 | $ 0.00 | 32,768 | 0 |
| text-moderation-007 | $ 0.00 | $ 0.00 | 32,768 | 0 |
| text-moderation-latest | $ 0.00 | $ 0.00 | 32,768 | 0 |
| 256-x-256/dall-e-2 | -- | -- | nan | nan |
| 512-x-512/dall-e-2 | -- | -- | nan | nan |
| 1024-x-1024/dall-e-2 | -- | -- | nan | nan |
| hd/1024-x-1792/dall-e-3 | -- | -- | nan | nan |
| hd/1792-x-1024/dall-e-3 | -- | -- | nan | nan |
| hd/1024-x-1024/dall-e-3 | -- | -- | nan | nan |
| standard/1024-x-1792/dall-e-3 | -- | -- | nan | nan |
| standard/1792-x-1024/dall-e-3 | -- | -- | nan | nan |
| standard/1024-x-1024/dall-e-3 | -- | -- | nan | nan |
| whisper-1 | -- | -- | nan | nan |
| tts-1 | -- | -- | nan | nan |
| tts-1-hd | -- | -- | nan | nan |
| azure/tts-1 | -- | -- | nan | nan |
| azure/tts-1-hd | -- | -- | nan | nan |
| azure/whisper-1 | -- | -- | nan | nan |
| azure/gpt-4o | $ 5.00 | $15.00 | 128,000 | 4096 |
| azure/gpt-4-turbo-2024-04-09 | $10.00 | $30.00 | 128,000 | 4096 |
| azure/gpt-4-0125-preview | $10.00 | $30.00 | 128,000 | 4096 |
| azure/gpt-4-1106-preview | $10.00 | $30.00 | 128,000 | 4096 |
| azure/gpt-4-0613 | $30.00 | $60.00 | 8,192 | 4096 |
| azure/gpt-4-32k-0613 | $60.00 | $120.00 | 32,768 | 4096 |
| azure/gpt-4-32k | $60.00 | $120.00 | 32,768 | 4096 |
| azure/gpt-4 | $30.00 | $60.00 | 8,192 | 4096 |
| azure/gpt-4-turbo | $10.00 | $30.00 | 128,000 | 4096 |
| azure/gpt-4-turbo-vision-preview | $10.00 | $30.00 | 128,000 | 4096 |
| azure/gpt-35-turbo-16k-0613 | $ 3.00 | $ 4.00 | 16,385 | 4096 |
| azure/gpt-35-turbo-1106 | $ 1.00 | $ 2.00 | 16,384 | 4096 |
| azure/gpt-35-turbo-0125 | $0.5 | $1.5 | 16,384 | 4096 |
| azure/gpt-35-turbo-16k | $ 3.00 | $ 4.00 | 16,385 | 4096 |
| azure/gpt-35-turbo | $0.5 | $1.5 | 4,097 | 4096 |
| azure/gpt-3.5-turbo-instruct-0914 | $1.5 | $ 2.00 | 4,097 | nan |
| azure/gpt-35-turbo-instruct | $1.5 | $ 2.00 | 4,097 | nan |
| azure/mistral-large-latest | $ 8.00 | $24.00 | 32,000 | nan |
| azure/mistral-large-2402 | $ 8.00 | $24.00 | 32,000 | nan |
| azure/command-r-plus | $ 3.00 | $15.00 | 128,000 | 4096 |
| azure/ada | $0.1 | $ 0.00 | 8,191 | nan |
| azure/text-embedding-ada-002 | $0.1 | $ 0.00 | 8,191 | nan |
| azure/text-embedding-3-large | $0.13 | $ 0.00 | 8,191 | nan |
| azure/text-embedding-3-small | $0.02 | $ 0.00 | 8,191 | nan |
| azure/standard/1024-x-1024/dall-e-3 | -- | $ 0.00 | nan | nan |
| azure/hd/1024-x-1024/dall-e-3 | -- | $ 0.00 | nan | nan |
| azure/standard/1024-x-1792/dall-e-3 | -- | $ 0.00 | nan | nan |
| azure/standard/1792-x-1024/dall-e-3 | -- | $ 0.00 | nan | nan |
| azure/hd/1024-x-1792/dall-e-3 | -- | $ 0.00 | nan | nan |
| azure/hd/1792-x-1024/dall-e-3 | -- | $ 0.00 | nan | nan |
| azure/standard/1024-x-1024/dall-e-2 | -- | $ 0.00 | nan | nan |
| azure_ai/jamba-instruct | $0.5 | $0.7 | 70,000 | 4096 |
| azure_ai/mistral-large | $ 4.00 | $12.00 | 32,000 | 8191 |
| azure_ai/mistral-small | $ 1.00 | $ 3.00 | 32,000 | 8191 |
| azure_ai/Meta-Llama-3-70B-Instruct | $1.1 | $0.37 | 8,192 | 8192 |
| azure_ai/Meta-Llama-31-8B-Instruct | $0.3 | $0.61 | 128,000 | 128000 |
| azure_ai/Meta-Llama-31-70B-Instruct | $2.68 | $3.54 | 128,000 | 128000 |
| azure_ai/Meta-Llama-31-405B-Instruct | $5.33 | $16.00 | 128,000 | 128000 |
| babbage-002 | $0.4 | $0.4 | 16,384 | 4096 |
| davinci-002 | $ 2.00 | $ 2.00 | 16,384 | 4096 |
| gpt-3.5-turbo-instruct | $1.5 | $ 2.00 | 8,192 | 4096 |
| gpt-3.5-turbo-instruct-0914 | $1.5 | $ 2.00 | 8,192 | 4097 |
| claude-instant-1 | $1.63 | $5.51 | 100,000 | 8191 |
| mistral/mistral-tiny | $0.25 | $0.25 | 32,000 | 8191 |
| mistral/mistral-small | $ 1.00 | $ 3.00 | 32,000 | 8191 |
| mistral/mistral-small-latest | $ 1.00 | $ 3.00 | 32,000 | 8191 |
| mistral/mistral-medium | $2.7 | $8.1 | 32,000 | 8191 |
| mistral/mistral-medium-latest | $2.7 | $8.1 | 32,000 | 8191 |
| mistral/mistral-medium-2312 | $2.7 | $8.1 | 32,000 | 8191 |
| mistral/mistral-large-latest | $ 3.00 | $ 9.00 | 128,000 | 128000 |
| mistral/mistral-large-2402 | $ 4.00 | $12.00 | 32,000 | 8191 |
| mistral/mistral-large-2407 | $ 3.00 | $ 9.00 | 128,000 | 128000 |
| mistral/open-mistral-7b | $0.25 | $0.25 | 32,000 | 8191 |
| mistral/open-mixtral-8x7b | $0.7 | $0.7 | 32,000 | 8191 |
| mistral/open-mixtral-8x22b | $ 2.00 | $ 6.00 | 64,000 | 8191 |
| mistral/codestral-latest | $ 1.00 | $ 3.00 | 32,000 | 8191 |
| mistral/codestral-2405 | $ 1.00 | $ 3.00 | 32,000 | 8191 |
| mistral/open-mistral-nemo | $0.3 | $0.3 | 128,000 | 128000 |
| mistral/open-mistral-nemo-2407 | $0.3 | $0.3 | 128,000 | 128000 |
| mistral/open-codestral-mamba | $0.25 | $0.25 | 256,000 | 256000 |
| mistral/codestral-mamba-latest | $0.25 | $0.25 | 256,000 | 256000 |
| mistral/mistral-embed | $0.1 | -- | 8,192 | nan |
| deepseek-chat | $0.14 | $0.28 | 32,000 | 4096 |
| codestral/codestral-latest | $ 0.00 | $ 0.00 | 32,000 | 8191 |
| codestral/codestral-2405 | $ 0.00 | $ 0.00 | 32,000 | 8191 |
| text-completion-codestral/codestral-latest | $ 0.00 | $ 0.00 | 32,000 | 8191 |
| text-completion-codestral/codestral-2405 | $ 0.00 | $ 0.00 | 32,000 | 8191 |
| deepseek-coder | $0.14 | $0.28 | 128,000 | 4096 |
| groq/llama2-70b-4096 | $0.7 | $0.8 | 4,096 | 4096 |
| groq/llama3-8b-8192 | $0.05 | $0.08 | 8,192 | 8192 |
| groq/llama3-70b-8192 | $0.59 | $0.79 | 8,192 | 8192 |
| groq/llama-3.1-8b-instant | $0.59 | $0.79 | 8,192 | 8192 |
| groq/llama-3.1-70b-versatile | $0.59 | $0.79 | 8,192 | 8192 |
| groq/llama-3.1-405b-reasoning | $0.59 | $0.79 | 8,192 | 8192 |
| groq/mixtral-8x7b-32768 | $0.24 | $0.24 | 32,768 | 32768 |
| groq/gemma-7b-it | $0.07 | $0.07 | 8,192 | 8192 |
| groq/llama3-groq-70b-8192-tool-use-preview | $0.89 | $0.89 | 8,192 | 8192 |
| groq/llama3-groq-8b-8192-tool-use-preview | $0.19 | $0.19 | 8,192 | 8192 |
| friendliai/mixtral-8x7b-instruct-v0-1 | $0.4 | $0.4 | 32,768 | 32768 |
| friendliai/meta-llama-3-8b-instruct | $0.1 | $0.1 | 8,192 | 8192 |
| friendliai/meta-llama-3-70b-instruct | $0.8 | $0.8 | 8,192 | 8192 |
| claude-instant-1.2 | $0.163 | $0.551 | 100,000 | 8191 |
| claude-2 | $ 8.00 | $24.00 | 100,000 | 8191 |
| claude-2.1 | $ 8.00 | $24.00 | 200,000 | 8191 |
| claude-3-haiku-20240307 | $0.25 | $1.25 | 200,000 | 4096 |
| claude-3-opus-20240229 | $15.00 | $75.00 | 200,000 | 4096 |
| claude-3-sonnet-20240229 | $ 3.00 | $15.00 | 200,000 | 4096 |
| claude-3-5-sonnet-20240620 | $ 3.00 | $15.00 | 200,000 | 4096 |
| text-bison | -- | -- | 8,192 | 2048 |
| text-bison@001 | -- | -- | 8,192 | 1024 |
| text-bison@002 | -- | -- | 8,192 | 1024 |
| text-bison32k | $0.125 | $0.125 | 8,192 | 1024 |
| text-bison32k@002 | $0.125 | $0.125 | 8,192 | 1024 |
| text-unicorn | $10.00 | $28.00 | 8,192 | 1024 |
| text-unicorn@001 | $10.00 | $28.00 | 8,192 | 1024 |
| chat-bison | $0.125 | $0.125 | 8,192 | 4096 |
| chat-bison@001 | $0.125 | $0.125 | 8,192 | 4096 |
| chat-bison@002 | $0.125 | $0.125 | 8,192 | 4096 |
| chat-bison-32k | $0.125 | $0.125 | 32,000 | 8192 |
| chat-bison-32k@002 | $0.125 | $0.125 | 32,000 | 8192 |
| code-bison | $0.125 | $0.125 | 6,144 | 1024 |
| code-bison@001 | $0.125 | $0.125 | 6,144 | 1024 |
| code-bison@002 | $0.125 | $0.125 | 6,144 | 1024 |
| code-bison32k | $0.125 | $0.125 | 6,144 | 1024 |
| code-bison-32k@002 | $0.125 | $0.125 | 6,144 | 1024 |
| code-gecko@001 | $0.125 | $0.125 | 2,048 | 64 |
| code-gecko@002 | $0.125 | $0.125 | 2,048 | 64 |
| code-gecko | $0.125 | $0.125 | 2,048 | 64 |
| code-gecko-latest | $0.125 | $0.125 | 2,048 | 64 |
| codechat-bison@latest | $0.125 | $0.125 | 6,144 | 1024 |
| codechat-bison | $0.125 | $0.125 | 6,144 | 1024 |
| codechat-bison@001 | $0.125 | $0.125 | 6,144 | 1024 |
| codechat-bison@002 | $0.125 | $0.125 | 6,144 | 1024 |
| codechat-bison-32k | $0.125 | $0.125 | 32,000 | 8192 |
| codechat-bison-32k@002 | $0.125 | $0.125 | 32,000 | 8192 |
| gemini-pro | $0.5 | $1.5 | 32,760 | 8192 |
| gemini-1.0-pro | $0.5 | $1.5 | 32,760 | 8192 |
| gemini-1.0-pro-001 | $0.5 | $1.5 | 32,760 | 8192 |
| gemini-1.0-ultra | $0.5 | $1.5 | 8,192 | 2048 |
| gemini-1.0-ultra-001 | $0.5 | $1.5 | 8,192 | 2048 |
| gemini-1.0-pro-002 | $0.5 | $1.5 | 32,760 | 8192 |
| gemini-1.5-pro | $ 5.00 | $15.00 | 2,097,152 | 8192 |
| gemini-1.5-pro-001 | $ 5.00 | $15.00 | 1,000,000 | 8192 |
| gemini-1.5-pro-preview-0514 | $ 5.00 | $15.00 | 1,000,000 | 8192 |
| gemini-1.5-pro-preview-0215 | $ 5.00 | $15.00 | 1,000,000 | 8192 |
| gemini-1.5-pro-preview-0409 | $ 5.00 | $15.00 | 1,000,000 | 8192 |
| gemini-1.5-flash | $0.5 | $1.5 | 1,000,000 | 8192 |
| gemini-1.5-flash-001 | $0.5 | $1.5 | 1,000,000 | 8192 |
| gemini-1.5-flash-preview-0514 | $0.5 | $1.5 | 1,000,000 | 8192 |
| gemini-experimental | $ 0.00 | $ 0.00 | 1,000,000 | 8192 |
| gemini-pro-vision | $0.25 | $0.5 | 16,384 | 2048 |
| gemini-1.0-pro-vision | $0.25 | $0.5 | 16,384 | 2048 |
| gemini-1.0-pro-vision-001 | $0.25 | $0.5 | 16,384 | 2048 |
| medlm-medium | -- | -- | 32,768 | 8192 |
| medlm-large | -- | -- | 8,192 | 1024 |
| vertex_ai/claude-3-sonnet@20240229 | $ 3.00 | $15.00 | 200,000 | 4096 |
| vertex_ai/claude-3-5-sonnet@20240620 | $ 3.00 | $15.00 | 200,000 | 4096 |
| vertex_ai/claude-3-haiku@20240307 | $0.25 | $1.25 | 200,000 | 4096 |
| vertex_ai/claude-3-opus@20240229 | $15.00 | $75.00 | 200,000 | 4096 |
| vertex_ai/meta/llama3-405b-instruct-maas | $ 0.00 | $ 0.00 | 32,000 | 32000 |
| vertex_ai/imagegeneration@006 | -- | -- | nan | nan |
| text-embedding-004 | $0.00625 | $ 0.00 | 3,072 | nan |
| text-multilingual-embedding-002 | $0.00625 | $ 0.00 | 2,048 | nan |
| textembedding-gecko | $0.00625 | $ 0.00 | 3,072 | nan |
| textembedding-gecko-multilingual | $0.00625 | $ 0.00 | 3,072 | nan |
| textembedding-gecko-multilingual@001 | $0.00625 | $ 0.00 | 3,072 | nan |
| textembedding-gecko@001 | $0.00625 | $ 0.00 | 3,072 | nan |
| textembedding-gecko@003 | $0.00625 | $ 0.00 | 3,072 | nan |
| text-embedding-preview-0409 | $0.00625 | $ 0.00 | 3,072 | nan |
| text-multilingual-embedding-preview-0409 | $0.00625 | $ 0.00 | 3,072 | nan |
| palm/chat-bison | $0.125 | $0.125 | 8,192 | 4096 |
| palm/chat-bison-001 | $0.125 | $0.125 | 8,192 | 4096 |
| palm/text-bison | $0.125 | $0.125 | 8,192 | 1024 |
| palm/text-bison-001 | $0.125 | $0.125 | 8,192 | 1024 |
| palm/text-bison-safety-off | $0.125 | $0.125 | 8,192 | 1024 |
| palm/text-bison-safety-recitation-off | $0.125 | $0.125 | 8,192 | 1024 |
| gemini/gemini-1.5-flash | $0.35 | $1.05 | 1,000,000 | 8192 |
| gemini/gemini-1.5-flash-latest | $0.35 | $1.05 | 1,000,000 | 8192 |
| gemini/gemini-pro | $0.35 | $1.05 | 32,760 | 8192 |
| gemini/gemini-1.5-pro | $3.5 | $10.5 | 2,097,152 | 8192 |
| gemini/gemini-1.5-pro-latest | $3.5 | $1.05 | 1,048,576 | 8192 |
| gemini/gemini-pro-vision | $0.35 | $1.05 | 30,720 | 2048 |
| gemini/gemini-gemma-2-27b-it | $0.35 | $1.05 | nan | 8192 |
| gemini/gemini-gemma-2-9b-it | $0.35 | $1.05 | nan | 8192 |
| command-r | $0.5 | $1.5 | 128,000 | 4096 |
| command-light | $15.00 | $15.00 | 4,096 | 4096 |
| command-r-plus | $ 3.00 | $15.00 | 128,000 | 4096 |
| command-nightly | $15.00 | $15.00 | 4,096 | 4096 |
| command | $15.00 | $15.00 | 4,096 | 4096 |
| command-medium-beta | $15.00 | $15.00 | 4,096 | 4096 |
| command-xlarge-beta | $15.00 | $15.00 | 4,096 | 4096 |
| replicate/meta/llama-2-13b | $0.1 | $0.5 | 4,096 | 4096 |
| replicate/meta/llama-2-13b-chat | $0.1 | $0.5 | 4,096 | 4096 |
| replicate/meta/llama-2-70b | $0.65 | $2.75 | 4,096 | 4096 |
| replicate/meta/llama-2-70b-chat | $0.65 | $2.75 | 4,096 | 4096 |
| replicate/meta/llama-2-7b | $0.05 | $0.25 | 4,096 | 4096 |
| replicate/meta/llama-2-7b-chat | $0.05 | $0.25 | 4,096 | 4096 |
| replicate/meta/llama-3-70b | $0.65 | $2.75 | 8,192 | 8192 |
| replicate/meta/llama-3-70b-instruct | $0.65 | $2.75 | 8,192 | 8192 |
| replicate/meta/llama-3-8b | $0.05 | $0.25 | 8,086 | 8086 |
| replicate/meta/llama-3-8b-instruct | $0.05 | $0.25 | 8,086 | 8086 |
| replicate/mistralai/mistral-7b-v0.1 | $0.05 | $0.25 | 4,096 | 4096 |
| replicate/mistralai/mistral-7b-instruct-v0.2 | $0.05 | $0.25 | 4,096 | 4096 |
| replicate/mistralai/mixtral-8x7b-instruct-v0.1 | $0.3 | $ 1.00 | 4,096 | 4096 |
| openrouter/deepseek/deepseek-coder | $0.14 | $0.28 | 32,000 | 4096 |
| openrouter/microsoft/wizardlm-2-8x22b:nitro | $ 1.00 | $ 1.00 | nan | nan |
| openrouter/google/gemini-pro-1.5 | $2.5 | $7.5 | 1,000,000 | 8192 |
| openrouter/mistralai/mixtral-8x22b-instruct | $0.65 | $0.65 | nan | nan |
| openrouter/cohere/command-r-plus | $ 3.00 | $15.00 | nan | nan |
| openrouter/databricks/dbrx-instruct | $0.6 | $0.6 | nan | nan |
| openrouter/anthropic/claude-3-haiku | $0.25 | $1.25 | nan | nan |
| openrouter/anthropic/claude-3-haiku-20240307 | $0.25 | $1.25 | 200,000 | 4096 |
| openrouter/anthropic/claude-3.5-sonnet | $ 3.00 | $15.00 | 200,000 | 4096 |
| openrouter/anthropic/claude-3-sonnet | $ 3.00 | $15.00 | nan | nan |
| openrouter/mistralai/mistral-large | $ 8.00 | $24.00 | nan | nan |
| openrouter/cognitivecomputations/dolphin-mixtral-8x7b | $0.5 | $0.5 | nan | nan |
| openrouter/google/gemini-pro-vision | $0.125 | $0.375 | nan | nan |
| openrouter/fireworks/firellava-13b | $0.2 | $0.2 | nan | nan |
| openrouter/meta-llama/llama-3-8b-instruct:free | $ 0.00 | $ 0.00 | nan | nan |
| openrouter/meta-llama/llama-3-8b-instruct:extended | $0.225 | $2.25 | nan | nan |
| openrouter/meta-llama/llama-3-70b-instruct:nitro | $0.9 | $0.9 | nan | nan |
| openrouter/meta-llama/llama-3-70b-instruct | $0.59 | $0.79 | nan | nan |
| openrouter/openai/gpt-4o | $ 5.00 | $15.00 | 128,000 | 4096 |
| openrouter/openai/gpt-4o-2024-05-13 | $ 5.00 | $15.00 | 128,000 | 4096 |
| openrouter/openai/gpt-4-vision-preview | $10.00 | $30.00 | nan | nan |
| openrouter/openai/gpt-3.5-turbo | $1.5 | $ 2.00 | nan | nan |
| openrouter/openai/gpt-3.5-turbo-16k | $ 3.00 | $ 4.00 | nan | nan |
| openrouter/openai/gpt-4 | $30.00 | $60.00 | nan | nan |
| openrouter/anthropic/claude-instant-v1 | $1.63 | $5.51 | nan | 8191 |
| openrouter/anthropic/claude-2 | $11.02 | $32.68 | nan | 8191 |
| openrouter/anthropic/claude-3-opus | $15.00 | $75.00 | 200,000 | 4096 |
| openrouter/google/palm-2-chat-bison | $0.5 | $0.5 | nan | nan |
| openrouter/google/palm-2-codechat-bison | $0.5 | $0.5 | nan | nan |
| openrouter/meta-llama/llama-2-13b-chat | $0.2 | $0.2 | nan | nan |
| openrouter/meta-llama/llama-2-70b-chat | $1.5 | $1.5 | nan | nan |
| openrouter/meta-llama/codellama-34b-instruct | $0.5 | $0.5 | nan | nan |
| openrouter/nousresearch/nous-hermes-llama2-13b | $0.2 | $0.2 | nan | nan |
| openrouter/mancer/weaver | $5.625 | $5.625 | nan | nan |
| openrouter/gryphe/mythomax-l2-13b | $1.875 | $1.875 | nan | nan |
| openrouter/jondurbin/airoboros-l2-70b-2.1 | $13.875 | $13.875 | nan | nan |
| openrouter/undi95/remm-slerp-l2-13b | $1.875 | $1.875 | nan | nan |
| openrouter/pygmalionai/mythalion-13b | $1.875 | $1.875 | nan | nan |
| openrouter/mistralai/mistral-7b-instruct | $0.13 | $0.13 | nan | nan |
| openrouter/mistralai/mistral-7b-instruct:free | $ 0.00 | $ 0.00 | nan | nan |
| j2-ultra | $15.00 | $15.00 | 8,192 | 8192 |
| j2-mid | $10.00 | $10.00 | 8,192 | 8192 |
| j2-light | $ 3.00 | $ 3.00 | 8,192 | 8192 |
| dolphin | $0.5 | $0.5 | 16,384 | 16384 |
| chatdolphin | $0.5 | $0.5 | 16,384 | 16384 |
| luminous-base | $30.00 | $33.00 | nan | nan |
| luminous-base-control | $37.5 | $41.25 | nan | nan |
| luminous-extended | $45.00 | $49.5 | nan | nan |
| luminous-extended-control | $56.25 | $61.875 | nan | nan |
| luminous-supreme | $175.00 | $192.5 | nan | nan |
| luminous-supreme-control | $218.75 | $240.625 | nan | nan |
| ai21.j2-mid-v1 | $12.5 | $12.5 | 8,191 | 8191 |
| ai21.j2-ultra-v1 | $18.8 | $18.8 | 8,191 | 8191 |
| ai21.jamba-instruct-v1:0 | $0.5 | $0.7 | 70,000 | 4096 |
| amazon.titan-text-lite-v1 | $0.3 | $0.4 | 42,000 | 4000 |
| amazon.titan-text-express-v1 | $1.3 | $1.7 | 42,000 | 8000 |
| amazon.titan-embed-text-v1 | $0.1 | $ 0.00 | 8,192 | nan |
| amazon.titan-embed-text-v2:0 | $0.2 | $ 0.00 | 8,192 | nan |
| mistral.mistral-7b-instruct-v0:2 | $0.15 | $0.2 | 32,000 | 8191 |
| mistral.mixtral-8x7b-instruct-v0:1 | $0.45 | $0.7 | 32,000 | 8191 |
| mistral.mistral-large-2402-v1:0 | $ 8.00 | $24.00 | 32,000 | 8191 |
| bedrock/us-west-2/mistral.mixtral-8x7b-instruct-v0:1 | $0.45 | $0.7 | 32,000 | 8191 |
| bedrock/us-east-1/mistral.mixtral-8x7b-instruct-v0:1 | $0.45 | $0.7 | 32,000 | 8191 |
| bedrock/eu-west-3/mistral.mixtral-8x7b-instruct-v0:1 | $0.59 | $0.91 | 32,000 | 8191 |
| bedrock/us-west-2/mistral.mistral-7b-instruct-v0:2 | $0.15 | $0.2 | 32,000 | 8191 |
| bedrock/us-east-1/mistral.mistral-7b-instruct-v0:2 | $0.15 | $0.2 | 32,000 | 8191 |
| bedrock/eu-west-3/mistral.mistral-7b-instruct-v0:2 | $0.2 | $0.26 | 32,000 | 8191 |
| bedrock/us-east-1/mistral.mistral-large-2402-v1:0 | $ 8.00 | $24.00 | 32,000 | 8191 |
| bedrock/us-west-2/mistral.mistral-large-2402-v1:0 | $ 8.00 | $24.00 | 32,000 | 8191 |
| bedrock/eu-west-3/mistral.mistral-large-2402-v1:0 | $10.4 | $31.2 | 32,000 | 8191 |
| anthropic.claude-3-sonnet-20240229-v1:0 | $ 3.00 | $15.00 | 200,000 | 4096 |
| anthropic.claude-3-5-sonnet-20240620-v1:0 | $ 3.00 | $15.00 | 200,000 | 4096 |
| anthropic.claude-3-haiku-20240307-v1:0 | $0.25 | $1.25 | 200,000 | 4096 |
| anthropic.claude-3-opus-20240229-v1:0 | $15.00 | $75.00 | 200,000 | 4096 |
| anthropic.claude-v1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/us-east-1/anthropic.claude-v1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/us-west-2/anthropic.claude-v1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/ap-northeast-1/anthropic.claude-v1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/ap-northeast-1/1-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/ap-northeast-1/6-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/anthropic.claude-v1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/eu-central-1/1-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/6-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/1-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/6-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/1-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/6-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| anthropic.claude-v2 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/us-east-1/anthropic.claude-v2 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/us-west-2/anthropic.claude-v2 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/ap-northeast-1/anthropic.claude-v2 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/ap-northeast-1/1-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/ap-northeast-1/6-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/anthropic.claude-v2 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/eu-central-1/1-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/6-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/1-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/6-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/1-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/6-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| anthropic.claude-v2:1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/us-east-1/anthropic.claude-v2:1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/us-west-2/anthropic.claude-v2:1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/ap-northeast-1/anthropic.claude-v2:1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/ap-northeast-1/1-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/ap-northeast-1/6-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/anthropic.claude-v2:1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/eu-central-1/1-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/6-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/1-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/6-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/1-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/6-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| anthropic.claude-instant-v1 | $1.63 | $5.51 | 100,000 | 8191 |
| bedrock/us-east-1/anthropic.claude-instant-v1 | $0.8 | $2.4 | 100,000 | 8191 |
| bedrock/us-east-1/1-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/6-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/1-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/6-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/anthropic.claude-instant-v1 | $0.8 | $2.4 | 100,000 | 8191 |
| bedrock/ap-northeast-1/anthropic.claude-instant-v1 | $2.23 | $7.55 | 100,000 | 8191 |
| bedrock/ap-northeast-1/1-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/ap-northeast-1/6-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/anthropic.claude-instant-v1 | $2.48 | $8.38 | 100,000 | 8191 |
| bedrock/eu-central-1/1-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/6-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| cohere.command-text-v14 | $1.5 | $ 2.00 | 4,096 | 4096 |
| bedrock/*/1-month-commitment/cohere.command-text-v14 | -- | -- | 4,096 | 4096 |
| bedrock/*/6-month-commitment/cohere.command-text-v14 | -- | -- | 4,096 | 4096 |
| cohere.command-light-text-v14 | $0.3 | $0.6 | 4,096 | 4096 |
| bedrock/*/1-month-commitment/cohere.command-light-text-v14 | -- | -- | 4,096 | 4096 |
| bedrock/*/6-month-commitment/cohere.command-light-text-v14 | -- | -- | 4,096 | 4096 |
| cohere.command-r-plus-v1:0 | $ 3.00 | $15.00 | 128,000 | 4096 |
| cohere.command-r-v1:0 | $0.5 | $1.5 | 128,000 | 4096 |
| cohere.embed-english-v3 | $0.1 | $ 0.00 | 512 | nan |
| cohere.embed-multilingual-v3 | $0.1 | $ 0.00 | 512 | nan |
| meta.llama2-13b-chat-v1 | $0.75 | $ 1.00 | 4,096 | 4096 |
| meta.llama2-70b-chat-v1 | $1.95 | $2.56 | 4,096 | 4096 |
| meta.llama3-8b-instruct-v1:0 | $0.4 | $0.6 | 8,192 | 8192 |
| meta.llama3-70b-instruct-v1:0 | $2.65 | $3.5 | 8,192 | 8192 |
| meta.llama3-1-8b-instruct-v1:0 | $0.4 | $0.6 | 128,000 | 2048 |
| meta.llama3-1-70b-instruct-v1:0 | $2.65 | $3.5 | 128,000 | 2048 |
| 512-x-512/50-steps/stability.stable-diffusion-xl-v0 | -- | -- | 77 | nan |
| 512-x-512/max-steps/stability.stable-diffusion-xl-v0 | -- | -- | 77 | nan |
| max-x-max/50-steps/stability.stable-diffusion-xl-v0 | -- | -- | 77 | nan |
| max-x-max/max-steps/stability.stable-diffusion-xl-v0 | -- | -- | 77 | nan |
| 1024-x-1024/50-steps/stability.stable-diffusion-xl-v1 | -- | -- | 77 | nan |
| 1024-x-1024/max-steps/stability.stable-diffusion-xl-v1 | -- | -- | 77 | nan |
| sagemaker/meta-textgeneration-llama-2-7b | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| sagemaker/meta-textgeneration-llama-2-7b-f | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| sagemaker/meta-textgeneration-llama-2-13b | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| sagemaker/meta-textgeneration-llama-2-13b-f | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| sagemaker/meta-textgeneration-llama-2-70b | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| sagemaker/meta-textgeneration-llama-2-70b-b-f | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| together-ai-up-to-4b | $0.1 | $0.1 | nan | nan |
| together-ai-4.1b-8b | $0.2 | $0.2 | nan | nan |
| together-ai-8.1b-21b | $0.3 | $0.3 | nan | nan |
| together-ai-21.1b-41b | $0.8 | $0.8 | nan | nan |
| together-ai-41.1b-80b | $0.9 | $0.9 | nan | nan |
| together-ai-81.1b-110b | $1.8 | $1.8 | nan | nan |
| together_ai/mistralai/Mixtral-8x7B-Instruct-v0.1 | $0.6 | $0.6 | nan | nan |
| together_ai/mistralai/Mistral-7B-Instruct-v0.1 | -- | -- | nan | nan |
| together_ai/togethercomputer/CodeLlama-34b-Instruct | -- | -- | nan | nan |
| ollama/codegemma | $ 0.00 | $ 0.00 | 8,192 | 8192 |
| ollama/llama2 | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| ollama/llama2:13b | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| ollama/llama2:70b | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| ollama/llama2-uncensored | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| ollama/llama3 | $ 0.00 | $ 0.00 | 8,192 | 8192 |
| ollama/llama3:70b | $ 0.00 | $ 0.00 | 8,192 | 8192 |
| ollama/mistral | $ 0.00 | $ 0.00 | 8,192 | 8192 |
| ollama/mistral-7B-Instruct-v0.1 | $ 0.00 | $ 0.00 | 8,192 | 8192 |
| ollama/mistral-7B-Instruct-v0.2 | $ 0.00 | $ 0.00 | 32,768 | 32768 |
| ollama/mixtral-8x7B-Instruct-v0.1 | $ 0.00 | $ 0.00 | 32,768 | 32768 |
| ollama/mixtral-8x22B-Instruct-v0.1 | $ 0.00 | $ 0.00 | 65,536 | 65536 |
| ollama/codellama | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| ollama/orca-mini | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| ollama/vicuna | $ 0.00 | $ 0.00 | 2,048 | 2048 |
| deepinfra/lizpreciatior/lzlv_70b_fp16_hf | $0.7 | $0.9 | 4,096 | 4096 |
| deepinfra/Gryphe/MythoMax-L2-13b | $0.22 | $0.22 | 4,096 | 4096 |
| deepinfra/mistralai/Mistral-7B-Instruct-v0.1 | $0.13 | $0.13 | 32,768 | 8191 |
| deepinfra/meta-llama/Llama-2-70b-chat-hf | $0.7 | $0.9 | 4,096 | 4096 |
| deepinfra/cognitivecomputations/dolphin-2.6-mixtral-8x7b | $0.27 | $0.27 | 32,768 | 8191 |
| deepinfra/codellama/CodeLlama-34b-Instruct-hf | $0.6 | $0.6 | 4,096 | 4096 |
| deepinfra/deepinfra/mixtral | $0.27 | $0.27 | 32,000 | 4096 |
| deepinfra/Phind/Phind-CodeLlama-34B-v2 | $0.6 | $0.6 | 16,384 | 4096 |
| deepinfra/mistralai/Mixtral-8x7B-Instruct-v0.1 | $0.27 | $0.27 | 32,768 | 8191 |
| deepinfra/deepinfra/airoboros-70b | $0.7 | $0.9 | 4,096 | 4096 |
| deepinfra/01-ai/Yi-34B-Chat | $0.6 | $0.6 | 4,096 | 4096 |
| deepinfra/01-ai/Yi-6B-200K | $0.13 | $0.13 | 200,000 | 4096 |
| deepinfra/jondurbin/airoboros-l2-70b-gpt4-1.4.1 | $0.7 | $0.9 | 4,096 | 4096 |
| deepinfra/meta-llama/Llama-2-13b-chat-hf | $0.22 | $0.22 | 4,096 | 4096 |
| deepinfra/amazon/MistralLite | $0.2 | $0.2 | 32,768 | 8191 |
| deepinfra/meta-llama/Llama-2-7b-chat-hf | $0.13 | $0.13 | 4,096 | 4096 |
| deepinfra/meta-llama/Meta-Llama-3-8B-Instruct | $0.08 | $0.08 | 8,191 | 4096 |
| deepinfra/meta-llama/Meta-Llama-3-70B-Instruct | $0.59 | $0.79 | 8,191 | 4096 |
| deepinfra/01-ai/Yi-34B-200K | $0.6 | $0.6 | 200,000 | 4096 |
| deepinfra/openchat/openchat_3.5 | $0.13 | $0.13 | 4,096 | 4096 |
| perplexity/codellama-34b-instruct | $0.35 | $1.4 | 16,384 | 16384 |
| perplexity/codellama-70b-instruct | $0.7 | $2.8 | 16,384 | 16384 |
| perplexity/pplx-7b-chat | $0.07 | $0.28 | 8,192 | 8192 |
| perplexity/pplx-70b-chat | $0.7 | $2.8 | 4,096 | 4096 |
| perplexity/pplx-7b-online | $ 0.00 | $0.28 | 4,096 | 4096 |
| perplexity/pplx-70b-online | $ 0.00 | $2.8 | 4,096 | 4096 |
| perplexity/llama-2-70b-chat | $0.7 | $2.8 | 4,096 | 4096 |
| perplexity/mistral-7b-instruct | $0.07 | $0.28 | 4,096 | 4096 |
| perplexity/mixtral-8x7b-instruct | $0.07 | $0.28 | 4,096 | 4096 |
| perplexity/sonar-small-chat | $0.07 | $0.28 | 16,384 | 16384 |
| perplexity/sonar-small-online | $ 0.00 | $0.28 | 12,000 | 12000 |
| perplexity/sonar-medium-chat | $0.6 | $1.8 | 16,384 | 16384 |
| perplexity/sonar-medium-online | $ 0.00 | $1.8 | 12,000 | 12000 |
| fireworks_ai/firefunction-v2 | $0.9 | $0.9 | 8,192 | 8192 |
| fireworks_ai/mixtral-8x22b-instruct-hf | $1.2 | $1.2 | 65,536 | 65536 |
| fireworks_ai/qwen2-72b-instruct | $0.9 | $0.9 | 32,768 | 32768 |
| fireworks_ai/yi-large | $ 3.00 | $ 3.00 | 32,768 | 32768 |
| fireworks_ai/deepseek-coder-v2-instruct | $1.2 | $1.2 | 65,536 | 8192 |
| anyscale/mistralai/Mistral-7B-Instruct-v0.1 | $0.15 | $0.15 | 16,384 | 16384 |
| anyscale/mistralai/Mixtral-8x7B-Instruct-v0.1 | $0.15 | $0.15 | 16,384 | 16384 |
| anyscale/mistralai/Mixtral-8x22B-Instruct-v0.1 | $0.9 | $0.9 | 65,536 | 65536 |
| anyscale/HuggingFaceH4/zephyr-7b-beta | $0.15 | $0.15 | 16,384 | 16384 |
| anyscale/google/gemma-7b-it | $0.15 | $0.15 | 8,192 | 8192 |
| anyscale/meta-llama/Llama-2-7b-chat-hf | $0.15 | $0.15 | 4,096 | 4096 |
| anyscale/meta-llama/Llama-2-13b-chat-hf | $0.25 | $0.25 | 4,096 | 4096 |
| anyscale/meta-llama/Llama-2-70b-chat-hf | $ 1.00 | $ 1.00 | 4,096 | 4096 |
| anyscale/codellama/CodeLlama-34b-Instruct-hf | $ 1.00 | $ 1.00 | 4,096 | 4096 |
| anyscale/codellama/CodeLlama-70b-Instruct-hf | $ 1.00 | $ 1.00 | 4,096 | 4096 |
| anyscale/meta-llama/Meta-Llama-3-8B-Instruct | $0.15 | $0.15 | 8,192 | 8192 |
| anyscale/meta-llama/Meta-Llama-3-70B-Instruct | $ 1.00 | $ 1.00 | 8,192 | 8192 |
| cloudflare/@cf/meta/llama-2-7b-chat-fp16 | $1.923 | $1.923 | 3,072 | 3072 |
| cloudflare/@cf/meta/llama-2-7b-chat-int8 | $1.923 | $1.923 | 2,048 | 2048 |
| cloudflare/@cf/mistral/mistral-7b-instruct-v0.1 | $1.923 | $1.923 | 8,192 | 8192 |
| cloudflare/@hf/thebloke/codellama-7b-instruct-awq | $1.923 | $1.923 | 4,096 | 4096 |
| voyage/voyage-01 | $0.1 | $ 0.00 | 4,096 | nan |
| voyage/voyage-lite-01 | $0.1 | $ 0.00 | 4,096 | nan |
| voyage/voyage-large-2 | $0.12 | $ 0.00 | 16,000 | nan |
| voyage/voyage-law-2 | $0.12 | $ 0.00 | 16,000 | nan |
| voyage/voyage-code-2 | $0.12 | $ 0.00 | 16,000 | nan |
| voyage/voyage-2 | $0.1 | $ 0.00 | 4,000 | nan |
| voyage/voyage-lite-02-instruct | $0.1 | $ 0.00 | 4,000 | nan |
| databricks/databricks-dbrx-instruct | $0.75 | $2.25 | 32,768 | 32768 |
| databricks/databricks-meta-llama-3-70b-instruct | $ 1.00 | $ 3.00 | 8,192 | 8192 |
| databricks/databricks-llama-2-70b-chat | $0.5 | $1.5 | 4,096 | 4096 |
| databricks/databricks-mixtral-8x7b-instruct | $0.5 | $ 1.00 | 4,096 | 4096 |
| databricks/databricks-mpt-30b-instruct | $ 1.00 | $ 1.00 | 8,192 | 8192 |
| databricks/databricks-mpt-7b-instruct | $0.5 | $0.5 | 8,192 | 8192 |
| databricks/databricks-bge-large-en | $0.1 | $ 0.00 | 512 | nan |
| :-------------------------------------------------------------------------------------------- | :---------------------------------- | :-------------------------------------- | :-------------------- | --------------------: |
| gpt-4 | $30.00 | $60.00 | 8,192 | 4096 |
| gpt-4o | $ 5.00 | $15.00 | 128,000 | 4096 |
| gpt-4o-mini | $0.15 | $0.6 | 128,000 | 4096 |
| gpt-4o-mini-2024-07-18 | $0.15 | $0.6 | 128,000 | 4096 |
| gpt-4o-2024-05-13 | $ 5.00 | $15.00 | 128,000 | 4096 |
| gpt-4-turbo-preview | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-4-0314 | $30.00 | $60.00 | 8,192 | 4096 |
| gpt-4-0613 | $30.00 | $60.00 | 8,192 | 4096 |
| gpt-4-32k | $60.00 | $120.00 | 32,768 | 4096 |
| gpt-4-32k-0314 | $60.00 | $120.00 | 32,768 | 4096 |
| gpt-4-32k-0613 | $60.00 | $120.00 | 32,768 | 4096 |
| gpt-4-turbo | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-4-turbo-2024-04-09 | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-4-1106-preview | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-4-0125-preview | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-4-vision-preview | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-4-1106-vision-preview | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-3.5-turbo | $1.5 | $ 2.00 | 16,385 | 4096 |
| gpt-3.5-turbo-0301 | $1.5 | $ 2.00 | 4,097 | 4096 |
| gpt-3.5-turbo-0613 | $1.5 | $ 2.00 | 4,097 | 4096 |
| gpt-3.5-turbo-1106 | $ 1.00 | $ 2.00 | 16,385 | 4096 |
| gpt-3.5-turbo-0125 | $0.5 | $1.5 | 16,385 | 4096 |
| gpt-3.5-turbo-16k | $ 3.00 | $ 4.00 | 16,385 | 4096 |
| gpt-3.5-turbo-16k-0613 | $ 3.00 | $ 4.00 | 16,385 | 4096 |
| ft:gpt-3.5-turbo | $ 3.00 | $ 6.00 | 4,097 | 4096 |
| ft:gpt-4-0613 | $30.00 | $60.00 | 8,192 | 4096 |
| ft:gpt-4o-2024-05-13 | $ 5.00 | $15.00 | 128,000 | 4096 |
| ft:davinci-002 | $ 2.00 | $ 2.00 | 16,384 | 4096 |
| ft:babbage-002 | $0.4 | $0.4 | 16,384 | 4096 |
| text-embedding-3-large | $0.13 | $ 0.00 | 8,191 | nan |
| text-embedding-3-small | $0.02 | $ 0.00 | 8,191 | nan |
| text-embedding-ada-002 | $0.1 | $ 0.00 | 8,191 | nan |
| text-embedding-ada-002-v2 | $0.1 | $ 0.00 | 8,191 | nan |
| text-moderation-stable | $ 0.00 | $ 0.00 | 32,768 | 0 |
| text-moderation-007 | $ 0.00 | $ 0.00 | 32,768 | 0 |
| text-moderation-latest | $ 0.00 | $ 0.00 | 32,768 | 0 |
| 256-x-256/dall-e-2 | -- | -- | nan | nan |
| 512-x-512/dall-e-2 | -- | -- | nan | nan |
| 1024-x-1024/dall-e-2 | -- | -- | nan | nan |
| hd/1024-x-1792/dall-e-3 | -- | -- | nan | nan |
| hd/1792-x-1024/dall-e-3 | -- | -- | nan | nan |
| hd/1024-x-1024/dall-e-3 | -- | -- | nan | nan |
| standard/1024-x-1792/dall-e-3 | -- | -- | nan | nan |
| standard/1792-x-1024/dall-e-3 | -- | -- | nan | nan |
| standard/1024-x-1024/dall-e-3 | -- | -- | nan | nan |
| whisper-1 | -- | -- | nan | nan |
| tts-1 | -- | -- | nan | nan |
| tts-1-hd | -- | -- | nan | nan |
| azure/tts-1 | -- | -- | nan | nan |
| azure/tts-1-hd | -- | -- | nan | nan |
| azure/whisper-1 | -- | -- | nan | nan |
| azure/gpt-4o | $ 5.00 | $15.00 | 128,000 | 4096 |
| azure/gpt-4-turbo-2024-04-09 | $10.00 | $30.00 | 128,000 | 4096 |
| azure/gpt-4-0125-preview | $10.00 | $30.00 | 128,000 | 4096 |
| azure/gpt-4-1106-preview | $10.00 | $30.00 | 128,000 | 4096 |
| azure/gpt-4-0613 | $30.00 | $60.00 | 8,192 | 4096 |
| azure/gpt-4-32k-0613 | $60.00 | $120.00 | 32,768 | 4096 |
| azure/gpt-4-32k | $60.00 | $120.00 | 32,768 | 4096 |
| azure/gpt-4 | $30.00 | $60.00 | 8,192 | 4096 |
| azure/gpt-4-turbo | $10.00 | $30.00 | 128,000 | 4096 |
| azure/gpt-4-turbo-vision-preview | $10.00 | $30.00 | 128,000 | 4096 |
| azure/gpt-35-turbo-16k-0613 | $ 3.00 | $ 4.00 | 16,385 | 4096 |
| azure/gpt-35-turbo-1106 | $ 1.00 | $ 2.00 | 16,384 | 4096 |
| azure/gpt-35-turbo-0125 | $0.5 | $1.5 | 16,384 | 4096 |
| azure/gpt-35-turbo-16k | $ 3.00 | $ 4.00 | 16,385 | 4096 |
| azure/gpt-35-turbo | $0.5 | $1.5 | 4,097 | 4096 |
| azure/gpt-3.5-turbo-instruct-0914 | $1.5 | $ 2.00 | 4,097 | nan |
| azure/gpt-35-turbo-instruct | $1.5 | $ 2.00 | 4,097 | nan |
| azure/mistral-large-latest | $ 8.00 | $24.00 | 32,000 | nan |
| azure/mistral-large-2402 | $ 8.00 | $24.00 | 32,000 | nan |
| azure/command-r-plus | $ 3.00 | $15.00 | 128,000 | 4096 |
| azure/ada | $0.1 | $ 0.00 | 8,191 | nan |
| azure/text-embedding-ada-002 | $0.1 | $ 0.00 | 8,191 | nan |
| azure/text-embedding-3-large | $0.13 | $ 0.00 | 8,191 | nan |
| azure/text-embedding-3-small | $0.02 | $ 0.00 | 8,191 | nan |
| azure/standard/1024-x-1024/dall-e-3 | -- | $ 0.00 | nan | nan |
| azure/hd/1024-x-1024/dall-e-3 | -- | $ 0.00 | nan | nan |
| azure/standard/1024-x-1792/dall-e-3 | -- | $ 0.00 | nan | nan |
| azure/standard/1792-x-1024/dall-e-3 | -- | $ 0.00 | nan | nan |
| azure/hd/1024-x-1792/dall-e-3 | -- | $ 0.00 | nan | nan |
| azure/hd/1792-x-1024/dall-e-3 | -- | $ 0.00 | nan | nan |
| azure/standard/1024-x-1024/dall-e-2 | -- | $ 0.00 | nan | nan |
| azure_ai/jamba-instruct | $0.5 | $0.7 | 70,000 | 4096 |
| azure_ai/mistral-large | $ 4.00 | $12.00 | 32,000 | 8191 |
| azure_ai/mistral-small | $ 1.00 | $ 3.00 | 32,000 | 8191 |
| azure_ai/Meta-Llama-3-70B-Instruct | $1.1 | $0.37 | 8,192 | 8192 |
| babbage-002 | $0.4 | $0.4 | 16,384 | 4096 |
| davinci-002 | $ 2.00 | $ 2.00 | 16,384 | 4096 |
| gpt-3.5-turbo-instruct | $1.5 | $ 2.00 | 8,192 | 4096 |
| gpt-3.5-turbo-instruct-0914 | $1.5 | $ 2.00 | 8,192 | 4097 |
| claude-instant-1 | $1.63 | $5.51 | 100,000 | 8191 |
| mistral/mistral-tiny | $0.25 | $0.25 | 32,000 | 8191 |
| mistral/mistral-small | $ 1.00 | $ 3.00 | 32,000 | 8191 |
| mistral/mistral-small-latest | $ 1.00 | $ 3.00 | 32,000 | 8191 |
| mistral/mistral-medium | $2.7 | $8.1 | 32,000 | 8191 |
| mistral/mistral-medium-latest | $2.7 | $8.1 | 32,000 | 8191 |
| mistral/mistral-medium-2312 | $2.7 | $8.1 | 32,000 | 8191 |
| mistral/mistral-large-latest | $ 4.00 | $12.00 | 32,000 | 8191 |
| mistral/mistral-large-2402 | $ 4.00 | $12.00 | 32,000 | 8191 |
| mistral/open-mistral-7b | $0.25 | $0.25 | 32,000 | 8191 |
| mistral/open-mixtral-8x7b | $0.7 | $0.7 | 32,000 | 8191 |
| mistral/open-mixtral-8x22b | $ 2.00 | $ 6.00 | 64,000 | 8191 |
| mistral/codestral-latest | $ 1.00 | $ 3.00 | 32,000 | 8191 |
| mistral/codestral-2405 | $ 1.00 | $ 3.00 | 32,000 | 8191 |
| mistral/mistral-embed | $0.1 | -- | 8,192 | nan |
| deepseek-chat | $0.14 | $0.28 | 32,000 | 4096 |
| codestral/codestral-latest | $ 0.00 | $ 0.00 | 32,000 | 8191 |
| codestral/codestral-2405 | $ 0.00 | $ 0.00 | 32,000 | 8191 |
| text-completion-codestral/codestral-latest | $ 0.00 | $ 0.00 | 32,000 | 8191 |
| text-completion-codestral/codestral-2405 | $ 0.00 | $ 0.00 | 32,000 | 8191 |
| deepseek-coder | $0.14 | $0.28 | 128,000 | 4096 |
| groq/llama2-70b-4096 | $0.7 | $0.8 | 4,096 | 4096 |
| groq/llama3-8b-8192 | $0.05 | $0.08 | 8,192 | 8192 |
| groq/llama3-70b-8192 | $0.59 | $0.79 | 8,192 | 8192 |
| groq/mixtral-8x7b-32768 | $0.24 | $0.24 | 32,768 | 32768 |
| groq/gemma-7b-it | $0.07 | $0.07 | 8,192 | 8192 |
| groq/llama3-groq-70b-8192-tool-use-preview | $0.89 | $0.89 | 8,192 | 8192 |
| groq/llama3-groq-8b-8192-tool-use-preview | $0.19 | $0.19 | 8,192 | 8192 |
| friendliai/mixtral-8x7b-instruct-v0-1 | $0.4 | $0.4 | 32,768 | 32768 |
| friendliai/meta-llama-3-8b-instruct | $0.1 | $0.1 | 8,192 | 8192 |
| friendliai/meta-llama-3-70b-instruct | $0.8 | $0.8 | 8,192 | 8192 |
| claude-instant-1.2 | $0.163 | $0.551 | 100,000 | 8191 |
| claude-2 | $ 8.00 | $24.00 | 100,000 | 8191 |
| claude-2.1 | $ 8.00 | $24.00 | 200,000 | 8191 |
| claude-3-haiku-20240307 | $0.25 | $1.25 | 200,000 | 4096 |
| claude-3-opus-20240229 | $15.00 | $75.00 | 200,000 | 4096 |
| claude-3-sonnet-20240229 | $ 3.00 | $15.00 | 200,000 | 4096 |
| claude-3-5-sonnet-20240620 | $ 3.00 | $15.00 | 200,000 | 4096 |
| text-bison | -- | -- | 8,192 | 2048 |
| text-bison@001 | -- | -- | 8,192 | 1024 |
| text-bison@002 | -- | -- | 8,192 | 1024 |
| text-bison32k | $0.125 | $0.125 | 8,192 | 1024 |
| text-bison32k@002 | $0.125 | $0.125 | 8,192 | 1024 |
| text-unicorn | $10.00 | $28.00 | 8,192 | 1024 |
| text-unicorn@001 | $10.00 | $28.00 | 8,192 | 1024 |
| chat-bison | $0.125 | $0.125 | 8,192 | 4096 |
| chat-bison@001 | $0.125 | $0.125 | 8,192 | 4096 |
| chat-bison@002 | $0.125 | $0.125 | 8,192 | 4096 |
| chat-bison-32k | $0.125 | $0.125 | 32,000 | 8192 |
| chat-bison-32k@002 | $0.125 | $0.125 | 32,000 | 8192 |
| code-bison | $0.125 | $0.125 | 6,144 | 1024 |
| code-bison@001 | $0.125 | $0.125 | 6,144 | 1024 |
| code-bison@002 | $0.125 | $0.125 | 6,144 | 1024 |
| code-bison32k | $0.125 | $0.125 | 6,144 | 1024 |
| code-bison-32k@002 | $0.125 | $0.125 | 6,144 | 1024 |
| code-gecko@001 | $0.125 | $0.125 | 2,048 | 64 |
| code-gecko@002 | $0.125 | $0.125 | 2,048 | 64 |
| code-gecko | $0.125 | $0.125 | 2,048 | 64 |
| code-gecko-latest | $0.125 | $0.125 | 2,048 | 64 |
| codechat-bison@latest | $0.125 | $0.125 | 6,144 | 1024 |
| codechat-bison | $0.125 | $0.125 | 6,144 | 1024 |
| codechat-bison@001 | $0.125 | $0.125 | 6,144 | 1024 |
| codechat-bison@002 | $0.125 | $0.125 | 6,144 | 1024 |
| codechat-bison-32k | $0.125 | $0.125 | 32,000 | 8192 |
| codechat-bison-32k@002 | $0.125 | $0.125 | 32,000 | 8192 |
| gemini-pro | $0.5 | $1.5 | 32,760 | 8192 |
| gemini-1.0-pro | $0.5 | $1.5 | 32,760 | 8192 |
| gemini-1.0-pro-001 | $0.5 | $1.5 | 32,760 | 8192 |
| gemini-1.0-ultra | $0.5 | $1.5 | 8,192 | 2048 |
| gemini-1.0-ultra-001 | $0.5 | $1.5 | 8,192 | 2048 |
| gemini-1.0-pro-002 | $0.5 | $1.5 | 32,760 | 8192 |
| gemini-1.5-pro | $ 5.00 | $15.00 | 2,097,152 | 8192 |
| gemini-1.5-pro-001 | $ 5.00 | $15.00 | 1,000,000 | 8192 |
| gemini-1.5-pro-preview-0514 | $ 5.00 | $15.00 | 1,000,000 | 8192 |
| gemini-1.5-pro-preview-0215 | $ 5.00 | $15.00 | 1,000,000 | 8192 |
| gemini-1.5-pro-preview-0409 | $ 5.00 | $15.00 | 1,000,000 | 8192 |
| gemini-1.5-flash | $0.5 | $1.5 | 1,000,000 | 8192 |
| gemini-1.5-flash-001 | $0.5 | $1.5 | 1,000,000 | 8192 |
| gemini-1.5-flash-preview-0514 | $0.5 | $1.5 | 1,000,000 | 8192 |
| gemini-experimental | $ 0.00 | $ 0.00 | 1,000,000 | 8192 |
| gemini-pro-vision | $0.25 | $0.5 | 16,384 | 2048 |
| gemini-1.0-pro-vision | $0.25 | $0.5 | 16,384 | 2048 |
| gemini-1.0-pro-vision-001 | $0.25 | $0.5 | 16,384 | 2048 |
| medlm-medium | -- | -- | 32,768 | 8192 |
| medlm-large | -- | -- | 8,192 | 1024 |
| vertex_ai/claude-3-sonnet@20240229 | $ 3.00 | $15.00 | 200,000 | 4096 |
| vertex_ai/claude-3-5-sonnet@20240620 | $ 3.00 | $15.00 | 200,000 | 4096 |
| vertex_ai/claude-3-haiku@20240307 | $0.25 | $1.25 | 200,000 | 4096 |
| vertex_ai/claude-3-opus@20240229 | $15.00 | $75.00 | 200,000 | 4096 |
| vertex_ai/imagegeneration@006 | -- | -- | nan | nan |
| text-embedding-004 | $0.00625 | $ 0.00 | 3,072 | nan |
| text-multilingual-embedding-002 | $0.00625 | $ 0.00 | 2,048 | nan |
| textembedding-gecko | $0.00625 | $ 0.00 | 3,072 | nan |
| textembedding-gecko-multilingual | $0.00625 | $ 0.00 | 3,072 | nan |
| textembedding-gecko-multilingual@001 | $0.00625 | $ 0.00 | 3,072 | nan |
| textembedding-gecko@001 | $0.00625 | $ 0.00 | 3,072 | nan |
| textembedding-gecko@003 | $0.00625 | $ 0.00 | 3,072 | nan |
| text-embedding-preview-0409 | $0.00625 | $ 0.00 | 3,072 | nan |
| text-multilingual-embedding-preview-0409 | $0.00625 | $ 0.00 | 3,072 | nan |
| palm/chat-bison | $0.125 | $0.125 | 8,192 | 4096 |
| palm/chat-bison-001 | $0.125 | $0.125 | 8,192 | 4096 |
| palm/text-bison | $0.125 | $0.125 | 8,192 | 1024 |
| palm/text-bison-001 | $0.125 | $0.125 | 8,192 | 1024 |
| palm/text-bison-safety-off | $0.125 | $0.125 | 8,192 | 1024 |
| palm/text-bison-safety-recitation-off | $0.125 | $0.125 | 8,192 | 1024 |
| gemini/gemini-1.5-flash | $0.35 | $1.05 | 1,000,000 | 8192 |
| gemini/gemini-1.5-flash-latest | $0.35 | $1.05 | 1,000,000 | 8192 |
| gemini/gemini-pro | $0.35 | $1.05 | 32,760 | 8192 |
| gemini/gemini-1.5-pro | $3.5 | $10.5 | 2,097,152 | 8192 |
| gemini/gemini-1.5-pro-latest | $3.5 | $1.05 | 1,048,576 | 8192 |
| gemini/gemini-pro-vision | $0.35 | $1.05 | 30,720 | 2048 |
| command-r | $0.5 | $1.5 | 128,000 | 4096 |
| command-light | $15.00 | $15.00 | 4,096 | 4096 |
| command-r-plus | $ 3.00 | $15.00 | 128,000 | 4096 |
| command-nightly | $15.00 | $15.00 | 4,096 | 4096 |
| command | $15.00 | $15.00 | 4,096 | 4096 |
| command-medium-beta | $15.00 | $15.00 | 4,096 | 4096 |
| command-xlarge-beta | $15.00 | $15.00 | 4,096 | 4096 |
| replicate/meta/llama-2-13b | $0.1 | $0.5 | 4,096 | 4096 |
| replicate/meta/llama-2-13b-chat | $0.1 | $0.5 | 4,096 | 4096 |
| replicate/meta/llama-2-70b | $0.65 | $2.75 | 4,096 | 4096 |
| replicate/meta/llama-2-70b-chat | $0.65 | $2.75 | 4,096 | 4096 |
| replicate/meta/llama-2-7b | $0.05 | $0.25 | 4,096 | 4096 |
| replicate/meta/llama-2-7b-chat | $0.05 | $0.25 | 4,096 | 4096 |
| replicate/meta/llama-3-70b | $0.65 | $2.75 | 8,192 | 8192 |
| replicate/meta/llama-3-70b-instruct | $0.65 | $2.75 | 8,192 | 8192 |
| replicate/meta/llama-3-8b | $0.05 | $0.25 | 8,086 | 8086 |
| replicate/meta/llama-3-8b-instruct | $0.05 | $0.25 | 8,086 | 8086 |
| replicate/mistralai/mistral-7b-v0.1 | $0.05 | $0.25 | 4,096 | 4096 |
| replicate/mistralai/mistral-7b-instruct-v0.2 | $0.05 | $0.25 | 4,096 | 4096 |
| replicate/mistralai/mixtral-8x7b-instruct-v0.1 | $0.3 | $ 1.00 | 4,096 | 4096 |
| openrouter/deepseek/deepseek-coder | $0.14 | $0.28 | 32,000 | 4096 |
| openrouter/microsoft/wizardlm-2-8x22b:nitro | $ 1.00 | $ 1.00 | nan | nan |
| openrouter/google/gemini-pro-1.5 | $2.5 | $7.5 | 1,000,000 | 8192 |
| openrouter/mistralai/mixtral-8x22b-instruct | $0.65 | $0.65 | nan | nan |
| openrouter/cohere/command-r-plus | $ 3.00 | $15.00 | nan | nan |
| openrouter/databricks/dbrx-instruct | $0.6 | $0.6 | nan | nan |
| openrouter/anthropic/claude-3-haiku | $0.25 | $1.25 | nan | nan |
| openrouter/anthropic/claude-3-haiku-20240307 | $0.25 | $1.25 | 200,000 | 4096 |
| openrouter/anthropic/claude-3.5-sonnet | $ 3.00 | $15.00 | 200,000 | 4096 |
| openrouter/anthropic/claude-3-sonnet | $ 3.00 | $15.00 | nan | nan |
| openrouter/mistralai/mistral-large | $ 8.00 | $24.00 | nan | nan |
| openrouter/cognitivecomputations/dolphin-mixtral-8x7b | $0.5 | $0.5 | nan | nan |
| openrouter/google/gemini-pro-vision | $0.125 | $0.375 | nan | nan |
| openrouter/fireworks/firellava-13b | $0.2 | $0.2 | nan | nan |
| openrouter/meta-llama/llama-3-8b-instruct:free | $ 0.00 | $ 0.00 | nan | nan |
| openrouter/meta-llama/llama-3-8b-instruct:extended | $0.225 | $2.25 | nan | nan |
| openrouter/meta-llama/llama-3-70b-instruct:nitro | $0.9 | $0.9 | nan | nan |
| openrouter/meta-llama/llama-3-70b-instruct | $0.59 | $0.79 | nan | nan |
| openrouter/openai/gpt-4o | $ 5.00 | $15.00 | 128,000 | 4096 |
| openrouter/openai/gpt-4o-2024-05-13 | $ 5.00 | $15.00 | 128,000 | 4096 |
| openrouter/openai/gpt-4-vision-preview | $10.00 | $30.00 | nan | nan |
| openrouter/openai/gpt-3.5-turbo | $1.5 | $ 2.00 | nan | nan |
| openrouter/openai/gpt-3.5-turbo-16k | $ 3.00 | $ 4.00 | nan | nan |
| openrouter/openai/gpt-4 | $30.00 | $60.00 | nan | nan |
| openrouter/anthropic/claude-instant-v1 | $1.63 | $5.51 | nan | 8191 |
| openrouter/anthropic/claude-2 | $11.02 | $32.68 | nan | 8191 |
| openrouter/anthropic/claude-3-opus | $15.00 | $75.00 | 200,000 | 4096 |
| openrouter/google/palm-2-chat-bison | $0.5 | $0.5 | nan | nan |
| openrouter/google/palm-2-codechat-bison | $0.5 | $0.5 | nan | nan |
| openrouter/meta-llama/llama-2-13b-chat | $0.2 | $0.2 | nan | nan |
| openrouter/meta-llama/llama-2-70b-chat | $1.5 | $1.5 | nan | nan |
| openrouter/meta-llama/codellama-34b-instruct | $0.5 | $0.5 | nan | nan |
| openrouter/nousresearch/nous-hermes-llama2-13b | $0.2 | $0.2 | nan | nan |
| openrouter/mancer/weaver | $5.625 | $5.625 | nan | nan |
| openrouter/gryphe/mythomax-l2-13b | $1.875 | $1.875 | nan | nan |
| openrouter/jondurbin/airoboros-l2-70b-2.1 | $13.875 | $13.875 | nan | nan |
| openrouter/undi95/remm-slerp-l2-13b | $1.875 | $1.875 | nan | nan |
| openrouter/pygmalionai/mythalion-13b | $1.875 | $1.875 | nan | nan |
| openrouter/mistralai/mistral-7b-instruct | $0.13 | $0.13 | nan | nan |
| openrouter/mistralai/mistral-7b-instruct:free | $ 0.00 | $ 0.00 | nan | nan |
| j2-ultra | $15.00 | $15.00 | 8,192 | 8192 |
| j2-mid | $10.00 | $10.00 | 8,192 | 8192 |
| j2-light | $ 3.00 | $ 3.00 | 8,192 | 8192 |
| dolphin | $0.5 | $0.5 | 16,384 | 16384 |
| chatdolphin | $0.5 | $0.5 | 16,384 | 16384 |
| luminous-base | $30.00 | $33.00 | nan | nan |
| luminous-base-control | $37.5 | $41.25 | nan | nan |
| luminous-extended | $45.00 | $49.5 | nan | nan |
| luminous-extended-control | $56.25 | $61.875 | nan | nan |
| luminous-supreme | $175.00 | $192.5 | nan | nan |
| luminous-supreme-control | $218.75 | $240.625 | nan | nan |
| ai21.j2-mid-v1 | $12.5 | $12.5 | 8,191 | 8191 |
| ai21.j2-ultra-v1 | $18.8 | $18.8 | 8,191 | 8191 |
| amazon.titan-text-lite-v1 | $0.3 | $0.4 | 42,000 | 4000 |
| amazon.titan-text-express-v1 | $1.3 | $1.7 | 42,000 | 8000 |
| amazon.titan-embed-text-v1 | $0.1 | $ 0.00 | 8,192 | nan |
| amazon.titan-embed-text-v2:0 | $0.2 | $ 0.00 | 8,192 | nan |
| mistral.mistral-7b-instruct-v0:2 | $0.15 | $0.2 | 32,000 | 8191 |
| mistral.mixtral-8x7b-instruct-v0:1 | $0.45 | $0.7 | 32,000 | 8191 |
| mistral.mistral-large-2402-v1:0 | $ 8.00 | $24.00 | 32,000 | 8191 |
| bedrock/us-west-2/mistral.mixtral-8x7b-instruct-v0:1 | $0.45 | $0.7 | 32,000 | 8191 |
| bedrock/us-east-1/mistral.mixtral-8x7b-instruct-v0:1 | $0.45 | $0.7 | 32,000 | 8191 |
| bedrock/eu-west-3/mistral.mixtral-8x7b-instruct-v0:1 | $0.59 | $0.91 | 32,000 | 8191 |
| bedrock/us-west-2/mistral.mistral-7b-instruct-v0:2 | $0.15 | $0.2 | 32,000 | 8191 |
| bedrock/us-east-1/mistral.mistral-7b-instruct-v0:2 | $0.15 | $0.2 | 32,000 | 8191 |
| bedrock/eu-west-3/mistral.mistral-7b-instruct-v0:2 | $0.2 | $0.26 | 32,000 | 8191 |
| bedrock/us-east-1/mistral.mistral-large-2402-v1:0 | $ 8.00 | $24.00 | 32,000 | 8191 |
| bedrock/us-west-2/mistral.mistral-large-2402-v1:0 | $ 8.00 | $24.00 | 32,000 | 8191 |
| bedrock/eu-west-3/mistral.mistral-large-2402-v1:0 | $10.4 | $31.2 | 32,000 | 8191 |
| anthropic.claude-3-sonnet-20240229-v1:0 | $ 3.00 | $15.00 | 200,000 | 4096 |
| anthropic.claude-3-5-sonnet-20240620-v1:0 | $ 3.00 | $15.00 | 200,000 | 4096 |
| anthropic.claude-3-haiku-20240307-v1:0 | $0.25 | $1.25 | 200,000 | 4096 |
| anthropic.claude-3-opus-20240229-v1:0 | $15.00 | $75.00 | 200,000 | 4096 |
| anthropic.claude-v1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/us-east-1/anthropic.claude-v1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/us-west-2/anthropic.claude-v1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/ap-northeast-1/anthropic.claude-v1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/ap-northeast-1/1-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/ap-northeast-1/6-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/anthropic.claude-v1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/eu-central-1/1-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/6-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/1-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/6-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/1-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/6-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| anthropic.claude-v2 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/us-east-1/anthropic.claude-v2 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/us-west-2/anthropic.claude-v2 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/ap-northeast-1/anthropic.claude-v2 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/ap-northeast-1/1-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/ap-northeast-1/6-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/anthropic.claude-v2 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/eu-central-1/1-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/6-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/1-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/6-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/1-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/6-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| anthropic.claude-v2:1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/us-east-1/anthropic.claude-v2:1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/us-west-2/anthropic.claude-v2:1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/ap-northeast-1/anthropic.claude-v2:1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/ap-northeast-1/1-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/ap-northeast-1/6-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/anthropic.claude-v2:1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/eu-central-1/1-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/6-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/1-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/6-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/1-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/6-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| anthropic.claude-instant-v1 | $1.63 | $5.51 | 100,000 | 8191 |
| bedrock/us-east-1/anthropic.claude-instant-v1 | $0.8 | $2.4 | 100,000 | 8191 |
| bedrock/us-east-1/1-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/6-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/1-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/6-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/anthropic.claude-instant-v1 | $0.8 | $2.4 | 100,000 | 8191 |
| bedrock/ap-northeast-1/anthropic.claude-instant-v1 | $2.23 | $7.55 | 100,000 | 8191 |
| bedrock/ap-northeast-1/1-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/ap-northeast-1/6-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/anthropic.claude-instant-v1 | $2.48 | $8.38 | 100,000 | 8191 |
| bedrock/eu-central-1/1-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/6-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| cohere.command-text-v14 | $1.5 | $ 2.00 | 4,096 | 4096 |
| bedrock/*/1-month-commitment/cohere.command-text-v14 | -- | -- | 4,096 | 4096 |
| bedrock/*/6-month-commitment/cohere.command-text-v14 | -- | -- | 4,096 | 4096 |
| cohere.command-light-text-v14 | $0.3 | $0.6 | 4,096 | 4096 |
| bedrock/*/1-month-commitment/cohere.command-light-text-v14 | -- | -- | 4,096 | 4096 |
| bedrock/*/6-month-commitment/cohere.command-light-text-v14 | -- | -- | 4,096 | 4096 |
| cohere.command-r-plus-v1:0 | $ 3.00 | $15.00 | 128,000 | 4096 |
| cohere.command-r-v1:0 | $0.5 | $1.5 | 128,000 | 4096 |
| cohere.embed-english-v3 | $0.1 | $ 0.00 | 512 | nan |
| cohere.embed-multilingual-v3 | $0.1 | $ 0.00 | 512 | nan |
| meta.llama2-13b-chat-v1 | $0.75 | $ 1.00 | 4,096 | 4096 |
| meta.llama2-70b-chat-v1 | $1.95 | $2.56 | 4,096 | 4096 |
| meta.llama3-8b-instruct-v1:0 | $0.4 | $0.6 | 8,192 | 8192 |
| meta.llama3-70b-instruct-v1:0 | $2.65 | $3.5 | 8,192 | 8192 |
| 512-x-512/50-steps/stability.stable-diffusion-xl-v0 | -- | -- | 77 | nan |
| 512-x-512/max-steps/stability.stable-diffusion-xl-v0 | -- | -- | 77 | nan |
| max-x-max/50-steps/stability.stable-diffusion-xl-v0 | -- | -- | 77 | nan |
| max-x-max/max-steps/stability.stable-diffusion-xl-v0 | -- | -- | 77 | nan |
| 1024-x-1024/50-steps/stability.stable-diffusion-xl-v1 | -- | -- | 77 | nan |
| 1024-x-1024/max-steps/stability.stable-diffusion-xl-v1 | -- | -- | 77 | nan |
| sagemaker/meta-textgeneration-llama-2-7b | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| sagemaker/meta-textgeneration-llama-2-7b-f | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| sagemaker/meta-textgeneration-llama-2-13b | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| sagemaker/meta-textgeneration-llama-2-13b-f | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| sagemaker/meta-textgeneration-llama-2-70b | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| sagemaker/meta-textgeneration-llama-2-70b-b-f | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| together-ai-up-to-4b | $0.1 | $0.1 | nan | nan |
| together-ai-4.1b-8b | $0.2 | $0.2 | nan | nan |
| together-ai-8.1b-21b | $0.3 | $0.3 | nan | nan |
| together-ai-21.1b-41b | $0.8 | $0.8 | nan | nan |
| together-ai-41.1b-80b | $0.9 | $0.9 | nan | nan |
| together-ai-81.1b-110b | $1.8 | $1.8 | nan | nan |
| together_ai/mistralai/Mixtral-8x7B-Instruct-v0.1 | $0.6 | $0.6 | nan | nan |
| together_ai/mistralai/Mistral-7B-Instruct-v0.1 | -- | -- | nan | nan |
| together_ai/togethercomputer/CodeLlama-34b-Instruct | -- | -- | nan | nan |
| ollama/codegemma | $ 0.00 | $ 0.00 | 8,192 | 8192 |
| ollama/llama2 | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| ollama/llama2:13b | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| ollama/llama2:70b | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| ollama/llama2-uncensored | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| ollama/llama3 | $ 0.00 | $ 0.00 | 8,192 | 8192 |
| ollama/llama3:70b | $ 0.00 | $ 0.00 | 8,192 | 8192 |
| ollama/mistral | $ 0.00 | $ 0.00 | 8,192 | 8192 |
| ollama/mistral-7B-Instruct-v0.1 | $ 0.00 | $ 0.00 | 8,192 | 8192 |
| ollama/mistral-7B-Instruct-v0.2 | $ 0.00 | $ 0.00 | 32,768 | 32768 |
| ollama/mixtral-8x7B-Instruct-v0.1 | $ 0.00 | $ 0.00 | 32,768 | 32768 |
| ollama/mixtral-8x22B-Instruct-v0.1 | $ 0.00 | $ 0.00 | 65,536 | 65536 |
| ollama/codellama | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| ollama/orca-mini | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| ollama/vicuna | $ 0.00 | $ 0.00 | 2,048 | 2048 |
| deepinfra/lizpreciatior/lzlv_70b_fp16_hf | $0.7 | $0.9 | 4,096 | 4096 |
| deepinfra/Gryphe/MythoMax-L2-13b | $0.22 | $0.22 | 4,096 | 4096 |
| deepinfra/mistralai/Mistral-7B-Instruct-v0.1 | $0.13 | $0.13 | 32,768 | 8191 |
| deepinfra/meta-llama/Llama-2-70b-chat-hf | $0.7 | $0.9 | 4,096 | 4096 |
| deepinfra/cognitivecomputations/dolphin-2.6-mixtral-8x7b | $0.27 | $0.27 | 32,768 | 8191 |
| deepinfra/codellama/CodeLlama-34b-Instruct-hf | $0.6 | $0.6 | 4,096 | 4096 |
| deepinfra/deepinfra/mixtral | $0.27 | $0.27 | 32,000 | 4096 |
| deepinfra/Phind/Phind-CodeLlama-34B-v2 | $0.6 | $0.6 | 16,384 | 4096 |
| deepinfra/mistralai/Mixtral-8x7B-Instruct-v0.1 | $0.27 | $0.27 | 32,768 | 8191 |
| deepinfra/deepinfra/airoboros-70b | $0.7 | $0.9 | 4,096 | 4096 |
| deepinfra/01-ai/Yi-34B-Chat | $0.6 | $0.6 | 4,096 | 4096 |
| deepinfra/01-ai/Yi-6B-200K | $0.13 | $0.13 | 200,000 | 4096 |
| deepinfra/jondurbin/airoboros-l2-70b-gpt4-1.4.1 | $0.7 | $0.9 | 4,096 | 4096 |
| deepinfra/meta-llama/Llama-2-13b-chat-hf | $0.22 | $0.22 | 4,096 | 4096 |
| deepinfra/amazon/MistralLite | $0.2 | $0.2 | 32,768 | 8191 |
| deepinfra/meta-llama/Llama-2-7b-chat-hf | $0.13 | $0.13 | 4,096 | 4096 |
| deepinfra/meta-llama/Meta-Llama-3-8B-Instruct | $0.08 | $0.08 | 8,191 | 4096 |
| deepinfra/meta-llama/Meta-Llama-3-70B-Instruct | $0.59 | $0.79 | 8,191 | 4096 |
| deepinfra/01-ai/Yi-34B-200K | $0.6 | $0.6 | 200,000 | 4096 |
| deepinfra/openchat/openchat_3.5 | $0.13 | $0.13 | 4,096 | 4096 |
| perplexity/codellama-34b-instruct | $0.35 | $1.4 | 16,384 | 16384 |
| perplexity/codellama-70b-instruct | $0.7 | $2.8 | 16,384 | 16384 |
| perplexity/pplx-7b-chat | $0.07 | $0.28 | 8,192 | 8192 |
| perplexity/pplx-70b-chat | $0.7 | $2.8 | 4,096 | 4096 |
| perplexity/pplx-7b-online | $ 0.00 | $0.28 | 4,096 | 4096 |
| perplexity/pplx-70b-online | $ 0.00 | $2.8 | 4,096 | 4096 |
| perplexity/llama-2-70b-chat | $0.7 | $2.8 | 4,096 | 4096 |
| perplexity/mistral-7b-instruct | $0.07 | $0.28 | 4,096 | 4096 |
| perplexity/mixtral-8x7b-instruct | $0.07 | $0.28 | 4,096 | 4096 |
| perplexity/sonar-small-chat | $0.07 | $0.28 | 16,384 | 16384 |
| perplexity/sonar-small-online | $ 0.00 | $0.28 | 12,000 | 12000 |
| perplexity/sonar-medium-chat | $0.6 | $1.8 | 16,384 | 16384 |
| perplexity/sonar-medium-online | $ 0.00 | $1.8 | 12,000 | 12000 |
| fireworks_ai/firefunction-v2 | $0.9 | $0.9 | 8,192 | 8192 |
| fireworks_ai/mixtral-8x22b-instruct-hf | $1.2 | $1.2 | 65,536 | 65536 |
| fireworks_ai/qwen2-72b-instruct | $0.9 | $0.9 | 32,768 | 32768 |
| fireworks_ai/yi-large | $ 3.00 | $ 3.00 | 32,768 | 32768 |
| fireworks_ai/deepseek-coder-v2-instruct | $1.2 | $1.2 | 65,536 | 8192 |
| anyscale/mistralai/Mistral-7B-Instruct-v0.1 | $0.15 | $0.15 | 16,384 | 16384 |
| anyscale/mistralai/Mixtral-8x7B-Instruct-v0.1 | $0.15 | $0.15 | 16,384 | 16384 |
| anyscale/mistralai/Mixtral-8x22B-Instruct-v0.1 | $0.9 | $0.9 | 65,536 | 65536 |
| anyscale/HuggingFaceH4/zephyr-7b-beta | $0.15 | $0.15 | 16,384 | 16384 |
| anyscale/google/gemma-7b-it | $0.15 | $0.15 | 8,192 | 8192 |
| anyscale/meta-llama/Llama-2-7b-chat-hf | $0.15 | $0.15 | 4,096 | 4096 |
| anyscale/meta-llama/Llama-2-13b-chat-hf | $0.25 | $0.25 | 4,096 | 4096 |
| anyscale/meta-llama/Llama-2-70b-chat-hf | $ 1.00 | $ 1.00 | 4,096 | 4096 |
| anyscale/codellama/CodeLlama-34b-Instruct-hf | $ 1.00 | $ 1.00 | 4,096 | 4096 |
| anyscale/codellama/CodeLlama-70b-Instruct-hf | $ 1.00 | $ 1.00 | 4,096 | 4096 |
| anyscale/meta-llama/Meta-Llama-3-8B-Instruct | $0.15 | $0.15 | 8,192 | 8192 |
| anyscale/meta-llama/Meta-Llama-3-70B-Instruct | $ 1.00 | $ 1.00 | 8,192 | 8192 |
| cloudflare/@cf/meta/llama-2-7b-chat-fp16 | $1.923 | $1.923 | 3,072 | 3072 |
| cloudflare/@cf/meta/llama-2-7b-chat-int8 | $1.923 | $1.923 | 2,048 | 2048 |
| cloudflare/@cf/mistral/mistral-7b-instruct-v0.1 | $1.923 | $1.923 | 8,192 | 8192 |
| cloudflare/@hf/thebloke/codellama-7b-instruct-awq | $1.923 | $1.923 | 4,096 | 4096 |
| voyage/voyage-01 | $0.1 | $ 0.00 | 4,096 | nan |
| voyage/voyage-lite-01 | $0.1 | $ 0.00 | 4,096 | nan |
| voyage/voyage-large-2 | $0.12 | $ 0.00 | 16,000 | nan |
| voyage/voyage-law-2 | $0.12 | $ 0.00 | 16,000 | nan |
| voyage/voyage-code-2 | $0.12 | $ 0.00 | 16,000 | nan |
| voyage/voyage-2 | $0.1 | $ 0.00 | 4,000 | nan |
| voyage/voyage-lite-02-instruct | $0.1 | $ 0.00 | 4,000 | nan |
| databricks/databricks-dbrx-instruct | $0.75 | $2.25 | 32,768 | 32768 |
| databricks/databricks-meta-llama-3-70b-instruct | $ 1.00 | $ 3.00 | 8,192 | 8192 |
| databricks/databricks-llama-2-70b-chat | $0.5 | $1.5 | 4,096 | 4096 |
| databricks/databricks-mixtral-8x7b-instruct | $0.5 | $ 1.00 | 4,096 | 4096 |
| databricks/databricks-mpt-30b-instruct | $ 1.00 | $ 1.00 | 8,192 | 8192 |
| databricks/databricks-mpt-7b-instruct | $0.5 | $0.5 | 8,192 | 8192 |
| databricks/databricks-bge-large-en | $0.1 | $ 0.00 | 512 | nan |
| :-------------------------------------------------------------------------------------------- | :---------------------------------- | :-------------------------------------- | :-------------------- | --------------------: |
| gpt-4 | $30.00 | $60.00 | 8,192 | 4096 |
| gpt-4o | $ 5.00 | $15.00 | 128,000 | 4096 |
| gpt-4o-2024-05-13 | $ 5.00 | $15.00 | 128,000 | 4096 |
| gpt-4-turbo-preview | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-4-0314 | $30.00 | $60.00 | 8,192 | 4096 |
| gpt-4-0613 | $30.00 | $60.00 | 8,192 | 4096 |
| gpt-4-32k | $60.00 | $120.00 | 32,768 | 4096 |
| gpt-4-32k-0314 | $60.00 | $120.00 | 32,768 | 4096 |
| gpt-4-32k-0613 | $60.00 | $120.00 | 32,768 | 4096 |
| gpt-4-turbo | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-4-turbo-2024-04-09 | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-4-1106-preview | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-4-0125-preview | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-4-vision-preview | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-4-1106-vision-preview | $10.00 | $30.00 | 128,000 | 4096 |
| gpt-3.5-turbo | $1.5 | $ 2.00 | 16,385 | 4096 |
| gpt-3.5-turbo-0301 | $1.5 | $ 2.00 | 4,097 | 4096 |
| gpt-3.5-turbo-0613 | $1.5 | $ 2.00 | 4,097 | 4096 |
| gpt-3.5-turbo-1106 | $ 1.00 | $ 2.00 | 16,385 | 4096 |
| gpt-3.5-turbo-0125 | $0.5 | $1.5 | 16,385 | 4096 |
| gpt-3.5-turbo-16k | $ 3.00 | $ 4.00 | 16,385 | 4096 |
| gpt-3.5-turbo-16k-0613 | $ 3.00 | $ 4.00 | 16,385 | 4096 |
| ft:gpt-3.5-turbo | $ 3.00 | $ 6.00 | 4,097 | 4096 |
| ft:gpt-4-0613 | $30.00 | $60.00 | 8,192 | 4096 |
| ft:gpt-4o-2024-05-13 | $ 5.00 | $15.00 | 128,000 | 4096 |
| ft:davinci-002 | $ 2.00 | $ 2.00 | 16,384 | 4096 |
| ft:babbage-002 | $0.4 | $0.4 | 16,384 | 4096 |
| text-embedding-3-large | $0.13 | $ 0.00 | 8,191 | nan |
| text-embedding-3-small | $0.02 | $ 0.00 | 8,191 | nan |
| text-embedding-ada-002 | $0.1 | $ 0.00 | 8,191 | nan |
| text-embedding-ada-002-v2 | $0.1 | $ 0.00 | 8,191 | nan |
| text-moderation-stable | $ 0.00 | $ 0.00 | 32,768 | 0 |
| text-moderation-007 | $ 0.00 | $ 0.00 | 32,768 | 0 |
| text-moderation-latest | $ 0.00 | $ 0.00 | 32,768 | 0 |
| 256-x-256/dall-e-2 | -- | -- | nan | nan |
| 512-x-512/dall-e-2 | -- | -- | nan | nan |
| 1024-x-1024/dall-e-2 | -- | -- | nan | nan |
| hd/1024-x-1792/dall-e-3 | -- | -- | nan | nan |
| hd/1792-x-1024/dall-e-3 | -- | -- | nan | nan |
| hd/1024-x-1024/dall-e-3 | -- | -- | nan | nan |
| standard/1024-x-1792/dall-e-3 | -- | -- | nan | nan |
| standard/1792-x-1024/dall-e-3 | -- | -- | nan | nan |
| standard/1024-x-1024/dall-e-3 | -- | -- | nan | nan |
| whisper-1 | -- | -- | nan | nan |
| azure/whisper-1 | -- | -- | nan | nan |
| azure/gpt-4o | $ 5.00 | $15.00 | 128,000 | 4096 |
| azure/gpt-4-turbo-2024-04-09 | $10.00 | $30.00 | 128,000 | 4096 |
| azure/gpt-4-0125-preview | $10.00 | $30.00 | 128,000 | 4096 |
| azure/gpt-4-1106-preview | $10.00 | $30.00 | 128,000 | 4096 |
| azure/gpt-4-0613 | $30.00 | $60.00 | 8,192 | 4096 |
| azure/gpt-4-32k-0613 | $60.00 | $120.00 | 32,768 | 4096 |
| azure/gpt-4-32k | $60.00 | $120.00 | 32,768 | 4096 |
| azure/gpt-4 | $30.00 | $60.00 | 8,192 | 4096 |
| azure/gpt-4-turbo | $10.00 | $30.00 | 128,000 | 4096 |
| azure/gpt-4-turbo-vision-preview | $10.00 | $30.00 | 128,000 | 4096 |
| azure/gpt-35-turbo-16k-0613 | $ 3.00 | $ 4.00 | 16,385 | 4096 |
| azure/gpt-35-turbo-1106 | $ 1.00 | $ 2.00 | 16,384 | 4096 |
| azure/gpt-35-turbo-0125 | $0.5 | $1.5 | 16,384 | 4096 |
| azure/gpt-35-turbo-16k | $ 3.00 | $ 4.00 | 16,385 | 4096 |
| azure/gpt-35-turbo | $0.5 | $1.5 | 4,097 | 4096 |
| azure/gpt-3.5-turbo-instruct-0914 | $1.5 | $ 2.00 | 4,097 | nan |
| azure/gpt-35-turbo-instruct | $1.5 | $ 2.00 | 4,097 | nan |
| azure/mistral-large-latest | $ 8.00 | $24.00 | 32,000 | nan |
| azure/mistral-large-2402 | $ 8.00 | $24.00 | 32,000 | nan |
| azure/command-r-plus | $ 3.00 | $15.00 | 128,000 | 4096 |
| azure/ada | $0.1 | $ 0.00 | 8,191 | nan |
| azure/text-embedding-ada-002 | $0.1 | $ 0.00 | 8,191 | nan |
| azure/text-embedding-3-large | $0.13 | $ 0.00 | 8,191 | nan |
| azure/text-embedding-3-small | $0.02 | $ 0.00 | 8,191 | nan |
| azure/standard/1024-x-1024/dall-e-3 | -- | $ 0.00 | nan | nan |
| azure/hd/1024-x-1024/dall-e-3 | -- | $ 0.00 | nan | nan |
| azure/standard/1024-x-1792/dall-e-3 | -- | $ 0.00 | nan | nan |
| azure/standard/1792-x-1024/dall-e-3 | -- | $ 0.00 | nan | nan |
| azure/hd/1024-x-1792/dall-e-3 | -- | $ 0.00 | nan | nan |
| azure/hd/1792-x-1024/dall-e-3 | -- | $ 0.00 | nan | nan |
| azure/standard/1024-x-1024/dall-e-2 | -- | $ 0.00 | nan | nan |
| babbage-002 | $0.4 | $0.4 | 16,384 | 4096 |
| davinci-002 | $ 2.00 | $ 2.00 | 16,384 | 4096 |
| gpt-3.5-turbo-instruct | $1.5 | $ 2.00 | 8,192 | 4096 |
| gpt-3.5-turbo-instruct-0914 | $1.5 | $ 2.00 | 8,192 | 4097 |
| claude-instant-1 | $1.63 | $5.51 | 100,000 | 8191 |
| mistral/mistral-tiny | $0.25 | $0.25 | 32,000 | 8191 |
| mistral/mistral-small | $ 1.00 | $ 3.00 | 32,000 | 8191 |
| mistral/mistral-small-latest | $ 1.00 | $ 3.00 | 32,000 | 8191 |
| mistral/mistral-medium | $2.7 | $8.1 | 32,000 | 8191 |
| mistral/mistral-medium-latest | $2.7 | $8.1 | 32,000 | 8191 |
| mistral/mistral-medium-2312 | $2.7 | $8.1 | 32,000 | 8191 |
| mistral/mistral-large-latest | $ 4.00 | $12.00 | 32,000 | 8191 |
| mistral/mistral-large-2402 | $ 4.00 | $12.00 | 32,000 | 8191 |
| mistral/open-mistral-7b | $0.25 | $0.25 | 32,000 | 8191 |
| mistral/open-mixtral-8x7b | $0.7 | $0.7 | 32,000 | 8191 |
| mistral/open-mixtral-8x22b | $ 2.00 | $ 6.00 | 64,000 | 8191 |
| mistral/codestral-latest | $ 1.00 | $ 3.00 | 32,000 | 8191 |
| mistral/codestral-2405 | $ 1.00 | $ 3.00 | 32,000 | 8191 |
| mistral/mistral-embed | $0.1 | -- | 8,192 | nan |
| deepseek-chat | $0.14 | $0.28 | 32,000 | 4096 |
| codestral/codestral-latest | $ 0.00 | $ 0.00 | 32,000 | 8191 |
| codestral/codestral-2405 | $ 0.00 | $ 0.00 | 32,000 | 8191 |
| text-completion-codestral/codestral-latest | $ 0.00 | $ 0.00 | 32,000 | 8191 |
| text-completion-codestral/codestral-2405 | $ 0.00 | $ 0.00 | 32,000 | 8191 |
| deepseek-coder | $0.14 | $0.28 | 32,000 | 4096 |
| groq/llama2-70b-4096 | $0.7 | $0.8 | 4,096 | 4096 |
| groq/llama3-8b-8192 | $0.05 | $0.08 | 8,192 | 8192 |
| groq/llama3-70b-8192 | $0.59 | $0.79 | 8,192 | 8192 |
| groq/mixtral-8x7b-32768 | $0.24 | $0.24 | 32,768 | 32768 |
| groq/gemma-7b-it | $0.07 | $0.07 | 8,192 | 8192 |
| friendliai/mixtral-8x7b-instruct-v0-1 | $0.4 | $0.4 | 32,768 | 32768 |
| friendliai/meta-llama-3-8b-instruct | $0.1 | $0.1 | 8,192 | 8192 |
| friendliai/meta-llama-3-70b-instruct | $0.8 | $0.8 | 8,192 | 8192 |
| claude-instant-1.2 | $0.163 | $0.551 | 100,000 | 8191 |
| claude-2 | $ 8.00 | $24.00 | 100,000 | 8191 |
| claude-2.1 | $ 8.00 | $24.00 | 200,000 | 8191 |
| claude-3-haiku-20240307 | $0.25 | $1.25 | 200,000 | 4096 |
| claude-3-opus-20240229 | $15.00 | $75.00 | 200,000 | 4096 |
| claude-3-sonnet-20240229 | $ 3.00 | $15.00 | 200,000 | 4096 |
| claude-3-5-sonnet-20240620 | $ 3.00 | $15.00 | 200,000 | 4096 |
| text-bison | -- | -- | 8,192 | 2048 |
| text-bison@001 | -- | -- | 8,192 | 1024 |
| text-bison@002 | -- | -- | 8,192 | 1024 |
| text-bison32k | $0.125 | $0.125 | 8,192 | 1024 |
| text-bison32k@002 | $0.125 | $0.125 | 8,192 | 1024 |
| text-unicorn | $10.00 | $28.00 | 8,192 | 1024 |
| text-unicorn@001 | $10.00 | $28.00 | 8,192 | 1024 |
| chat-bison | $0.125 | $0.125 | 8,192 | 4096 |
| chat-bison@001 | $0.125 | $0.125 | 8,192 | 4096 |
| chat-bison@002 | $0.125 | $0.125 | 8,192 | 4096 |
| chat-bison-32k | $0.125 | $0.125 | 32,000 | 8192 |
| chat-bison-32k@002 | $0.125 | $0.125 | 32,000 | 8192 |
| code-bison | $0.125 | $0.125 | 6,144 | 1024 |
| code-bison@001 | $0.125 | $0.125 | 6,144 | 1024 |
| code-bison@002 | $0.125 | $0.125 | 6,144 | 1024 |
| code-bison32k | $0.125 | $0.125 | 6,144 | 1024 |
| code-bison-32k@002 | $0.125 | $0.125 | 6,144 | 1024 |
| code-gecko@001 | $0.125 | $0.125 | 2,048 | 64 |
| code-gecko@002 | $0.125 | $0.125 | 2,048 | 64 |
| code-gecko | $0.125 | $0.125 | 2,048 | 64 |
| code-gecko-latest | $0.125 | $0.125 | 2,048 | 64 |
| codechat-bison@latest | $0.125 | $0.125 | 6,144 | 1024 |
| codechat-bison | $0.125 | $0.125 | 6,144 | 1024 |
| codechat-bison@001 | $0.125 | $0.125 | 6,144 | 1024 |
| codechat-bison@002 | $0.125 | $0.125 | 6,144 | 1024 |
| codechat-bison-32k | $0.125 | $0.125 | 32,000 | 8192 |
| codechat-bison-32k@002 | $0.125 | $0.125 | 32,000 | 8192 |
| gemini-pro | $0.5 | $1.5 | 32,760 | 8192 |
| gemini-1.0-pro | $0.5 | $1.5 | 32,760 | 8192 |
| gemini-1.0-pro-001 | $0.5 | $1.5 | 32,760 | 8192 |
| gemini-1.0-ultra | $0.5 | $1.5 | 8,192 | 2048 |
| gemini-1.0-ultra-001 | $0.5 | $1.5 | 8,192 | 2048 |
| gemini-1.0-pro-002 | $0.5 | $1.5 | 32,760 | 8192 |
| gemini-1.5-pro | $ 5.00 | $15.00 | 2,097,152 | 8192 |
| gemini-1.5-pro-001 | $ 5.00 | $15.00 | 1,000,000 | 8192 |
| gemini-1.5-pro-preview-0514 | $ 5.00 | $15.00 | 1,000,000 | 8192 |
| gemini-1.5-pro-preview-0215 | $ 5.00 | $15.00 | 1,000,000 | 8192 |
| gemini-1.5-pro-preview-0409 | $ 5.00 | $15.00 | 1,000,000 | 8192 |
| gemini-1.5-flash | $0.5 | $1.5 | 1,000,000 | 8192 |
| gemini-1.5-flash-001 | $0.5 | $1.5 | 1,000,000 | 8192 |
| gemini-1.5-flash-preview-0514 | $0.5 | $1.5 | 1,000,000 | 8192 |
| gemini-experimental | $ 0.00 | $ 0.00 | 1,000,000 | 8192 |
| gemini-pro-vision | $0.25 | $0.5 | 16,384 | 2048 |
| gemini-1.0-pro-vision | $0.25 | $0.5 | 16,384 | 2048 |
| gemini-1.0-pro-vision-001 | $0.25 | $0.5 | 16,384 | 2048 |
| vertex_ai/claude-3-sonnet@20240229 | $ 3.00 | $15.00 | 200,000 | 4096 |
| vertex_ai/claude-3-5-sonnet@20240620 | $ 3.00 | $15.00 | 200,000 | 4096 |
| vertex_ai/claude-3-haiku@20240307 | $0.25 | $1.25 | 200,000 | 4096 |
| vertex_ai/claude-3-opus@20240229 | $15.00 | $75.00 | 200,000 | 4096 |
| vertex_ai/imagegeneration@006 | -- | -- | nan | nan |
| text-embedding-004 | $0.00625 | $ 0.00 | 3,072 | nan |
| text-multilingual-embedding-002 | $0.00625 | $ 0.00 | 2,048 | nan |
| textembedding-gecko | $0.00625 | $ 0.00 | 3,072 | nan |
| textembedding-gecko-multilingual | $0.00625 | $ 0.00 | 3,072 | nan |
| textembedding-gecko-multilingual@001 | $0.00625 | $ 0.00 | 3,072 | nan |
| textembedding-gecko@001 | $0.00625 | $ 0.00 | 3,072 | nan |
| textembedding-gecko@003 | $0.00625 | $ 0.00 | 3,072 | nan |
| text-embedding-preview-0409 | $0.00625 | $ 0.00 | 3,072 | nan |
| text-multilingual-embedding-preview-0409 | $0.00625 | $ 0.00 | 3,072 | nan |
| palm/chat-bison | $0.125 | $0.125 | 8,192 | 4096 |
| palm/chat-bison-001 | $0.125 | $0.125 | 8,192 | 4096 |
| palm/text-bison | $0.125 | $0.125 | 8,192 | 1024 |
| palm/text-bison-001 | $0.125 | $0.125 | 8,192 | 1024 |
| palm/text-bison-safety-off | $0.125 | $0.125 | 8,192 | 1024 |
| palm/text-bison-safety-recitation-off | $0.125 | $0.125 | 8,192 | 1024 |
| gemini/gemini-1.5-flash | $0.35 | $1.05 | 1,000,000 | 8192 |
| gemini/gemini-1.5-flash-latest | $0.35 | $1.05 | 1,000,000 | 8192 |
| gemini/gemini-pro | $0.35 | $1.05 | 32,760 | 8192 |
| gemini/gemini-1.5-pro | $0.35 | $1.05 | 2,097,152 | 8192 |
| gemini/gemini-1.5-pro-latest | $0.35 | $1.05 | 1,048,576 | 8192 |
| gemini/gemini-pro-vision | $0.35 | $1.05 | 30,720 | 2048 |
| command-r | $0.5 | $1.5 | 128,000 | 4096 |
| command-light | $15.00 | $15.00 | 4,096 | 4096 |
| command-r-plus | $ 3.00 | $15.00 | 128,000 | 4096 |
| command-nightly | $15.00 | $15.00 | 4,096 | 4096 |
| command | $15.00 | $15.00 | 4,096 | 4096 |
| command-medium-beta | $15.00 | $15.00 | 4,096 | 4096 |
| command-xlarge-beta | $15.00 | $15.00 | 4,096 | 4096 |
| replicate/meta/llama-2-13b | $0.1 | $0.5 | 4,096 | 4096 |
| replicate/meta/llama-2-13b-chat | $0.1 | $0.5 | 4,096 | 4096 |
| replicate/meta/llama-2-70b | $0.65 | $2.75 | 4,096 | 4096 |
| replicate/meta/llama-2-70b-chat | $0.65 | $2.75 | 4,096 | 4096 |
| replicate/meta/llama-2-7b | $0.05 | $0.25 | 4,096 | 4096 |
| replicate/meta/llama-2-7b-chat | $0.05 | $0.25 | 4,096 | 4096 |
| replicate/meta/llama-3-70b | $0.65 | $2.75 | 8,192 | 8192 |
| replicate/meta/llama-3-70b-instruct | $0.65 | $2.75 | 8,192 | 8192 |
| replicate/meta/llama-3-8b | $0.05 | $0.25 | 8,086 | 8086 |
| replicate/meta/llama-3-8b-instruct | $0.05 | $0.25 | 8,086 | 8086 |
| replicate/mistralai/mistral-7b-v0.1 | $0.05 | $0.25 | 4,096 | 4096 |
| replicate/mistralai/mistral-7b-instruct-v0.2 | $0.05 | $0.25 | 4,096 | 4096 |
| replicate/mistralai/mixtral-8x7b-instruct-v0.1 | $0.3 | $ 1.00 | 4,096 | 4096 |
| openrouter/deepseek/deepseek-coder | $0.14 | $0.28 | 32,000 | 4096 |
| openrouter/microsoft/wizardlm-2-8x22b:nitro | $ 1.00 | $ 1.00 | nan | nan |
| openrouter/google/gemini-pro-1.5 | $2.5 | $7.5 | 1,000,000 | 8192 |
| openrouter/mistralai/mixtral-8x22b-instruct | $0.65 | $0.65 | nan | nan |
| openrouter/cohere/command-r-plus | $ 3.00 | $15.00 | nan | nan |
| openrouter/databricks/dbrx-instruct | $0.6 | $0.6 | nan | nan |
| openrouter/anthropic/claude-3-haiku | $0.25 | $1.25 | nan | nan |
| openrouter/anthropic/claude-3-haiku-20240307 | $0.25 | $1.25 | 200,000 | 4096 |
| openrouter/anthropic/claude-3.5-sonnet | $ 3.00 | $15.00 | 200,000 | 4096 |
| openrouter/anthropic/claude-3-sonnet | $ 3.00 | $15.00 | nan | nan |
| openrouter/mistralai/mistral-large | $ 8.00 | $24.00 | nan | nan |
| openrouter/cognitivecomputations/dolphin-mixtral-8x7b | $0.5 | $0.5 | nan | nan |
| openrouter/google/gemini-pro-vision | $0.125 | $0.375 | nan | nan |
| openrouter/fireworks/firellava-13b | $0.2 | $0.2 | nan | nan |
| openrouter/meta-llama/llama-3-8b-instruct:free | $ 0.00 | $ 0.00 | nan | nan |
| openrouter/meta-llama/llama-3-8b-instruct:extended | $0.225 | $2.25 | nan | nan |
| openrouter/meta-llama/llama-3-70b-instruct:nitro | $0.9 | $0.9 | nan | nan |
| openrouter/meta-llama/llama-3-70b-instruct | $0.59 | $0.79 | nan | nan |
| openrouter/openai/gpt-4o | $ 5.00 | $15.00 | 128,000 | 4096 |
| openrouter/openai/gpt-4o-2024-05-13 | $ 5.00 | $15.00 | 128,000 | 4096 |
| openrouter/openai/gpt-4-vision-preview | $10.00 | $30.00 | nan | nan |
| openrouter/openai/gpt-3.5-turbo | $1.5 | $ 2.00 | nan | nan |
| openrouter/openai/gpt-3.5-turbo-16k | $ 3.00 | $ 4.00 | nan | nan |
| openrouter/openai/gpt-4 | $30.00 | $60.00 | nan | nan |
| openrouter/anthropic/claude-instant-v1 | $1.63 | $5.51 | nan | 8191 |
| openrouter/anthropic/claude-2 | $11.02 | $32.68 | nan | 8191 |
| openrouter/anthropic/claude-3-opus | $15.00 | $75.00 | 200,000 | 4096 |
| openrouter/google/palm-2-chat-bison | $0.5 | $0.5 | nan | nan |
| openrouter/google/palm-2-codechat-bison | $0.5 | $0.5 | nan | nan |
| openrouter/meta-llama/llama-2-13b-chat | $0.2 | $0.2 | nan | nan |
| openrouter/meta-llama/llama-2-70b-chat | $1.5 | $1.5 | nan | nan |
| openrouter/meta-llama/codellama-34b-instruct | $0.5 | $0.5 | nan | nan |
| openrouter/nousresearch/nous-hermes-llama2-13b | $0.2 | $0.2 | nan | nan |
| openrouter/mancer/weaver | $5.625 | $5.625 | nan | nan |
| openrouter/gryphe/mythomax-l2-13b | $1.875 | $1.875 | nan | nan |
| openrouter/jondurbin/airoboros-l2-70b-2.1 | $13.875 | $13.875 | nan | nan |
| openrouter/undi95/remm-slerp-l2-13b | $1.875 | $1.875 | nan | nan |
| openrouter/pygmalionai/mythalion-13b | $1.875 | $1.875 | nan | nan |
| openrouter/mistralai/mistral-7b-instruct | $0.13 | $0.13 | nan | nan |
| openrouter/mistralai/mistral-7b-instruct:free | $ 0.00 | $ 0.00 | nan | nan |
| j2-ultra | $15.00 | $15.00 | 8,192 | 8192 |
| j2-mid | $10.00 | $10.00 | 8,192 | 8192 |
| j2-light | $ 3.00 | $ 3.00 | 8,192 | 8192 |
| dolphin | $0.5 | $0.5 | 16,384 | 16384 |
| chatdolphin | $0.5 | $0.5 | 16,384 | 16384 |
| luminous-base | $30.00 | $33.00 | nan | nan |
| luminous-base-control | $37.5 | $41.25 | nan | nan |
| luminous-extended | $45.00 | $49.5 | nan | nan |
| luminous-extended-control | $56.25 | $61.875 | nan | nan |
| luminous-supreme | $175.00 | $192.5 | nan | nan |
| luminous-supreme-control | $218.75 | $240.625 | nan | nan |
| ai21.j2-mid-v1 | $12.5 | $12.5 | 8,191 | 8191 |
| ai21.j2-ultra-v1 | $18.8 | $18.8 | 8,191 | 8191 |
| amazon.titan-text-lite-v1 | $0.3 | $0.4 | 42,000 | 4000 |
| amazon.titan-text-express-v1 | $1.3 | $1.7 | 42,000 | 8000 |
| amazon.titan-embed-text-v1 | $0.1 | $ 0.00 | 8,192 | nan |
| amazon.titan-embed-text-v2:0 | $0.2 | $ 0.00 | 8,192 | nan |
| mistral.mistral-7b-instruct-v0:2 | $0.15 | $0.2 | 32,000 | 8191 |
| mistral.mixtral-8x7b-instruct-v0:1 | $0.45 | $0.7 | 32,000 | 8191 |
| mistral.mistral-large-2402-v1:0 | $ 8.00 | $24.00 | 32,000 | 8191 |
| bedrock/us-west-2/mistral.mixtral-8x7b-instruct-v0:1 | $0.45 | $0.7 | 32,000 | 8191 |
| bedrock/us-east-1/mistral.mixtral-8x7b-instruct-v0:1 | $0.45 | $0.7 | 32,000 | 8191 |
| bedrock/eu-west-3/mistral.mixtral-8x7b-instruct-v0:1 | $0.59 | $0.91 | 32,000 | 8191 |
| bedrock/us-west-2/mistral.mistral-7b-instruct-v0:2 | $0.15 | $0.2 | 32,000 | 8191 |
| bedrock/us-east-1/mistral.mistral-7b-instruct-v0:2 | $0.15 | $0.2 | 32,000 | 8191 |
| bedrock/eu-west-3/mistral.mistral-7b-instruct-v0:2 | $0.2 | $0.26 | 32,000 | 8191 |
| bedrock/us-east-1/mistral.mistral-large-2402-v1:0 | $ 8.00 | $24.00 | 32,000 | 8191 |
| bedrock/us-west-2/mistral.mistral-large-2402-v1:0 | $ 8.00 | $24.00 | 32,000 | 8191 |
| bedrock/eu-west-3/mistral.mistral-large-2402-v1:0 | $10.4 | $31.2 | 32,000 | 8191 |
| anthropic.claude-3-sonnet-20240229-v1:0 | $ 3.00 | $15.00 | 200,000 | 4096 |
| anthropic.claude-3-5-sonnet-20240620-v1:0 | $ 3.00 | $15.00 | 200,000 | 4096 |
| anthropic.claude-3-haiku-20240307-v1:0 | $0.25 | $1.25 | 200,000 | 4096 |
| anthropic.claude-3-opus-20240229-v1:0 | $15.00 | $75.00 | 200,000 | 4096 |
| anthropic.claude-v1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/us-east-1/anthropic.claude-v1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/us-west-2/anthropic.claude-v1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/ap-northeast-1/anthropic.claude-v1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/ap-northeast-1/1-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/ap-northeast-1/6-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/anthropic.claude-v1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/eu-central-1/1-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/6-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/1-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/6-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/1-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/6-month-commitment/anthropic.claude-v1 | -- | -- | 100,000 | 8191 |
| anthropic.claude-v2 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/us-east-1/anthropic.claude-v2 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/us-west-2/anthropic.claude-v2 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/ap-northeast-1/anthropic.claude-v2 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/ap-northeast-1/1-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/ap-northeast-1/6-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/anthropic.claude-v2 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/eu-central-1/1-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/6-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/1-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/6-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/1-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/6-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| anthropic.claude-v2:1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/us-east-1/anthropic.claude-v2:1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/us-west-2/anthropic.claude-v2:1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/ap-northeast-1/anthropic.claude-v2:1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/ap-northeast-1/1-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/ap-northeast-1/6-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/anthropic.claude-v2:1 | $ 8.00 | $24.00 | 100,000 | 8191 |
| bedrock/eu-central-1/1-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/6-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/1-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/6-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/1-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/6-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| anthropic.claude-instant-v1 | $1.63 | $5.51 | 100,000 | 8191 |
| bedrock/us-east-1/anthropic.claude-instant-v1 | $0.8 | $2.4 | 100,000 | 8191 |
| bedrock/us-east-1/1-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/6-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/1-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/6-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/anthropic.claude-instant-v1 | $0.8 | $2.4 | 100,000 | 8191 |
| bedrock/ap-northeast-1/anthropic.claude-instant-v1 | $2.23 | $7.55 | 100,000 | 8191 |
| bedrock/ap-northeast-1/1-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/ap-northeast-1/6-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/anthropic.claude-instant-v1 | $2.48 | $8.38 | 100,000 | 8191 |
| bedrock/eu-central-1/1-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/6-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| cohere.command-text-v14 | $1.5 | $ 2.00 | 4,096 | 4096 |
| bedrock/*/1-month-commitment/cohere.command-text-v14 | -- | -- | 4,096 | 4096 |
| bedrock/*/6-month-commitment/cohere.command-text-v14 | -- | -- | 4,096 | 4096 |
| cohere.command-light-text-v14 | $0.3 | $0.6 | 4,096 | 4096 |
| bedrock/*/1-month-commitment/cohere.command-light-text-v14 | -- | -- | 4,096 | 4096 |
| bedrock/*/6-month-commitment/cohere.command-light-text-v14 | -- | -- | 4,096 | 4096 |
| cohere.command-r-plus-v1:0 | $ 3.00 | $15.00 | 128,000 | 4096 |
| cohere.command-r-v1:0 | $0.5 | $1.5 | 128,000 | 4096 |
| cohere.embed-english-v3 | $0.1 | $ 0.00 | 512 | nan |
| cohere.embed-multilingual-v3 | $0.1 | $ 0.00 | 512 | nan |
| meta.llama2-13b-chat-v1 | $0.75 | $ 1.00 | 4,096 | 4096 |
| meta.llama2-70b-chat-v1 | $1.95 | $2.56 | 4,096 | 4096 |
| meta.llama3-8b-instruct-v1:0 | $0.4 | $0.6 | 8,192 | 8192 |
| meta.llama3-70b-instruct-v1:0 | $2.65 | $3.5 | 8,192 | 8192 |
| 512-x-512/50-steps/stability.stable-diffusion-xl-v0 | -- | -- | 77 | nan |
| 512-x-512/max-steps/stability.stable-diffusion-xl-v0 | -- | -- | 77 | nan |
| max-x-max/50-steps/stability.stable-diffusion-xl-v0 | -- | -- | 77 | nan |
| max-x-max/max-steps/stability.stable-diffusion-xl-v0 | -- | -- | 77 | nan |
| 1024-x-1024/50-steps/stability.stable-diffusion-xl-v1 | -- | -- | 77 | nan |
| 1024-x-1024/max-steps/stability.stable-diffusion-xl-v1 | -- | -- | 77 | nan |
| sagemaker/meta-textgeneration-llama-2-7b | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| sagemaker/meta-textgeneration-llama-2-7b-f | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| sagemaker/meta-textgeneration-llama-2-13b | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| sagemaker/meta-textgeneration-llama-2-13b-f | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| sagemaker/meta-textgeneration-llama-2-70b | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| sagemaker/meta-textgeneration-llama-2-70b-b-f | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| together-ai-up-to-4b | $0.1 | $0.1 | nan | nan |
| together-ai-4.1b-8b | $0.2 | $0.2 | nan | nan |
| together-ai-8.1b-21b | $0.3 | $0.3 | nan | nan |
| together-ai-21.1b-41b | $0.8 | $0.8 | nan | nan |
| together-ai-41.1b-80b | $0.9 | $0.9 | nan | nan |
| together-ai-81.1b-110b | $1.8 | $1.8 | nan | nan |
| together_ai/mistralai/Mixtral-8x7B-Instruct-v0.1 | $0.6 | $0.6 | nan | nan |
| together_ai/mistralai/Mistral-7B-Instruct-v0.1 | -- | -- | nan | nan |
| together_ai/togethercomputer/CodeLlama-34b-Instruct | -- | -- | nan | nan |
| ollama/codegemma | $ 0.00 | $ 0.00 | 8,192 | 8192 |
| ollama/llama2 | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| ollama/llama2:13b | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| ollama/llama2:70b | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| ollama/llama2-uncensored | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| ollama/llama3 | $ 0.00 | $ 0.00 | 8,192 | 8192 |
| ollama/llama3:70b | $ 0.00 | $ 0.00 | 8,192 | 8192 |
| ollama/mistral | $ 0.00 | $ 0.00 | 8,192 | 8192 |
| ollama/mistral-7B-Instruct-v0.1 | $ 0.00 | $ 0.00 | 8,192 | 8192 |
| ollama/mistral-7B-Instruct-v0.2 | $ 0.00 | $ 0.00 | 32,768 | 32768 |
| ollama/mixtral-8x7B-Instruct-v0.1 | $ 0.00 | $ 0.00 | 32,768 | 32768 |
| ollama/mixtral-8x22B-Instruct-v0.1 | $ 0.00 | $ 0.00 | 65,536 | 65536 |
| ollama/codellama | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| ollama/orca-mini | $ 0.00 | $ 0.00 | 4,096 | 4096 |
| ollama/vicuna | $ 0.00 | $ 0.00 | 2,048 | 2048 |
| deepinfra/lizpreciatior/lzlv_70b_fp16_hf | $0.7 | $0.9 | 4,096 | 4096 |
| deepinfra/Gryphe/MythoMax-L2-13b | $0.22 | $0.22 | 4,096 | 4096 |
| deepinfra/mistralai/Mistral-7B-Instruct-v0.1 | $0.13 | $0.13 | 32,768 | 8191 |
| deepinfra/meta-llama/Llama-2-70b-chat-hf | $0.7 | $0.9 | 4,096 | 4096 |
| deepinfra/cognitivecomputations/dolphin-2.6-mixtral-8x7b | $0.27 | $0.27 | 32,768 | 8191 |
| deepinfra/codellama/CodeLlama-34b-Instruct-hf | $0.6 | $0.6 | 4,096 | 4096 |
| deepinfra/deepinfra/mixtral | $0.27 | $0.27 | 32,000 | 4096 |
| deepinfra/Phind/Phind-CodeLlama-34B-v2 | $0.6 | $0.6 | 16,384 | 4096 |
| deepinfra/mistralai/Mixtral-8x7B-Instruct-v0.1 | $0.27 | $0.27 | 32,768 | 8191 |
| deepinfra/deepinfra/airoboros-70b | $0.7 | $0.9 | 4,096 | 4096 |
| deepinfra/01-ai/Yi-34B-Chat | $0.6 | $0.6 | 4,096 | 4096 |
| deepinfra/01-ai/Yi-6B-200K | $0.13 | $0.13 | 200,000 | 4096 |
| deepinfra/jondurbin/airoboros-l2-70b-gpt4-1.4.1 | $0.7 | $0.9 | 4,096 | 4096 |
| deepinfra/meta-llama/Llama-2-13b-chat-hf | $0.22 | $0.22 | 4,096 | 4096 |
| deepinfra/amazon/MistralLite | $0.2 | $0.2 | 32,768 | 8191 |
| deepinfra/meta-llama/Llama-2-7b-chat-hf | $0.13 | $0.13 | 4,096 | 4096 |
| deepinfra/meta-llama/Meta-Llama-3-8B-Instruct | $0.08 | $0.08 | 8,191 | 4096 |
| deepinfra/meta-llama/Meta-Llama-3-70B-Instruct | $0.59 | $0.79 | 8,191 | 4096 |
| deepinfra/01-ai/Yi-34B-200K | $0.6 | $0.6 | 200,000 | 4096 |
| deepinfra/openchat/openchat_3.5 | $0.13 | $0.13 | 4,096 | 4096 |
| perplexity/codellama-34b-instruct | $0.35 | $1.4 | 16,384 | 16384 |
| perplexity/codellama-70b-instruct | $0.7 | $2.8 | 16,384 | 16384 |
| perplexity/pplx-7b-chat | $0.07 | $0.28 | 8,192 | 8192 |
| perplexity/pplx-70b-chat | $0.7 | $2.8 | 4,096 | 4096 |
| perplexity/pplx-7b-online | $ 0.00 | $0.28 | 4,096 | 4096 |
| perplexity/pplx-70b-online | $ 0.00 | $2.8 | 4,096 | 4096 |
| perplexity/llama-2-70b-chat | $0.7 | $2.8 | 4,096 | 4096 |
| perplexity/mistral-7b-instruct | $0.07 | $0.28 | 4,096 | 4096 |
| perplexity/mixtral-8x7b-instruct | $0.07 | $0.28 | 4,096 | 4096 |
| perplexity/sonar-small-chat | $0.07 | $0.28 | 16,384 | 16384 |
| perplexity/sonar-small-online | $ 0.00 | $0.28 | 12,000 | 12000 |
| perplexity/sonar-medium-chat | $0.6 | $1.8 | 16,384 | 16384 |
| perplexity/sonar-medium-online | $ 0.00 | $1.8 | 12,000 | 12000 |
| anyscale/mistralai/Mistral-7B-Instruct-v0.1 | $0.15 | $0.15 | 16,384 | 16384 |
| anyscale/mistralai/Mixtral-8x7B-Instruct-v0.1 | $0.15 | $0.15 | 16,384 | 16384 |
| anyscale/mistralai/Mixtral-8x22B-Instruct-v0.1 | $0.9 | $0.9 | 65,536 | 65536 |
| anyscale/HuggingFaceH4/zephyr-7b-beta | $0.15 | $0.15 | 16,384 | 16384 |
| anyscale/google/gemma-7b-it | $0.15 | $0.15 | 8,192 | 8192 |
| anyscale/meta-llama/Llama-2-7b-chat-hf | $0.15 | $0.15 | 4,096 | 4096 |
| anyscale/meta-llama/Llama-2-13b-chat-hf | $0.25 | $0.25 | 4,096 | 4096 |
| anyscale/meta-llama/Llama-2-70b-chat-hf | $ 1.00 | $ 1.00 | 4,096 | 4096 |
| anyscale/codellama/CodeLlama-34b-Instruct-hf | $ 1.00 | $ 1.00 | 4,096 | 4096 |
| anyscale/codellama/CodeLlama-70b-Instruct-hf | $ 1.00 | $ 1.00 | 4,096 | 4096 |
| anyscale/meta-llama/Meta-Llama-3-8B-Instruct | $0.15 | $0.15 | 8,192 | 8192 |
| anyscale/meta-llama/Meta-Llama-3-70B-Instruct | $ 1.00 | $ 1.00 | 8,192 | 8192 |
| cloudflare/@cf/meta/llama-2-7b-chat-fp16 | $1.923 | $1.923 | 3,072 | 3072 |
| cloudflare/@cf/meta/llama-2-7b-chat-int8 | $1.923 | $1.923 | 2,048 | 2048 |
| cloudflare/@cf/mistral/mistral-7b-instruct-v0.1 | $1.923 | $1.923 | 8,192 | 8192 |
| cloudflare/@hf/thebloke/codellama-7b-instruct-awq | $1.923 | $1.923 | 4,096 | 4096 |
| voyage/voyage-01 | $0.1 | $ 0.00 | 4,096 | nan |
| voyage/voyage-lite-01 | $0.1 | $ 0.00 | 4,096 | nan |
| voyage/voyage-large-2 | $0.12 | $ 0.00 | 16,000 | nan |
| voyage/voyage-law-2 | $0.12 | $ 0.00 | 16,000 | nan |
| voyage/voyage-code-2 | $0.12 | $ 0.00 | 16,000 | nan |
| voyage/voyage-2 | $0.1 | $ 0.00 | 4,000 | nan |
| voyage/voyage-lite-02-instruct | $0.1 | $ 0.00 | 4,000 | nan |
| databricks/databricks-dbrx-instruct | $0.75 | $2.25 | 32,768 | 32768 |
| databricks/databricks-meta-llama-3-70b-instruct | $ 1.00 | $ 3.00 | 8,192 | 8192 |
| databricks/databricks-llama-2-70b-chat | $0.5 | $1.5 | 4,096 | 4096 |
| databricks/databricks-mixtral-8x7b-instruct | $0.5 | $ 1.00 | 4,096 | 4096 |
| databricks/databricks-mpt-30b-instruct | $ 1.00 | $ 1.00 | 8,192 | 8192 |
| databricks/databricks-mpt-7b-instruct | $0.5 | $0.5 | 8,192 | 8192 |
| databricks/databricks-bge-large-en | $0.1 | $ 0.00 | 512 | nan |
You may also calculate token costs in LLM wrapper/framework libraries using callbacks.
pip install `'tokencost[llama-index]'`To use the base callback handler, you may import it:
from tokencost.callbacks.llama_index import TokenCostHandlerand pass to your framework callback handler.
(Coming Soon)
Installation via GitHub:
git clone [email protected]:AgentOps-AI/tokencost.git
cd tokencost
pip install -e .- Install
pytestif you don't have it already
pip install pytest- Run the
tests/folder while in the parent directory
pytest testsThis repo also supports tox, simply run python -m tox.
Contributions to TokenCost are welcome! Feel free to create an issue for any bug reports, complaints, or feature suggestions.
TokenCost is released under the MIT License.
For Tasks:
Click tags to check more tools for each tasksFor Jobs:
Alternative AI tools for tokencost
Similar Open Source Tools
tokencost
Tokencost is a clientside tool for calculating the USD cost of using major Large Language Model (LLMs) APIs by estimating the cost of prompts and completions. It helps track the latest price changes of major LLM providers, accurately count prompt tokens before sending OpenAI requests, and easily integrate to get the cost of a prompt or completion with a single function. Users can calculate prompt and completion costs using OpenAI requests, count tokens in prompts formatted as message lists or string prompts, and refer to a cost table with updated prices for various LLM models. The tool also supports callback handlers for LLM wrapper/framework libraries like LlamaIndex and Langchain.
BlossomLM
BlossomLM is a series of open-source conversational large language models. This project aims to provide a high-quality general-purpose SFT dataset in both Chinese and English, making fine-tuning accessible while also providing pre-trained model weights. **Hint**: BlossomLM is a personal non-commercial project.
ML-AI-2-LT
ML-AI-2-LT is a repository that serves as a glossary for machine learning and deep learning concepts. It contains translations and explanations of various terms related to artificial intelligence, including definitions and notes. Users can contribute by filling issues for unclear concepts or by submitting pull requests with suggestions or additions. The repository aims to provide a comprehensive resource for understanding key terminology in the field of AI and machine learning.
yudao-cloud
Yudao-cloud is an open-source project designed to provide a fast development platform for developers in China. It includes various system functions, infrastructure, member center, data reports, workflow, mall system, WeChat public account, CRM, ERP, etc. The project is based on Java backend with Spring Boot and Spring Cloud Alibaba microservices architecture. It supports multiple databases, message queues, authentication systems, dynamic menu loading, SaaS multi-tenant system, code generator, real-time communication, integration with third-party services like WeChat, Alipay, and more. The project is well-documented and follows the Alibaba Java development guidelines, ensuring clean code and architecture.
adata
AData is a free and open-source A-share database that focuses on transaction-related data. It provides comprehensive data on stocks, including basic information, market data, and sentiment analysis. AData is designed to be easy to use and integrate with other applications, making it a valuable tool for quantitative trading and AI training.
Awesome-AGI
Awesome-AGI is a curated list of resources related to Artificial General Intelligence (AGI), including models, pipelines, applications, and concepts. It provides a comprehensive overview of the current state of AGI research and development, covering various aspects such as model training, fine-tuning, deployment, and applications in different domains. The repository also includes resources on prompt engineering, RLHF, LLM vocabulary expansion, long text generation, hallucination mitigation, controllability and safety, and text detection. It serves as a valuable resource for researchers, practitioners, and anyone interested in the field of AGI.
yudao-ui-admin-vue3
The yudao-ui-admin-vue3 repository is an open-source project focused on building a fast development platform for developers in China. It utilizes Vue3 and Element Plus to provide features such as configurable themes, internationalization, dynamic route permission generation, common component encapsulation, and rich examples. The project supports the latest front-end technologies like Vue3 and Vite4, and also includes tools like TypeScript, pinia, vueuse, vue-i18n, vue-router, unocss, iconify, and wangeditor. It offers a range of development tools and features for system functions, infrastructure, workflow management, payment systems, member centers, data reporting, e-commerce systems, WeChat public accounts, ERP systems, and CRM systems.
ruoyi-vue-pro
The ruoyi-vue-pro repository is an open-source project that provides a comprehensive development platform with various functionalities such as system features, infrastructure, member center, data reports, workflow, payment system, mall system, ERP system, CRM system, and AI big model. It is built using Java backend with Spring Boot framework and Vue frontend with different versions like Vue3 with element-plus, Vue3 with vben(ant-design-vue), and Vue2 with element-ui. The project aims to offer a fast development platform for developers and enterprises, supporting features like dynamic menu loading, button-level access control, SaaS multi-tenancy, code generator, real-time communication, integration with third-party services like WeChat, Alipay, and cloud services, and more.
yudao-boot-mini
yudao-boot-mini is an open-source project focused on developing a rapid development platform for developers in China. It includes features like system functions, infrastructure, member center, data reports, workflow, mall system, WeChat official account, CRM, ERP, etc. The project is based on Spring Boot with Java backend and Vue for frontend. It offers various functionalities such as user management, role management, menu management, department management, workflow management, payment system, code generation, API documentation, database documentation, file service, WebSocket integration, message queue, Java monitoring, and more. The project is licensed under the MIT License, allowing both individuals and enterprises to use it freely without restrictions.
PaddleScience
PaddleScience is a scientific computing suite developed based on the deep learning framework PaddlePaddle. It utilizes the learning ability of deep neural networks and the automatic (higher-order) differentiation mechanism of PaddlePaddle to solve problems in physics, chemistry, meteorology, and other fields. It supports three solving methods: physics mechanism-driven, data-driven, and mathematical fusion, and provides basic APIs and detailed documentation for users to use and further develop.
LLM-for-Healthcare
The repository 'LLM-for-Healthcare' provides a comprehensive survey of large language models (LLMs) for healthcare, covering data, technology, applications, and accountability and ethics. It includes information on various LLM models, training data, evaluation methods, and computation costs. The repository also discusses tasks such as NER, text classification, question answering, dialogue systems, and generation of medical reports from images in the healthcare domain.
Chinese-LLaMA-Alpaca-3
Chinese-LLaMA-Alpaca-3 is a project based on Meta's latest release of the new generation open-source large model Llama-3. It is the third phase of the Chinese-LLaMA-Alpaca open-source large model series projects (Phase 1, Phase 2). This project open-sources the Chinese Llama-3 base model and the Chinese Llama-3-Instruct instruction fine-tuned large model. These models incrementally pre-train with a large amount of Chinese data on the basis of the original Llama-3 and further fine-tune using selected instruction data, enhancing Chinese basic semantics and instruction understanding capabilities. Compared to the second-generation related models, significant performance improvements have been achieved.
linktre-tools
The 'linktre-tools' repository is a collection of tools and resources for independent developers, AI products, cross-border e-commerce, and self-media office assistance. It aims to provide a curated list of tools and products in these areas. Users are encouraged to contribute by submitting pull requests and raising issues for continuous updates. The repository covers a wide range of topics including AI tools, independent development tools, popular AI products, tools for web development, online tools, media operations, and cross-border e-commerce resources.
kumo-search
Kumo search is an end-to-end search engine framework that supports full-text search, inverted index, forward index, sorting, caching, hierarchical indexing, intervention system, feature collection, offline computation, storage system, and more. It runs on the EA (Elastic automic infrastructure architecture) platform, enabling engineering automation, service governance, real-time data, service degradation, and disaster recovery across multiple data centers and clusters. The framework aims to provide a ready-to-use search engine framework to help users quickly build their own search engines. Users can write business logic in Python using the AOT compiler in the project, which generates C++ code and binary dynamic libraries for rapid iteration of the search engine.
ai-hub
AI Hub Project aims to continuously test and evaluate mainstream large language models, while accumulating and managing various effective model invocation prompts. It has integrated all mainstream large language models in China, including OpenAI GPT-4 Turbo, Baidu ERNIE-Bot-4, Tencent ChatPro, MiniMax abab5.5-chat, and more. The project plans to continuously track, integrate, and evaluate new models. Users can access the models through REST services or Java code integration. The project also provides a testing suite for translation, coding, and benchmark testing.
For similar tasks
tokencost
Tokencost is a clientside tool for calculating the USD cost of using major Large Language Model (LLMs) APIs by estimating the cost of prompts and completions. It helps track the latest price changes of major LLM providers, accurately count prompt tokens before sending OpenAI requests, and easily integrate to get the cost of a prompt or completion with a single function. Users can calculate prompt and completion costs using OpenAI requests, count tokens in prompts formatted as message lists or string prompts, and refer to a cost table with updated prices for various LLM models. The tool also supports callback handlers for LLM wrapper/framework libraries like LlamaIndex and Langchain.
llm
The 'llm' package for Emacs provides an interface for interacting with Large Language Models (LLMs). It abstracts functionality to a higher level, concealing API variations and ensuring compatibility with various LLMs. Users can set up providers like OpenAI, Gemini, Vertex, Claude, Ollama, GPT4All, and a fake client for testing. The package allows for chat interactions, embeddings, token counting, and function calling. It also offers advanced prompt creation and logging capabilities. Users can handle conversations, create prompts with placeholders, and contribute by creating providers.
gigachat
GigaChat is a Python library that allows GigaChain to interact with GigaChat, a neural network model capable of engaging in dialogue, writing code, creating texts, and images on demand. Data exchange with the service is facilitated through the GigaChat API. The library supports processing token streaming, as well as working in synchronous or asynchronous mode. It enables precise token counting in text using the GigaChat API.
client
Gemini API PHP Client is a library that allows you to interact with Google's generative AI models, such as Gemini Pro and Gemini Pro Vision. It provides functionalities for basic text generation, multimodal input, chat sessions, streaming responses, tokens counting, listing models, and advanced usages like safety settings and custom HTTP client usage. The library requires an API key to access Google's Gemini API and can be installed using Composer. It supports various features like generating content, starting chat sessions, embedding content, counting tokens, and listing available models.
gemini-cli
gemini-cli is a versatile command-line interface for Google's Gemini LLMs, written in Go. It includes tools for chatting with models, generating/comparing embeddings, and storing data in SQLite for analysis. Users can interact with Gemini models through various subcommands like prompt, chat, counttok, embed content, embed db, and embed similar.
client
Gemini PHP is a PHP API client for interacting with the Gemini AI API. It allows users to generate content, chat, count tokens, configure models, embed resources, list models, get model information, troubleshoot timeouts, and test API responses. The client supports various features such as text-only input, text-and-image input, multi-turn conversations, streaming content generation, token counting, model configuration, and embedding techniques. Users can interact with Gemini's API to perform tasks related to natural language generation and text analysis.
ai21-python
The AI21 Labs Python SDK is a comprehensive tool for interacting with the AI21 API. It provides functionalities for chat completions, conversational RAG, token counting, error handling, and support for various cloud providers like AWS, Azure, and Vertex. The SDK offers both synchronous and asynchronous usage, along with detailed examples and documentation. Users can quickly get started with the SDK to leverage AI21's powerful models for various natural language processing tasks.
Tiktoken
Tiktoken is a high-performance implementation focused on token count operations. It provides various encodings like o200k_base, cl100k_base, r50k_base, p50k_base, and p50k_edit. Users can easily encode and decode text using the provided API. The repository also includes a benchmark console app for performance tracking. Contributions in the form of PRs are welcome.
For similar jobs
responsible-ai-toolbox
Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment interfaces and libraries for understanding AI systems. It empowers developers and stakeholders to develop and monitor AI responsibly, enabling better data-driven actions. The toolbox includes visualization widgets for model assessment, error analysis, interpretability, fairness assessment, and mitigations library. It also offers a JupyterLab extension for managing machine learning experiments and a library for measuring gender bias in NLP datasets.
LLMLingua
LLMLingua is a tool that utilizes a compact, well-trained language model to identify and remove non-essential tokens in prompts. This approach enables efficient inference with large language models, achieving up to 20x compression with minimal performance loss. The tool includes LLMLingua, LongLLMLingua, and LLMLingua-2, each offering different levels of prompt compression and performance improvements for tasks involving large language models.
llm-examples
Starter examples for building LLM apps with Streamlit. This repository showcases a growing collection of LLM minimum working examples, including a Chatbot, File Q&A, Chat with Internet search, LangChain Quickstart, LangChain PromptTemplate, and Chat with user feedback. Users can easily get their own OpenAI API key and set it as an environment variable in Streamlit apps to run the examples locally.
LMOps
LMOps is a research initiative focusing on fundamental research and technology for building AI products with foundation models, particularly enabling AI capabilities with Large Language Models (LLMs) and Generative AI models. The project explores various aspects such as prompt optimization, longer context handling, LLM alignment, acceleration of LLMs, LLM customization, and understanding in-context learning. It also includes tools like Promptist for automatic prompt optimization, Structured Prompting for efficient long-sequence prompts consumption, and X-Prompt for extensible prompts beyond natural language. Additionally, LLMA accelerators are developed to speed up LLM inference by referencing and copying text spans from documents. The project aims to advance technologies that facilitate prompting language models and enhance the performance of LLMs in various scenarios.
awesome-tool-llm
This repository focuses on exploring tools that enhance the performance of language models for various tasks. It provides a structured list of literature relevant to tool-augmented language models, covering topics such as tool basics, tool use paradigm, scenarios, advanced methods, and evaluation. The repository includes papers, preprints, and books that discuss the use of tools in conjunction with language models for tasks like reasoning, question answering, mathematical calculations, accessing knowledge, interacting with the world, and handling non-textual modalities.
gaianet-node
GaiaNet-node is a tool that allows users to run their own GaiaNet node, enabling them to interact with an AI agent. The tool provides functionalities to install the default node software stack, initialize the node with model files and vector database files, start the node, stop the node, and update configurations. Users can use pre-set configurations or pass a custom URL for initialization. The tool is designed to facilitate communication with the AI agent and access node information via a browser. GaiaNet-node requires sudo privilege for installation but can also be installed without sudo privileges with specific commands.
llmops-duke-aipi
LLMOps Duke AIPI is a course focused on operationalizing Large Language Models, teaching methodologies for developing applications using software development best practices with large language models. The course covers various topics such as generative AI concepts, setting up development environments, interacting with large language models, using local large language models, applied solutions with LLMs, extensibility using plugins and functions, retrieval augmented generation, introduction to Python web frameworks for APIs, DevOps principles, deploying machine learning APIs, LLM platforms, and final presentations. Students will learn to build, share, and present portfolios using Github, YouTube, and Linkedin, as well as develop non-linear life-long learning skills. Prerequisites include basic Linux and programming skills, with coursework available in Python or Rust. Additional resources and references are provided for further learning and exploration.
Awesome-AISourceHub
Awesome-AISourceHub is a repository that collects high-quality information sources in the field of AI technology. It serves as a synchronized source of information to avoid information gaps and information silos. The repository aims to provide valuable resources for individuals such as AI book authors, enterprise decision-makers, and tool developers who frequently use Twitter to share insights and updates related to AI advancements. The platform emphasizes the importance of accessing information closer to the source for better quality content. Users can contribute their own high-quality information sources to the repository by following specific steps outlined in the contribution guidelines. The repository covers various platforms such as Twitter, public accounts, knowledge planets, podcasts, blogs, websites, YouTube channels, and more, offering a comprehensive collection of AI-related resources for individuals interested in staying updated with the latest trends and developments in the AI field.