pinecone-ts-client
The official TypeScript/Node client for the Pinecone vector database
Stars: 173
The official Node.js client for Pinecone, written in TypeScript. This client library provides a high-level interface for interacting with the Pinecone vector database service. With this client, you can create and manage indexes, upsert and query vector data, and perform other operations related to vector search and retrieval. The client is designed to be easy to use and provides a consistent and idiomatic experience for Node.js developers. It supports all the features and functionality of the Pinecone API, making it a comprehensive solution for building vector-powered applications in Node.js.
README:
This is the official Node.js SDK for Pinecone, written in TypeScript.
- Reference Documentation
- If you are upgrading from
v0.x
, check out the v1 Migration Guide. - If you are upgrading from
v1.x
, check out the v2 Migration Guide.
The snippets shown in this README are intended to be concise. For more realistic examples, explore these examples:
There is a breaking change involving the configureIndex
operation in this update. The structure of the object passed
when configuring an index has changed to include deletionProtection
. The podType
and replicas
fields can now be updated through the spec.pod
object. See Configure pod-based indexes for an example of the code.
-
Upgrading to
2.x
: There were many changes made in this release to support Pinecone's new Serverless index offering. The changes are covered in detail in the v2 Migration Guide. Serverless indexes are only available in2.x
release versions or greater. -
Upgrading to
1.x
: This release officially moved the SDK out of beta, and there are a number of breaking changes that need to be addressed when upgrading from a0.x
version. See the v1 Migration Guide for details.
The Pinecone TypeScript SDK is compatible with TypeScript >=4.1 and Node >=18.x.
npm install @pinecone-database/pinecone
The Pinecone Typescript SDK is intended for server-side use only. Using the SDK within a browser context can expose your API key(s). If you have deployed the SDK to production in a browser, please rotate your API keys.
An API key is required to initialize the client. It can be passed using an environment variable or in code through a configuration object. Get an API key in the console.
The environment variable used to configure the API key for the client is the following:
PINECONE_API_KEY="your_api_key"
PINECONE_API_KEY
is the only required variable. When this environment variable is set, the client constructor does not require any additional arguments.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
If you prefer to pass configuration in code, the constructor accepts a config object containing the apiKey
value.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone({
apiKey: 'your_api_key',
});
At a minimum, to create a serverless index you must specify a name
, dimension
, and spec
. The dimension
indicates the size of the records you intend to store in the index. For example, if your intention was to store and query embeddings generated with OpenAI's textembedding-ada-002 model, you would need to create an index with dimension 1536
to match the output of that model.
The spec
configures how the index should be deployed. For serverless indexes, you define only the cloud and region where the index should be hosted. For pod-based indexes, you define the environment where the index should be hosted, the pod type and size to use, and other index characteristics. For more information on serverless and regional availability, see Understanding indexes.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
await pc.createIndex({
name: 'sample-index',
dimension: 1536,
spec: {
serverless: {
cloud: 'aws',
region: 'us-west-2',
},
},
});
To create a pod-based index, you define pod
in the spec
object which contains the environment
where the index should be hosted, and the podType
and pods
size to use. Many optional configuration fields allow greater control over hardware resources and availability. To learn more about the purpose of these fields, see Understanding indexes and Scale pod-based indexes.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
await pc.createIndex({
name: 'sample-index-2',
dimension: 1536,
metric: 'dotproduct',
spec: {
pod: {
environment: 'us-east4-gcp',
pods: 2,
podType: 'p1.x2',
metadataConfig: {
indexed: ['product_type'],
},
},
},
// This option tells the client not to throw if the index already exists.
suppressConflicts: true,
// This option tells the client not to resolve the promise until the
// index is ready.
waitUntilReady: true,
});
The createIndex
method issues a create request to the API that returns quickly, but the resulting index is
not immediately ready for upserting, querying, or performing other data operations. You can use the
describeIndex
method to find out the status of an index and see whether it is ready for use.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
await pc.describeIndex('serverless-index');
// {
// name: 'serverless-index',
// dimension: 1536,
// metric: 'cosine',
// host: 'serverless-index-4zo0ijk.svc.us-west2-aws.pinecone.io',
// deletionProtection: 'disabled',
// spec: {
// serverless: {
// cloud: 'aws',
// region: 'us-west-2'
// }
// },
// status: {
// ready: false,
// state: 'Initializing'
// }
// }
If you pass the waitUntilReady
option, the client will handle polling for status updates on a newly created index. The promise returned by createIndex
will not be resolved until the index status indicates it is ready to handle data operations. This can be especially useful for integration testing, where index creation in a setup step will be immediately followed by data operations.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
await pc.createIndex({
name: 'serverless-index',
dimension: 1536,
spec: {
serverless: {
cloud: 'aws',
region: 'us-west-2',
},
},
waitUntilReady: true,
});
ℹ️ Note
Serverless and starter indexes do not support collections.
As you use Pinecone for more things, you may wish to explore different index configurations with the same vector data. Collections provide an easy way to do this. See other client methods for working with collections here.
Given that you have an existing collection:
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
await pc.describeCollection('product-description-embeddings');
// {
// name: 'product-description-embeddings',
// size: 543427063,
// status: 'Ready',
// dimension: 2,
// vectorCount: 10001498,
// environment: 'us-east4-gcp'
// }
Note: For pod-based indexes, you can specify a sourceCollection
from which to create an index. The
collection must be in the same environment as the index.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
await pc.createIndex({
name: 'product-description-p1x1',
dimension: 256,
metric: 'cosine',
spec: {
pod: {
environment: 'us-east4-gcp',
pods: 1,
podType: 'p1.x1',
sourceCollection: 'product-description-embeddings',
},
},
});
When the new index is ready, it should contain all the data that was in the collection, ready to be queried.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
await pc.index('product-description-p2x2').describeIndexStats();
// {
// namespaces: { '': { recordCount: 78000 } },
// dimension: 256,
// indexFullness: 0.9,
// totalRecordCount: 78000
// }
You can configure both serverless and pod indexes with deletionProtection
. Any index with this property set to 'enabled'
will be unable to be deleted. By default, deletionProtection
will be set to 'disabled'
if not provided as a part of the createIndex
request. To enable deletionProtection
you can pass the value while calling createIndex
.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
await pc.createIndex({
name: 'deletion-protected-index',
dimension: 1536,
metric: 'cosine',
deletionProtection: 'enabled',
spec: {
serverless: {
cloud: 'aws',
region: 'us-west-2',
},
},
});
To disable deletion protection, you can use the configureIndex
operation.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
await pc.configureIndex('deletion-protected-index', {
deletionProtection: 'disabled',
});
You can fetch the description of any index by name using describeIndex
.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
await pc.describeIndex('serverless-index');
// {
// name: 'serverless-index',
// dimension: 1536,
// metric: 'cosine',
// host: 'serverless-index-4zo0ijk.svc.us-west2-aws.pinecone.io',
// deletionProtection: 'disabled',
// spec: {
// serverless: {
// cloud: 'aws',
// region: 'us-west-2'
// },
// },
// status: {
// ready: true,
// state: 'Ready'
// }
// }
ℹ️ Note
This section applies to pod-based indexes only. With serverless indexes, you don't configure any compute or storage resources. Instead, serverless indexes scale automatically based on usage.
You can adjust the number of replicas or scale to a larger pod size (specified with podType
). See Scale pod-based indexes. You cannot downgrade pod size or change the base pod type.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
await pc.configureIndex('pod-index', {
spec: {
pod: {
replicas: 2,
podType: 'p1.x4',
},
},
});
const config = await pc.describeIndex('pod-index');
// {
// name: 'pod-index',
// dimension: 1536,
// metric: 'cosine',
// host: 'pod-index-4zo0ijk.svc.us-east1-gcp.pinecone.io',
// deletionProtection: 'disabled',
// spec: {
// pod: {
// environment: 'us-east1-gcp',
// replicas: 2,
// shards: 2,
// podType: 'p1.x4',
// pods: 4,
// metadataConfig: [Object],
// sourceCollection: undefined
// }
// },
// status: {
// ready: true,
// state: 'ScalingUpPodSize'
// }
// }
Indexes are deleted by name.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
await pc.deleteIndex('sample-index');
The listIndexes
command returns an object with an array of index models under indexes
.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
await pc.listIndexes();
// {
// indexes: [
// {
// name: 'serverless-index',
// dimension: 1536,
// metric: 'cosine',
// host: 'serverless-index-4zo0ijk.svc.us-west2-aws.pinecone.io',
// deletionProtection: 'disabled',
// spec: {
// serverless: {
// cloud: 'aws',
// region: 'us-west-2',
// },
// },
// status: {
// ready: true,
// state: 'Ready',
// },
// },
// {
// name: 'pod-index',
// dimension: 1536,
// metric: 'cosine',
// host: 'pod-index-4zo0ijk.svc.us-west2-aws.pinecone.io',
// deletionProtection: 'disabled',
// spec: {
// pod: {
// environment: 'us-west2-aws',
// replicas: 1,
// shards: 1,
// podType: 'p1.x1',
// pods: 1,
// },
// },
// status: {
// ready: true,
// state: 'Ready',
// },
// },
// ],
// }
ℹ️ Note
Serverless and starter indexes do not support collections.
A collection is a static copy of a pod-based index that may be used to create backups, to create copies of indexes, or to perform experiments with different index configurations. To learn more about Pinecone collections, see Understanding collections.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
await pc.createCollection({
name: 'collection-name',
source: 'index-name',
});
This API call should return quickly, but the creation of a collection can take from minutes to hours depending on the size of the source index and the index's configuration. Use describeCollection
to check the status of a collection.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
await pc.deleteCollection('collection-name');
You can use listCollections
to confirm the deletion.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const describeCollection = await pc.describeCollection('collection3');
// {
// name: 'collection3',
// size: 3126700,
// status: 'Ready',
// dimension: 3,
// vectorCount: 1234,
// environment: 'us-east1-gcp',
// }
The listCollections
command returns an object with an array of collection models under collections
.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const list = await pc.listCollections();
// {
// collections: [
// {
// name: 'collection1',
// size: 3089687,
// status: 'Ready',
// dimension: 3,
// vectorCount: 17378,
// environment: 'us-west1-gcp',
// },
// {
// name: 'collection2',
// size: 208309,
// status: 'Ready',
// dimension: 3,
// vectorCount: 1000,
// environment: 'us-east4-gcp',
// },
// ];
// }
Pinecone indexes support operations for working with vector data using operations such as upsert, query, fetch, and delete.
To perform data operations on an index, you target it using the index
method.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const index = pc.index('test-index');
// Now perform index operations
await index.fetch(['1']);
The first argument is the name of the index you are targeting. There's an optional second argument for providing an
index host override. Providing this second argument allows you to bypass the SDK's default behavior of resolving
your index host via the provided index name. You can find your index host in the Pinecone console, or by using the describeIndex
or listIndexes
operations.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const index = pc.index('test-index', 'my-index-host-1532-svc.io');
// Now perform index operations against: https://my-index-host-1532-svc.io
await index.fetch(['1']);
If you are storing metadata alongside your vector values, you can pass a type parameter to index()
in order to get proper TypeScript typechecking.
import { Pinecone, PineconeRecord } from '@pinecone-database/pinecone';
const pc = new Pinecone();
type MovieMetadata = {
title: string,
runtime: number,
genre: 'comedy' | 'horror' | 'drama' | 'action'
}
// Specify a custom metadata type while targeting the index
const index = pc.index<MovieMetadata>('test-index');
// Now you get type errors if upserting malformed metadata
await index.upsert([{
id: '1234',
values: [
.... // embedding values
],
metadata: {
genre: 'Gone with the Wind',
runtime: 238,
genre: 'drama',
// @ts-expect-error because category property not in MovieMetadata
category: 'classic'
}
}])
const results = await index.query({
vector: [
... // query embedding
],
filter: { genre: { '$eq': 'drama' }}
})
const movie = results.matches[0];
if (movie.metadata) {
// Since we passed the MovieMetadata type parameter above,
// we can interact with metadata fields without having to
// do any typecasting.
const { title, runtime, genre } = movie.metadata;
console.log(`The best match in drama was ${title}`)
}
By default, all data operations take place inside the default namespace of ''
. If you are working with other non-default namespaces, you can target the namespace by chaining a call to namespace()
.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const index = pc.index('test-index').namespace('ns1');
// Now perform index operations in the targeted index and namespace
await index.fetch(['1']);
See Use namespaces for more information.
Pinecone expects records inserted into indexes to have the following form:
type PineconeRecord = {
id: string;
values: Array<number>;
sparseValues?: Array<number>;
metadata?: object;
};
To upsert some records, you can use the client like so:
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
// Target an index
const index = pc.index('sample-index');
// Prepare your data. The length of each array
// of vector values must match the dimension of
// the index where you plan to store them.
const records = [
{
id: '1',
values: [0.236, 0.971, 0.559],
sparseValues: { indices: [0, 1], values: [0.236, 0.34] }, // Optional; for hybrid search
},
{
id: '2',
values: [0.685, 0.111, 0.857],
sparseValues: { indices: [0, 1], values: [0.345, 0.98] }, // Optional; for hybrid search
},
];
// Upsert the data into your index
await index.upsert(records);
When experimenting with data operations, it's sometimes helpful to know how many records are stored in each namespace. In that case,
target the index and use the describeIndexStats()
command.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const index = pc.index('example-index');
await index.describeIndexStats();
// {
// namespaces: {
// '': { recordCount: 10 }
// foo: { recordCount: 2000 },
// bar: { recordCount: 2000 }
// },
// dimension: 1536,
// indexFullness: 0,
// totalRecordCount: 4010
// }
The query method accepts a large number of options. The dimension of the query vector must match the dimension of your index.
type QueryOptions = {
topK: number; // number of results desired
vector?: Array<number>; // must match dimension of index
sparseVector?: {
indices: Array<integer>; // indices must fall within index dimension
values: Array<number>; // indices and values arrays must have same length
};
id?: string;
includeMetadata: boolean;
includeValues: boolean;
};
For example, to query by vector values you would pass the vector
param in the options configuration. For brevity sake this example query vector is tiny (dimension 2), but in a more realistic use case this query vector would be an embedding outputted by a model. Look at the Example code to see more realistic examples of how to use query
.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const index = pc.index('my-index');
await index.query({ topK: 3, vector: [0.22, 0.66] });
// {
// matches: [
// {
// id: '556',
// score: 1.00000012,
// values: [],
// sparseValues: undefined,
// metadata: undefined
// },
// {
// id: '137',
// score: 1.00000012,
// values: [],
// sparseValues: undefined,
// metadata: undefined
// },
// {
// id: '129',
// score: 1.00000012,
// values: [],
// sparseValues: undefined,
// metadata: undefined
// }
// ],
// namespace: '',
// usage: {
// readUnits: 5
// }
// }
You include options to includeMetadata: true
or includeValues: true
if you need this information. By default,
these are not returned to keep the response payload small.
Remember that data operations take place within the context of a namespace
, so if you are working with namespaces and do not see expected results you should check that you are targeting the correct namespace with your query.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
// Target the index and namespace
const index = pc.index('my-index').namespace('my-namespace');
const results = await index.query({ topK: 3, vector: [0.22, 0.66] });
You can query using the vector values of an existing record in the index by passing a record id.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const index = pc.index('my-index');
const results = await index.query({ topK: 10, id: '1' });
If you are working with sparse-dense vectors, you can add sparse vector values to perform a hybrid search.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
await pc.createIndex({
name: 'hyrbid-image-search',
metric: 'dotproduct',
dimension: 512,
spec: {
pod: {
environment: 'us-west4-gcp',
pods: 1,
podType: 's1.x1',
}
},
waitUntilReady: true
});
const index = pc.index('hybrid-image-search');
// Create some vector embeddings using your model of choice.
const records = [...]
// Upsert data
await index.upsert(records)
// Prepare query values. In a more realistic example, these would both come out of a model.
const vector = [
// The dimension of this index needs to match the index dimension.
// Pretend this is a 512 dimension vector.
]
const sparseVector = {
indices: [23, 399, 251, 17],
values: [ 0.221, 0.967, 0.016, 0.572]
}
// Execute the query
const results = await index.query({ topK: 10, vector, sparseVector, includeMetadata: true })
You may want to update vector values
, sparseValues
, or metadata
. Specify the id and the attribute value you want to update.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const index = pc.index('imdb-movies');
await index.update({
id: '18593',
metadata: { genre: 'romance' },
});
The listPaginated
method can be used to list record ids matching a particular id prefix in a paginated format. With clever assignment
of record ids, this can be used to help model hierarchical relationships between different records such as when there are embeddings for multiple chunks or fragments related to the same document.
const pc = new Pinecone();
const index = pc.index('my-index').namespace('my-namespace');
const results = await index.listPaginated({ prefix: 'doc1#' });
console.log(results);
// {
// vectors: [
// { id: 'doc1#01' }, { id: 'doc1#02' }, { id: 'doc1#03' },
// { id: 'doc1#04' }, { id: 'doc1#05' }, { id: 'doc1#06' },
// { id: 'doc1#07' }, { id: 'doc1#08' }, { id: 'doc1#09' },
// ...
// ],
// pagination: {
// next: 'eyJza2lwX3Bhc3QiOiJwcmVUZXN0LS04MCIsInByZWZpeCI6InByZVRlc3QifQ=='
// },
// namespace: 'my-namespace',
// usage: { readUnits: 1 }
// }
// Fetch the next page of results
await index.listPaginated({
prefix: 'doc1#',
paginationToken: results.pagination?.next,
});
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const index = pc.index('my-index');
const fetchResult = await index.fetch(['id-1', 'id-2']);
For convenience there are several delete-related methods. You can verify the results of a delete operation by trying to fetch()
a record or looking at the index summary with describeIndexStats()
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const index = pc.index('my-index');
await index.deleteOne('id-to-delete');
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const index = pc.index('my-index');
await index.deleteMany(['id-1', 'id-2', 'id-3']);
Note: deletion by metadata filter only applies to pod-based indexes.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const index = pc.index('albums-database');
await index.deleteMany({ genre: 'rock' });
ℹ️ NOTE
Indexes in the gcp-starter environment do not support namespaces.
To nuke everything in the targeted namespace, use the deleteAll
method.
import { Pinecone } from '@pinecone-database/pinecone';
const pc = new Pinecone();
const index = pc.index('my-index');
await index.namespace('foo-namespace').deleteAll();
If you do not specify a namespace, the records in the default namespace ''
will be deleted.
Interact with Pinecone's Inference API (currently in public preview). The Pinecone Inference API is a service that gives you access to embedding models hosted on Pinecone's infrastructure. Read more at Understanding Pinecone Inference.
Notes:
Models currently supported:
Send text to Pinecone's Inference API to generate embeddings for documents and queries.
import { Pinecone } from '@pinecone-database/pinecone';
const client = new Pinecone({ apiKey: '<Your API key from app.pinecone.io>' });
const embeddingModel = 'multilingual-e5-large';
const documents = [
'Turkey is a classic meat to eat at American Thanksgiving.',
'Many people enjoy the beautiful mosques in Turkey.',
];
const docParameters = {
inputType: 'passage',
truncate: 'END',
};
async function generateDocEmbeddings() {
try {
return await client.inference.embed(
embeddingModel,
documents,
docParameters
);
} catch (error) {
console.error('Error generating embeddings:', error);
}
}
generateDocEmbeddings().then((embeddingsResponse) => {
if (embeddingsResponse) {
console.log(embeddingsResponse);
}
});
// << Upsert documents into Pinecone >>
const userQuery = ['How should I prepare my turkey?'];
const queryParameters = {
inputType: 'query',
truncate: 'END',
};
async function generateQueryEmbeddings() {
try {
return await client.inference.embed(
embeddingModel,
userQuery,
queryParameters
);
} catch (error) {
console.error('Error generating embeddings:', error);
}
}
generateQueryEmbeddings().then((embeddingsResponse) => {
if (embeddingsResponse) {
console.log(embeddingsResponse);
}
});
// << Send query to Pinecone to retrieve similar documents >>
For Tasks:
Click tags to check more tools for each tasksFor Jobs:
Alternative AI tools for pinecone-ts-client
Similar Open Source Tools
pinecone-ts-client
The official Node.js client for Pinecone, written in TypeScript. This client library provides a high-level interface for interacting with the Pinecone vector database service. With this client, you can create and manage indexes, upsert and query vector data, and perform other operations related to vector search and retrieval. The client is designed to be easy to use and provides a consistent and idiomatic experience for Node.js developers. It supports all the features and functionality of the Pinecone API, making it a comprehensive solution for building vector-powered applications in Node.js.
client-js
The Mistral JavaScript client is a library that allows you to interact with the Mistral AI API. With this client, you can perform various tasks such as listing models, chatting with streaming, chatting without streaming, and generating embeddings. To use the client, you can install it in your project using npm and then set up the client with your API key. Once the client is set up, you can use it to perform the desired tasks. For example, you can use the client to chat with a model by providing a list of messages. The client will then return the response from the model. You can also use the client to generate embeddings for a given input. The embeddings can then be used for various downstream tasks such as clustering or classification.
model.nvim
model.nvim is a tool designed for Neovim users who want to utilize AI models for completions or chat within their text editor. It allows users to build prompts programmatically with Lua, customize prompts, experiment with multiple providers, and use both hosted and local models. The tool supports features like provider agnosticism, programmatic prompts in Lua, async and multistep prompts, streaming completions, and chat functionality in 'mchat' filetype buffer. Users can customize prompts, manage responses, and context, and utilize various providers like OpenAI ChatGPT, Google PaLM, llama.cpp, ollama, and more. The tool also supports treesitter highlights and folds for chat buffers.
deepgram-js-sdk
Deepgram JavaScript SDK. Power your apps with world-class speech and Language AI models.
llm-scraper
LLM Scraper is a TypeScript library that allows you to convert any webpages into structured data using LLMs. It supports Local (GGUF), OpenAI, Groq chat models, and schemas defined with Zod. With full type-safety in TypeScript and based on the Playwright framework, it offers streaming when crawling multiple pages and supports four input modes: html, markdown, text, and image.
llm-client
LLMClient is a JavaScript/TypeScript library that simplifies working with large language models (LLMs) by providing an easy-to-use interface for building and composing efficient prompts using prompt signatures. These signatures enable the automatic generation of typed prompts, allowing developers to leverage advanced capabilities like reasoning, function calling, RAG, ReAcT, and Chain of Thought. The library supports various LLMs and vector databases, making it a versatile tool for a wide range of applications.
ax
Ax is a Typescript library that allows users to build intelligent agents inspired by agentic workflows and the Stanford DSP paper. It seamlessly integrates with multiple Large Language Models (LLMs) and VectorDBs to create RAG pipelines or collaborative agents capable of solving complex problems. The library offers advanced features such as streaming validation, multi-modal DSP, and automatic prompt tuning using optimizers. Users can easily convert documents of any format to text, perform smart chunking, embedding, and querying, and ensure output validation while streaming. Ax is production-ready, written in Typescript, and has zero dependencies.
LlamaIndexTS
LlamaIndex.TS is a data framework for your LLM application. Use your own data with large language models (LLMs, OpenAI ChatGPT and others) in Typescript and Javascript.
instructor
Instructor is a popular Python library for managing structured outputs from large language models (LLMs). It offers a user-friendly API for validation, retries, and streaming responses. With support for various LLM providers and multiple languages, Instructor simplifies working with LLM outputs. The library includes features like response models, retry management, validation, streaming support, and flexible backends. It also provides hooks for logging and monitoring LLM interactions, and supports integration with Anthropic, Cohere, Gemini, Litellm, and Google AI models. Instructor facilitates tasks such as extracting user data from natural language, creating fine-tuned models, managing uploaded files, and monitoring usage of OpenAI models.
lmstudio.js
lmstudio.js is a pre-release alpha client SDK for LM Studio, allowing users to use local LLMs in JS/TS/Node. It is currently undergoing rapid development with breaking changes expected. Users can follow LM Studio's announcements on Twitter and Discord. The SDK provides API usage for loading models, predicting text, setting up the local LLM server, and more. It supports features like custom loading progress tracking, model unloading, structured output prediction, and cancellation of predictions. Users can interact with LM Studio through the CLI tool 'lms' and perform tasks like text completion, conversation, and getting prediction statistics.
react-native-vercel-ai
Run Vercel AI package on React Native, Expo, Web and Universal apps. Currently React Native fetch API does not support streaming which is used as a default on Vercel AI. This package enables you to use AI library on React Native but the best usage is when used on Expo universal native apps. On mobile you get back responses without streaming with the same API of `useChat` and `useCompletion` and on web it will fallback to `ai/react`
mini.ai
This plugin extends and creates `a`/`i` textobjects in Neovim. It enhances some builtin textobjects (like `a(`, `a)`, `a'`, and more), creates new ones (like `a*`, `a
dexter
Dexter is a set of mature LLM tools used in production at Dexa, with a focus on real-world RAG (Retrieval Augmented Generation). It is a production-quality RAG that is extremely fast and minimal, and handles caching, throttling, and batching for ingesting large datasets. It also supports optional hybrid search with SPLADE embeddings, and is a minimal TS package with full typing that uses `fetch` everywhere and supports Node.js 18+, Deno, Cloudflare Workers, Vercel edge functions, etc. Dexter has full docs and includes examples for basic usage, caching, Redis caching, AI function, AI runner, and chatbot.
python-tgpt
Python-tgpt is a Python package that enables seamless interaction with over 45 free LLM providers without requiring an API key. It also provides image generation capabilities. The name _python-tgpt_ draws inspiration from its parent project tgpt, which operates on Golang. Through this Python adaptation, users can effortlessly engage with a number of free LLMs available, fostering a smoother AI interaction experience.
strictjson
Strict JSON is a framework designed to handle JSON outputs with complex structures, fixing issues that standard json.loads() cannot resolve. It provides functionalities for parsing LLM outputs into dictionaries, supporting various data types, type forcing, and error correction. The tool allows easy integration with OpenAI JSON Mode and offers community support through tutorials and discussions. Users can download the package via pip, set up API keys, and import functions for usage. The tool works by extracting JSON values using regex, matching output values to literals, and ensuring all JSON fields are output by LLM with optional type checking. It also supports LLM-based checks for type enforcement and error correction loops.
hezar
Hezar is an all-in-one AI library designed specifically for the Persian community. It brings together various AI models and tools, making it easy to use AI with just a few lines of code. The library seamlessly integrates with Hugging Face Hub, offering a developer-friendly interface and task-based model interface. In addition to models, Hezar provides tools like word embeddings, tokenizers, feature extractors, and more. It also includes supplementary ML tools for deployment, benchmarking, and optimization.
For similar tasks
pinecone-ts-client
The official Node.js client for Pinecone, written in TypeScript. This client library provides a high-level interface for interacting with the Pinecone vector database service. With this client, you can create and manage indexes, upsert and query vector data, and perform other operations related to vector search and retrieval. The client is designed to be easy to use and provides a consistent and idiomatic experience for Node.js developers. It supports all the features and functionality of the Pinecone API, making it a comprehensive solution for building vector-powered applications in Node.js.
azure-search-vector-samples
This repository provides code samples in Python, C#, REST, and JavaScript for vector support in Azure AI Search. It includes demos for various languages showcasing vectorization of data, creating indexes, and querying vector data. Additionally, it offers tools like Azure AI Search Lab for experimenting with AI-enabled search scenarios in Azure and templates for deploying custom chat-with-your-data solutions. The repository also features documentation on vector search, hybrid search, creating and querying vector indexes, and REST API references for Azure AI Search and Azure OpenAI Service.
venice
Venice is a derived data storage platform, providing the following characteristics: 1. High throughput asynchronous ingestion from batch and streaming sources (e.g. Hadoop and Samza). 2. Low latency online reads via remote queries or in-process caching. 3. Active-active replication between regions with CRDT-based conflict resolution. 4. Multi-cluster support within each region with operator-driven cluster assignment. 5. Multi-tenancy, horizontal scalability and elasticity within each cluster. The above makes Venice particularly suitable as the stateful component backing a Feature Store, such as Feathr. AI applications feed the output of their ML training jobs into Venice and then query the data for use during online inference workloads.
honey
Bee is an ORM framework that provides easy and high-efficiency database operations, allowing developers to focus on business logic development. It supports various databases and features like automatic filtering, partial field queries, pagination, and JSON format results. Bee also offers advanced functionalities like sharding, transactions, complex queries, and MongoDB ORM. The tool is designed for rapid application development in Java, offering faster development for Java Web and Spring Cloud microservices. The Enterprise Edition provides additional features like financial computing support, automatic value insertion, desensitization, dictionary value conversion, multi-tenancy, and more.
llama_index
LlamaIndex is a data framework for building LLM applications. It provides tools for ingesting, structuring, and querying data, as well as integrating with LLMs and other tools. LlamaIndex is designed to be easy to use for both beginner and advanced users, and it provides a comprehensive set of features for building LLM applications.
kernel-memory
Kernel Memory (KM) is a multi-modal AI Service specialized in the efficient indexing of datasets through custom continuous data hybrid pipelines, with support for Retrieval Augmented Generation (RAG), synthetic memory, prompt engineering, and custom semantic memory processing. KM is available as a Web Service, as a Docker container, a Plugin for ChatGPT/Copilot/Semantic Kernel, and as a .NET library for embedded applications. Utilizing advanced embeddings and LLMs, the system enables Natural Language querying for obtaining answers from the indexed data, complete with citations and links to the original sources. Designed for seamless integration as a Plugin with Semantic Kernel, Microsoft Copilot and ChatGPT, Kernel Memory enhances data-driven features in applications built for most popular AI platforms.
deeplake
Deep Lake is a Database for AI powered by a storage format optimized for deep-learning applications. Deep Lake can be used for: 1. Storing data and vectors while building LLM applications 2. Managing datasets while training deep learning models Deep Lake simplifies the deployment of enterprise-grade LLM-based products by offering storage for all data types (embeddings, audio, text, videos, images, pdfs, annotations, etc.), querying and vector search, data streaming while training models at scale, data versioning and lineage, and integrations with popular tools such as LangChain, LlamaIndex, Weights & Biases, and many more. Deep Lake works with data of any size, it is serverless, and it enables you to store all of your data in your own cloud and in one place. Deep Lake is used by Intel, Bayer Radiology, Matterport, ZERO Systems, Red Cross, Yale, & Oxford.
databend
Databend is an open-source cloud data warehouse that serves as a cost-effective alternative to Snowflake. With its focus on fast query execution and data ingestion, it's designed for complex analysis of the world's largest datasets.
For similar jobs
resonance
Resonance is a framework designed to facilitate interoperability and messaging between services in your infrastructure and beyond. It provides AI capabilities and takes full advantage of asynchronous PHP, built on top of Swoole. With Resonance, you can: * Chat with Open-Source LLMs: Create prompt controllers to directly answer user's prompts. LLM takes care of determining user's intention, so you can focus on taking appropriate action. * Asynchronous Where it Matters: Respond asynchronously to incoming RPC or WebSocket messages (or both combined) with little overhead. You can set up all the asynchronous features using attributes. No elaborate configuration is needed. * Simple Things Remain Simple: Writing HTTP controllers is similar to how it's done in the synchronous code. Controllers have new exciting features that take advantage of the asynchronous environment. * Consistency is Key: You can keep the same approach to writing software no matter the size of your project. There are no growing central configuration files or service dependencies registries. Every relation between code modules is local to those modules. * Promises in PHP: Resonance provides a partial implementation of Promise/A+ spec to handle various asynchronous tasks. * GraphQL Out of the Box: You can build elaborate GraphQL schemas by using just the PHP attributes. Resonance takes care of reusing SQL queries and optimizing the resources' usage. All fields can be resolved asynchronously.
aiogram_bot_template
Aiogram bot template is a boilerplate for creating Telegram bots using Aiogram framework. It provides a solid foundation for building robust and scalable bots with a focus on code organization, database integration, and localization.
pluto
Pluto is a development tool dedicated to helping developers **build cloud and AI applications more conveniently** , resolving issues such as the challenging deployment of AI applications and open-source models. Developers are able to write applications in familiar programming languages like **Python and TypeScript** , **directly defining and utilizing the cloud resources necessary for the application within their code base** , such as AWS SageMaker, DynamoDB, and more. Pluto automatically deduces the infrastructure resource needs of the app through **static program analysis** and proceeds to create these resources on the specified cloud platform, **simplifying the resources creation and application deployment process**.
pinecone-ts-client
The official Node.js client for Pinecone, written in TypeScript. This client library provides a high-level interface for interacting with the Pinecone vector database service. With this client, you can create and manage indexes, upsert and query vector data, and perform other operations related to vector search and retrieval. The client is designed to be easy to use and provides a consistent and idiomatic experience for Node.js developers. It supports all the features and functionality of the Pinecone API, making it a comprehensive solution for building vector-powered applications in Node.js.
aiohttp-pydantic
Aiohttp pydantic is an aiohttp view to easily parse and validate requests. You define using function annotations what your methods for handling HTTP verbs expect, and Aiohttp pydantic parses the HTTP request for you, validates the data, and injects the parameters you want. It provides features like query string, request body, URL path, and HTTP headers validation, as well as Open API Specification generation.
gcloud-aio
This repository contains shared codebase for two projects: gcloud-aio and gcloud-rest. gcloud-aio is built for Python 3's asyncio, while gcloud-rest is a threadsafe requests-based implementation. It provides clients for Google Cloud services like Auth, BigQuery, Datastore, KMS, PubSub, Storage, and Task Queue. Users can install the library using pip and refer to the documentation for usage details. Developers can contribute to the project by following the contribution guide.
aioconsole
aioconsole is a Python package that provides asynchronous console and interfaces for asyncio. It offers asynchronous equivalents to input, print, exec, and code.interact, an interactive loop running the asynchronous Python console, customization and running of command line interfaces using argparse, stream support to serve interfaces instead of using standard streams, and the apython script to access asyncio code at runtime without modifying the sources. The package requires Python version 3.8 or higher and can be installed from PyPI or GitHub. It allows users to run Python files or modules with a modified asyncio policy, replacing the default event loop with an interactive loop. aioconsole is useful for scenarios where users need to interact with asyncio code in a console environment.
aiosqlite
aiosqlite is a Python library that provides a friendly, async interface to SQLite databases. It replicates the standard sqlite3 module but with async versions of all the standard connection and cursor methods, along with context managers for automatically closing connections and cursors. It allows interaction with SQLite databases on the main AsyncIO event loop without blocking execution of other coroutines while waiting for queries or data fetches. The library also replicates most of the advanced features of sqlite3, such as row factories and total changes tracking.