vector-search-class-notes
Class notes for the course "Long Term Memory in AI - Vector Search and Databases" COS 597A @ Princeton Fall 2023
Stars: 316
The 'vector-search-class-notes' repository contains class materials for a course on Long Term Memory in AI, focusing on vector search and databases. The course covers theoretical foundations and practical implementation of vector search applications, algorithms, and systems. It explores the intersection of Artificial Intelligence and Database Management Systems, with topics including text embeddings, image embeddings, low dimensional vector search, dimensionality reduction, approximate nearest neighbor search, clustering, quantization, and graph-based indexes. The repository also includes information on the course syllabus, project details, selected literature, and contributions from industry experts in the field.
README:
NOTE: COS 597A class times changed for Fall semester 2023. Classes will be held 9am-12noon.
-
Edo Liberty, Founder and CEO of Pinecone, the world's leading Vector Database. Publications.
-
Matthijs Douze, Research Scientist at Meta. Architect and main developer of FAISS the most popular and advanced open source library for vector search. Publications.
-
Teaching assistant Nataly Brukhim PhD sdudent working with Prof. Elad Hazan and researcher at Google AI Princeton. email: [email protected]. Publications.
-
Guest lecture by Harsha Vardhan Simhadri Senior Principal Researcher, at Microsoft Research. Creator of DiskANN. Publications
Long Term Memory is a foundational capability in the modern AI Stack. At their core, these systems use vector search. Vector search is also a basic tool for systems that manipulate large collections of media like search engines, knowledge bases, content moderation tools, recommendation systems, etc. As such, the discipline lays at the intersection of Artificial Intelligence and Database Management Systems. This course will cover the theoretical foundations and practical implementation of vector search applications, algorithms, and systems. The course will be evaluated with project and in-class presentation.
All class materials are intended to be used freely by academics anywhere, students and professors alike. Please contribute in the form of pull requests or by opening issues.
https://github.com/edoliberty/vector-search-class-notes
On unix-like systems (e.g. macos) with bibtex and pdflatex available you should be able to run this:
git clone [email protected]:edoliberty/vector-search-class-notes.git
cd vector-search-class-notes
./build
-
9/8 - Class 1 - Introduction to Vector Search [Matthijs + Edo + Nataly]
- Intro to the course: Topic, Schedule, Project, Grading, ...
- Embeddings as an information bottleneck. Instead of learning end-to-end, use embeddings as an intermediate representation
- Advantages: scalability, instant updates, and explainability
- Typical volumes of data and scalability. Embeddings are the only way to manage / access large databases
- The embedding contract: the embedding extractor and embedding indexer agree on the meaning of the distance. Separation of concerns.
- The vector space model in information retrieval
- Vector embeddings in machine learning
- Define vector, vector search, ranking, retrieval, recall
-
9/15 - Class 2 - Text embeddings [Matthijs]
- 2-layer word embeddings. Word2vec and fastText, obtained via a factorization of a co-occurrence matrix. Embedding arithmetic: king + woman - man = queen, (already based on similarity search)
- Sentence embeddings: How to train, masked LM. Properties of sentence embeddings.
- Large Language Models: reasoning as an emerging property of a LM. What happens when the training set = the whole web
-
9/22 - Class 3 - Image embeddings [Matthijs]
- Pixel structures of images. Early works on direct pixel indexing
- Traditional CV models. Global descriptors (GIST). Local descriptors (SIFT and friends)Direct indexing of local descriptors for image matching, local descriptor pooling (Fisher, VLAD)
- Convolutional Neural Nets. Off-the-shelf models. Trained specifically (contrastive learning, self-supervised learning)
- Modern Computer Vision models
-
9/29 - Class 4 - Low Dimensional Vector Search [Edo]
- Vector search problem definition
- k-d tree, space partitioning data structures
- Worst case proof for kd-trees
- Probabilistic inequalities. Recap of basic inequalities: Markov, Chernoof, Hoeffding
- Concentration Of Measure phenomena. Orthogonality of random vectors in high dimensions
- Curse of dimensionality and the failure of space partitioning
-
10/6 - Class 5 - Dimensionality Reduction [Edo]
- Singular Value Decomposition (SVD)
- Applications of the SVD
- Rank-k approximation in the spectral norm
- Rank-k approximation in the Frobenius norm
- Linear regression in the least-squared loss
- PCA, Optimal squared loss dimension reduction
- Closest orthogonal matrix
- Computing the SVD: The power method
- Random-projection
- Matrices with normally distributed independent entries
- Fast Random Projections
-
10/13 - No Class - Midterm Examination Week
-
10/20 - No Class - Fall Recess
-
10/27 - Class 6 - Approximate Nearest Neighbor Search [Edo]
- Definition of Approximate Nearest Neighbor Search (ANNS)
- Criteria: Speed / accuracy / memory usage / updateability / index construction time
- Definition of Locality Sensitive Hashing and examples
- The LSH Algorithm
- LSH Analysis, proof of correctness, and asymptotics
-
11/3 - Class 7 - Clustering [Edo]
- K-means clustering - mean squared error criterion.
- Lloyd’s Algorithm
- k-means and PCA
- ε-net argument for fixed dimensions
- Sampling based seeding for k-means
- k-means++
- The Inverted File Model (IVF)
-
11/10 - Class 8 - Quantization for lossy vector compression This class will take place remotely via zoom, see the edstem message to get the link [Matthijs]
- Python notebook corresponding to the class: Class_08_runbook_for_students.ipynb
- Vector quantization is a topline (directly optimizes the objective)
- Binary quantization and hamming comparison
- Product quantization. Chunked vector quantization. Optimized vector quantization
- Additive quantization. Extension of product quantization. Difficulty in training approximations (Residual quantization, CQ, TQ, LSQ, etc.)
- Cost of coarse quantization vs. inverted list scanning
-
11/17 - Class 9 - Graph based indexes by Guest lecturer Harsha Vardhan Simhadri.
- Early works: hierarchical k-means
- Neighborhood graphs. How to construct them. Nearest Neighbor Descent
- Greedy search in Neighborhood graphs. That does not work -- need long jumps
- HNSW. A practical hierarchical graph-based index
- NSG. Evolving a graph k-NN graph
-
11/24 - No Class - Thanksgiving Recess
-
12/1 - Class 10 - Student project and paper presentations [Edo + Nataly]
Class work includes a final project. It will be graded based on
- 50% - Project submission
- 50% - In-class presentation
Projects can be in three different flavors
- Theory/Research: propose a new algorithm for a problem we explored in class (or modify an existing one), explain what it achieves, give experimental evidence or a proof for its behavior. If you choose this kind of project you are expected to submit a write up.
- Data Science/AI: create an interesting use case for vector search using Pinecone, explain what data you used, what value your application brings, and what insights you gained. If you choose this kind of project you are expected to submit code (e.g. Jupyter Notebooks) and a writeup of your results and insights.
- Engineering/HPC: adapt or add to FAISS, explain your improvements, show experimental results. If you choose this kind of project you are expected to submit a branch of FAISS for review along with a short writeup of your suggested improvement and experiments.
Project schedule
- 11/24 - One-page project proposal approved by the instructors
- 12/1 - Final project submission
- 12/1 - In-class presentation
Some more details
- Project Instructor: Nataly [email protected]
- Projects can be worked on individually, in teams of two or at most three students.
- Expect to spend a few hours over the semester on the project proposal. Try to get it approved well ahead of the deadline.
- Expect to spent 3-5 full days on the project itself (on par with preparing for a final exam)
- In class project project presentation are 5 minutes per student (teams of two students present for 10 minutes. Teams of three, 15 minutes).
- A fast random sampling algorithm for sparsifying matrices - Arora, Sanjeev and Hazan, Elad and Kale, Satyen - 2006
- A Randomized Algorithm for Principal Component Analysis - Vladimir Rokhlin and Arthur Szlam and Mark Tygert - 2009
- A search structure based on kd trees for efficient ray tracing - Subramanian, KR and Fussel, DS - 1990
- A Short Proof for Gap Independence of Simultaneous Iteration - Edo Liberty - 2016
- Accelerating Large-Scale Inference with Anisotropic Vector Quantization - Ruiqi Guo and Philip Sun and Erik Lindgren and Quan Geng and David Simcha and Felix Chern and Sanjiv Kumar - 2020
- Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada - 2015
- An Algorithm for Online K-Means Clustering - Edo Liberty and Ram Sriharsha and Maxim Sviridenko
- An Almost Optimal Unrestricted Fast Johnson-Lindenstrauss Transform - Nir Ailon and Edo Liberty - 2011
- An elementary proof of the Johnson-Lindenstrauss lemma - S. DasGupta and A. Gupta - 1999
- Approximate nearest neighbors and the fast Johnson-Lindenstrauss transform - Nir Ailon and Bernard Chazelle - 2006
- Billion-scale similarity search with GPUs - Jeff Johnson and Matthijs Douze and Herv{'e} J{'e}gou - 2017
- Clustering Data Streams: Theory and Practice - Sudipto Guha and Adam Meyerson and Nina Mishra and Rajeev Motwani and Liadan O'Callaghan - 2003
- DiskANN: Fast Accurate Billion-point Nearest Neighbor Search on a Single Node - Jayaram Subramanya, Suhas and Devvrit, Fnu and Simhadri, Harsha Vardhan and Krishnawamy, Ravishankar and Kadekodi, Rohan - 2019
- Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs - Yu. A. Malkov and D. A. Yashunin - 2018
- Efficient K-Nearest Neighbor Graph Construction for Generic Similarity Measures - Dong, Wei and Moses, Charikar and Li, Kai - 2011
- Even Simpler Deterministic Matrix Sketching - Edo Liberty - 2022
- Extensions of Lipschitz mappings into a Hilbert space - W. B. Johnson and J. Lindenstrauss - 1984
- Fast Approximate Nearest Neighbor Search With The Navigating Spreading-out Graph - Cong Fu and Chao Xiang and Changxu Wang and Deng Cai - 2018
- Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions - Halko, N. and Martinsson, P. G. and Tropp, J. A. - 2011
- Invertibility of random matrices: norm of the inverse - Mark Rudelson - 2008
- K-means clustering via principal component analysis - Chris H. Q. Ding and Xiaofeng He - 2004
- k-means++: the advantages of careful seeding - David Arthur and Sergei Vassilvitskii - 2007
- Least squares quantization in pcm - Stuart P. Lloyd - 1982
- LSQ++: Lower Running Time and Higher Recall in Multi-Codebook Quantization - Martinez, Julieta and Zakhmi, Shobhit and Hoos, Holger H. and Little, James J. - 2018
- Multidimensional binary search trees used for associative searching - Bentley, Jon Louis - 1975
- Near-Optimal Entrywise Sampling for Data Matrices - Achlioptas, Dimitris and Karnin, Zohar S and Liberty, Edo - 2013
- Pass Efficient Algorithms for Approximating Large Matrices - Petros Drineas and Ravi Kannan - 2003
- Product Quantization for Nearest Neighbor Search - Jegou, Herve and Douze, Matthijs and Schmid, Cordelia - 2011
- QuickCSG: Arbitrary and faster boolean combinations of n solids - Douze, Matthijs and Franco, Jean-S{'e}bastien and Raffin, Bruno - 2015
- Quicker {ADC} : Unlocking the Hidden Potential of Product Quantization With {SIMD - Fabien Andre and Anne-Marie Kermarrec and Nicolas Le Scouarnec - 2021
- Random Projection Trees and Low Dimensional Manifolds - Dasgupta, Sanjoy and Freund, Yoav - 2008
- Randomized Algorithms for Low-Rank Matrix Factorizations: Sharp Performance Bounds - Witten, Rafi and Cand`{e}s, Emmanuel - 2015
- Randomized Block Krylov Methods for Stronger and Faster Approximate Singular Value Decomposition - Cameron Musco and Christopher Musco - 2015
- Revisiting Additive Quantization - Julieta Martinez and Joris Clement and Holger H. Hoos and J. Little - 2016
- Sampling from large matrices: An approach through geometric functional analysis - Rudelson, Mark and Vershynin, Roman - 2007
- Similarity estimation techniques from rounding algorithms - Moses Charikar - 2002
- Similarity Search in High Dimensions via Hashing - Aristides Gionis and Piotr Indyk and Rajeev Motwani - 1999
- Simple and Deterministic Matrix Sketching - Edo Liberty - 2012
- Smaller Coresets for k-Median and k-Means Clustering - S. {Har-Peled} and A. Kushal - 2005
- Sparser Johnson-Lindenstrauss transforms - Daniel M. Kane and Jelani Nelson - 2012
- Sparsity Lower Bounds for Dimensionality Reducing Maps - Jelani Nelson and Huy L. Nguyen - 2012
- Spectral Relaxation for K-means Clustering - Hongyuan Zha and Xiaofeng He and Chris H. Q. Ding and Ming Gu and Horst D. Simon - 2001
- Streaming k-means approximation - Nir Ailon and Ragesh Jaiswal and Claire Monteleoni - 2009
- Strong converse for identification via quantum channels - Rudolf Ahlswede and Andreas Winter - 2002
- Transformer Memory as a Differentiable Search Index - Yi Tay and Vinh Q. Tran and Mostafa Dehghani and Jianmo Ni and Dara Bahri and Harsh Mehta and Zhen Qin and Kai Hui and Zhe Zhao and Jai Gupta and Tal Schuster and William W. Cohen and Donald Metzler - 2022
- Unsupervised Neural Quantization for Compressed-Domain Similarity Search - S. Morozov and A. Babenko - 2019
- Worst-Case Analysis for Region and Partial Region Searches in Multidimensional Binary Search Trees and Balanced Quad Trees - Lee, D. T. and Wong, C. K. - 1977
- A Comprehensive Survey and Experimental Comparison of Graph-Based Approximate Nearest Neighbor Search - Mengzhao Wang and Xiaoliang Xu and Qiang Yue and Yuxiang Wang - 2021
- Approximate Nearest Neighbor Search on High Dimensional Data - Experiments, Analyses, and Improvement - Wen Li and Ying Zhang and Yifang Sun and Wei Wang and Mingjie Li and Wenjie Zhang and Xuemin Lin - 2020
- Survey of vector database management systems - James Jie Pan and Jianguo Wang and Guoliang Li - 2024
For Tasks:
Click tags to check more tools for each tasksFor Jobs:
Alternative AI tools for vector-search-class-notes
Similar Open Source Tools
vector-search-class-notes
The 'vector-search-class-notes' repository contains class materials for a course on Long Term Memory in AI, focusing on vector search and databases. The course covers theoretical foundations and practical implementation of vector search applications, algorithms, and systems. It explores the intersection of Artificial Intelligence and Database Management Systems, with topics including text embeddings, image embeddings, low dimensional vector search, dimensionality reduction, approximate nearest neighbor search, clustering, quantization, and graph-based indexes. The repository also includes information on the course syllabus, project details, selected literature, and contributions from industry experts in the field.
awesome-llms-fine-tuning
This repository is a curated collection of resources for fine-tuning Large Language Models (LLMs) like GPT, BERT, RoBERTa, and their variants. It includes tutorials, papers, tools, frameworks, and best practices to aid researchers, data scientists, and machine learning practitioners in adapting pre-trained models to specific tasks and domains. The resources cover a wide range of topics related to fine-tuning LLMs, providing valuable insights and guidelines to streamline the process and enhance model performance.
Awesome-LLM-Prune
This repository is dedicated to the pruning of large language models (LLMs). It aims to serve as a comprehensive resource for researchers and practitioners interested in the efficient reduction of model size while maintaining or enhancing performance. The repository contains various papers, summaries, and links related to different pruning approaches for LLMs, along with author information and publication details. It covers a wide range of topics such as structured pruning, unstructured pruning, semi-structured pruning, and benchmarking methods. Researchers and practitioners can explore different pruning techniques, understand their implications, and access relevant resources for further study and implementation.
Medical_Image_Analysis
The Medical_Image_Analysis repository focuses on X-ray image-based medical report generation using large language models. It provides pre-trained models and benchmarks for CheXpert Plus dataset, context sample retrieval for X-ray report generation, and pre-training on high-definition X-ray images. The goal is to enhance diagnostic accuracy and reduce patient wait times by improving X-ray report generation through advanced AI techniques.
nlp-llms-resources
The 'nlp-llms-resources' repository is a comprehensive resource list for Natural Language Processing (NLP) and Large Language Models (LLMs). It covers a wide range of topics including traditional NLP datasets, data acquisition, libraries for NLP, neural networks, sentiment analysis, optical character recognition, information extraction, semantics, topic modeling, multilingual NLP, domain-specific LLMs, vector databases, ethics, costing, books, courses, surveys, aggregators, newsletters, papers, conferences, and societies. The repository provides valuable information and resources for individuals interested in NLP and LLMs.
HEC-Commander
HEC-Commander Tools is a suite of python notebooks developed with AI assistance for water resource engineering workflows, providing automation for HEC-RAS and HEC-HMS through Jupyter Notebooks. It contains automation scripts for HEC-HMS, HEC-RAS, and DSS, along with miscellaneous tools. The repository also includes blog posts, ChatGPT assistants, and presentations related to H&H modeling and water resources workflows. Developed to support Region 4 of the Louisiana Watershed Initiative by Fenstermaker.
only_train_once
Only Train Once (OTO) is an automatic, architecture-agnostic DNN training and compression framework that allows users to train a general DNN from scratch or a pretrained checkpoint to achieve high performance and slimmer architecture simultaneously in a one-shot manner without fine-tuning. The framework includes features for automatic structured pruning and erasing operators, as well as hybrid structured sparse optimizers for efficient model compression. OTO provides tools for pruning zero-invariant group partitioning, constructing pruned models, and visualizing pruning and erasing dependency graphs. It supports the HESSO optimizer and offers a sanity check for compliance testing on various DNNs. The repository also includes publications, installation instructions, quick start guides, and a roadmap for future enhancements and collaborations.
awesome-transformer-nlp
This repository contains a hand-curated list of great machine (deep) learning resources for Natural Language Processing (NLP) with a focus on Generative Pre-trained Transformer (GPT), Bidirectional Encoder Representations from Transformers (BERT), attention mechanism, Transformer architectures/networks, Chatbot, and transfer learning in NLP.
awesome-llm-json
This repository is an awesome list dedicated to resources for using Large Language Models (LLMs) to generate JSON or other structured outputs. It includes terminology explanations, hosted and local models, Python libraries, blog articles, videos, Jupyter notebooks, and leaderboards related to LLMs and JSON generation. The repository covers various aspects such as function calling, JSON mode, guided generation, and tool usage with different providers and models.
awesome-RK3588
RK3588 is a flagship 8K SoC chip by Rockchip, integrating Cortex-A76 and Cortex-A55 cores with NEON coprocessor for 8K video codec. This repository curates resources for developing with RK3588, including official resources, RKNN models, projects, development boards, documentation, tools, and sample code.
animal-ai
Animal-Artificial Intelligence (Animal-AI) is an interdisciplinary research platform designed to understand human, animal, and artificial cognition. It supports AI research to unlock cognitive capabilities and explore the space of possible minds. The open-source project facilitates testing across animals, humans, and AI, providing a comprehensive AI environment with a library of 900 tasks. It offers compatibility with Windows, Linux, and macOS, supporting Python 3.6.x and above. The environment utilizes Unity3D Game Engine, Unity ML-Agents toolkit, and provides interactive elements for AI training scenarios.
ChatLaw
ChatLaw is an open-source legal large language model tailored for Chinese legal scenarios. It aims to combine LLM and knowledge bases to provide solutions for legal scenarios. The models include ChatLaw-13B and ChatLaw-33B, trained on various legal texts to construct dialogue data. The project focuses on improving logical reasoning abilities and plans to train models with parameters exceeding 30B for better performance. The dataset consists of forum posts, news, legal texts, judicial interpretations, legal consultations, exam questions, and court judgments, cleaned and enhanced to create dialogue data. The tool is designed to assist in legal tasks requiring complex logical reasoning, with a focus on accuracy and reliability.
vision-llms-are-blind
This repository contains the code and data for the paper 'Vision Language Models Are Blind'. It explores the limitations of large language models with vision capabilities (VLMs) in performing basic visual tasks that are easy for humans. The repository presents benchmark results showcasing the poor performance of state-of-the-art VLMs on tasks like counting line intersections, identifying circles, letters, and shapes, and following color-coded paths. The research highlights the challenges faced by VLMs in understanding visual information accurately, drawing parallels to myopia and blindness in human vision.
AlphaFold3
AlphaFold3 is an implementation of the Alpha Fold 3 model in PyTorch for accurate structure prediction of biomolecular interactions. It includes modules for genetic diffusion and full model examples for forward pass computations. The tool allows users to generate random pair and single representations, operate on atomic coordinates, and perform structure predictions based on input tensors. The implementation also provides functionalities for training and evaluating the model.
OpenNARS-for-Applications
OpenNARS-for-Applications is an implementation of a Non-Axiomatic Reasoning System, a general-purpose reasoner that adapts under the Assumption of Insufficient Knowledge and Resources. The system combines the logic and conceptual ideas of OpenNARS, event handling and procedure learning capabilities of ANSNA and 20NAR1, and the control model from ALANN. It is written in C, offers improved reasoning performance, and has been compared with Reinforcement Learning and means-end reasoning approaches. The system has been used in real-world applications such as assisting first responders, real-time traffic surveillance, and experiments with autonomous robots. It has been developed with a pragmatic mindset focusing on effective implementation of existing theory.
For similar tasks
OpenAI
OpenAI is a Swift community-maintained implementation over OpenAI public API. It is a non-profit artificial intelligence research organization founded in San Francisco, California in 2015. OpenAI's mission is to ensure safe and responsible use of AI for civic good, economic growth, and other public benefits. The repository provides functionalities for text completions, chats, image generation, audio processing, edits, embeddings, models, moderations, utilities, and Combine extensions.
vector-search-class-notes
The 'vector-search-class-notes' repository contains class materials for a course on Long Term Memory in AI, focusing on vector search and databases. The course covers theoretical foundations and practical implementation of vector search applications, algorithms, and systems. It explores the intersection of Artificial Intelligence and Database Management Systems, with topics including text embeddings, image embeddings, low dimensional vector search, dimensionality reduction, approximate nearest neighbor search, clustering, quantization, and graph-based indexes. The repository also includes information on the course syllabus, project details, selected literature, and contributions from industry experts in the field.
For similar jobs
weave
Weave is a toolkit for developing Generative AI applications, built by Weights & Biases. With Weave, you can log and debug language model inputs, outputs, and traces; build rigorous, apples-to-apples evaluations for language model use cases; and organize all the information generated across the LLM workflow, from experimentation to evaluations to production. Weave aims to bring rigor, best-practices, and composability to the inherently experimental process of developing Generative AI software, without introducing cognitive overhead.
LLMStack
LLMStack is a no-code platform for building generative AI agents, workflows, and chatbots. It allows users to connect their own data, internal tools, and GPT-powered models without any coding experience. LLMStack can be deployed to the cloud or on-premise and can be accessed via HTTP API or triggered from Slack or Discord.
VisionCraft
The VisionCraft API is a free API for using over 100 different AI models. From images to sound.
kaito
Kaito is an operator that automates the AI/ML inference model deployment in a Kubernetes cluster. It manages large model files using container images, avoids tuning deployment parameters to fit GPU hardware by providing preset configurations, auto-provisions GPU nodes based on model requirements, and hosts large model images in the public Microsoft Container Registry (MCR) if the license allows. Using Kaito, the workflow of onboarding large AI inference models in Kubernetes is largely simplified.
PyRIT
PyRIT is an open access automation framework designed to empower security professionals and ML engineers to red team foundation models and their applications. It automates AI Red Teaming tasks to allow operators to focus on more complicated and time-consuming tasks and can also identify security harms such as misuse (e.g., malware generation, jailbreaking), and privacy harms (e.g., identity theft). The goal is to allow researchers to have a baseline of how well their model and entire inference pipeline is doing against different harm categories and to be able to compare that baseline to future iterations of their model. This allows them to have empirical data on how well their model is doing today, and detect any degradation of performance based on future improvements.
tabby
Tabby is a self-hosted AI coding assistant, offering an open-source and on-premises alternative to GitHub Copilot. It boasts several key features: * Self-contained, with no need for a DBMS or cloud service. * OpenAPI interface, easy to integrate with existing infrastructure (e.g Cloud IDE). * Supports consumer-grade GPUs.
spear
SPEAR (Simulator for Photorealistic Embodied AI Research) is a powerful tool for training embodied agents. It features 300 unique virtual indoor environments with 2,566 unique rooms and 17,234 unique objects that can be manipulated individually. Each environment is designed by a professional artist and features detailed geometry, photorealistic materials, and a unique floor plan and object layout. SPEAR is implemented as Unreal Engine assets and provides an OpenAI Gym interface for interacting with the environments via Python.
Magick
Magick is a groundbreaking visual AIDE (Artificial Intelligence Development Environment) for no-code data pipelines and multimodal agents. Magick can connect to other services and comes with nodes and templates well-suited for intelligent agents, chatbots, complex reasoning systems and realistic characters.