cia
Comprehensive open-source intelligence platform analyzing Swedish political activities using AI and data visualization. Tracks politicians, government institutions, and parliamentary data, offering detailed insights, performance metrics, and advanced analytics.
Stars: 160
CIA is a powerful open-source tool designed for data analysis and visualization. It provides a user-friendly interface for processing large datasets and generating insightful reports. With CIA, users can easily explore data, perform statistical analysis, and create interactive visualizations to communicate findings effectively. Whether you are a data scientist, analyst, or researcher, CIA offers a comprehensive set of features to streamline your data analysis workflow and uncover valuable insights.
README:
An independent, volunteer-driven OSINT platform monitoring Swedish political activity
The Citizen Intelligence Agency is a volunteer-driven, open-source intelligence (OSINT) project that provides comprehensive analysis of political activities in Sweden. Through advanced monitoring of key political figures and institutions, we deliver:
- 📊 Financial performance metrics
⚠️ Risk assessment analytics- 📈 Political trend analysis
- 🏆 Politician ranking system
- 📉 Performance comparisons
- 🔍 Transparency insights
Our initiative remains strictly independent and non-partisan, focused on fostering informed decision-making and enhancing democratic engagement.
Explore our comprehensive feature set including:
- 📊 Interactive dashboards
- 🏆 Political scoreboard systems
- 📈 Critical analytics tools
- 🔍 Transparency metrics
- ⚖️ Accountability measures
- 📱 Data-driven insights
- 🌐 Website: www.hack23.com
- 💼 LinkedIn: James Sörling
Our analysis is powered by authoritative Swedish government and international data sources:
| Source | Description |
|---|---|
| 🏛️ Swedish Parliament Open Data | Parliamentary members, committees, and official documents |
| 🗳️ Swedish Election Authority | Election data, political parties, and voting results |
| 🌍 World Bank Open Data | Global economic indicators and demographic data |
| 💹 Swedish Financial Management Authority | Government finances and economic trends |
[
- Project Documentation
- Features of the CIA Project This document showcases the extensive features, providing detailed screenshots of dashboards, scoreboards, and critical analytics. The focus is on transparency, accountability, and data-driven decision-making in Sweden’s political ecosystem.
- Project Architecture - Delve into the architecture of the Citizen Intelligence Agency. This overview provides a look at the enterprise context, system context, system containers, web application components, deployment strategy, and AWS account structure of the project.
- Entity Model - Explore our Entity Model which provides a detailed look at the entities in our system and their relationships. This page is particularly useful for understanding the data structure of our project.
- Api docs - Access the API documentation for the Citizen Intelligence Agency project. This documentation provides a detailed view of the various packages within the system, helping developers understand and work with the project's API.
- FinancialSecurityPlan.md - This financial plan provides a structured and cost-efficient deployment for your application infrastructure in the AWS eu-west-1 (Ireland) region. It integrates key components of scalability, security, and resilience to support critical workloads while maintaining budgetary control
- End-of-Life-Strategy.md Project End-of-Life (EOL) Strategy
Please follow the instructions in our SECURITY.md file for reporting security issues.
This document provides a high-level overview of the key technologies used within the Citizen Intelligence Agency (CIA) project. Each technology plays a vital role in supporting CIA’s goals for data analysis, security, and scalability within the political intelligence domain.
| Category | Technologies |
|---|---|
| Core Framework | Spring Framework |
| Security | Spring Security, Bouncy Castle |
| Data Access | Hibernate, JPA, PostgreSQL, JDBC |
| Transaction Management |
Narayana (Integrated with Spring JpaTransactionManager) |
| Data Auditing | Javers |
| Business Rules Engine | Drools |
| Messaging | ActiveMQ Artemis, Spring JMS |
| Web/UI Layer | Vaadin, Vaadin Sass Compiler, Vaadin Themes |
| Monitoring | JavaMelody, AWS SDK for CloudWatch |
| Testing | JUnit, Mockito, Spring Test, Selenium WebDriver |
| Utilities | Apache Commons, Google Guava, SLF4J, Logback, Jackson |
| Build & Dependency Management | Maven |
This stack comprises:
- Core Framework: The project uses Spring Framework to provide a foundation for dependency injection, component management, and service configuration across modules.
- Security: Spring Security manages authentication and authorization, complemented by Bouncy Castle for cryptographic operations.
- Data Access: A combination of Hibernate, JPA, and PostgreSQL supports robust ORM-based data persistence, with JDBC facilitating additional database connectivity needs.
- Transaction Management: The project uses Narayana as the transaction manager implementation, integrated with Spring’s JpaTransactionManager for distributed transaction support and ensuring transactional integrity.
- Data Auditing: Javers provides auditing and historical versioning, allowing for tracking and comparing changes to data over time.
- Business Rules Engine : Drools is integrated into the CIA project to enable a robust business rules engine.
- Messaging: ActiveMQ Artemis and Spring JMS enable asynchronous communication between application components, supporting distributed and event-driven designs.
- Web/UI Layer: Vaadin powers the UI with a server-driven architecture, providing components like Vaadin Themes and Sass Compiler for a rich, interactive frontend experience directly in Java.
- Monitoring: JavaMelody and AWS SDK for CloudWatch provide real-time application monitoring and logging capabilities, supporting both local and cloud environments.
- Testing: JUnit, Mockito, Spring Test and Selenium WebDriver are used extensively for unit, integration, system, browser and mock testing to ensure application reliability and robustness.
- Utilities: Apache Commons, Google Guava, SLF4J, and Logback offer utility functions and structured logging, enhancing application maintainability and monitoring.
- Build & Dependency Management: Maven handles project builds, dependency management, and plugin configurations, enabling smooth project management and modular builds.
This document provides a comprehensive summary of the AWS services utilized in the Citizen Intelligence Agency (CIA) project infrastructure, as defined by its CloudFormation template. These services work together to ensure a secure, resilient, and scalable deployment environment.
| Category | AWS Services | NIST CSF Function, Category & Subcategory | ISO 27001:2022 Control & Link |
|---|---|---|---|
| Networking and Security | - Amazon VPC: Configures a custom network environment with public/private subnets, route tables, NAT Gateway, Network ACLs (NACLs) for traffic control, and VPC Flow Logs. - VPC Endpoints: Enables private access to AWS services (e.g., S3, EC2, SSM, CloudWatch Logs). - AWS WAF: Protects against web attacks at the ALB layer. - AWS IAM: Manages role-based access control. - AWS KMS: Manages encryption for data at rest. |
Identify (ID): - Asset Management (ID.AM-2) Protect (PR): - Access Control (PR.AC-1, PR.AC-3, PR.AC-5) - Data Security (PR.DS-1, PR.DS-2) - Protective Technology (PR.PT-3) Detect (DE): - Security Continuous Monitoring (DE.CM-3) |
- A.8.1: Asset management - A.9.4.1: Access control policy - A.13.1.1: Network controls - A.13.1.3: Segregation in networks - A.18.1.5: Regulation and compliance (see ISO 27001) |
| Domain and SSL Management | - Amazon Route 53: Manages domain registration and DNS routing. - AWS Certificate Manager (ACM): Issues and manages SSL/TLS certificates. |
Protect (PR): - Data Security (PR.DS-5) Detect (DE): - Anomalies and Events (DE.AE-3) |
- A.10.1.1: Cryptographic controls for data protection - A.12.4.3: Security of network services |
| Compute | - Amazon EC2: Provides scalable compute instances. |
Protect (PR): - Protective Technology (PR.PT-1) Respond (RS): - Analysis (RS.AN-1), Mitigation (RS.MI-2) |
- A.12.1.3: Capacity management for IT infrastructure and services |
| Load Balancing | - Application Load Balancer (ALB): Distributes HTTP/HTTPS traffic across EC2 instances. |
Protect (PR): - Protective Technology (PR.PT-3) Respond (RS): - Communications (RS.CO-2) |
- A.13.1.1: Network controls - A.13.2.1: Information transfer policies |
| Data Storage | - Amazon S3: Stores application artifacts and logs with encryption, access control, and lifecycle policies. - Amazon RDS: PostgreSQL database with multi-AZ deployment. |
Protect (PR): - Data Security (PR.DS-1, PR.DS-5) - Information Protection Processes and Procedures (PR.IP-3, PR.IP-4) - Maintenance (PR.MA-1) Recover (RC): - Recovery Planning (RC.RP-1), Communications (RC.CO-2) |
- A.8.2.3: Information backup - A.10.1.1: Use of cryptographic controls |
| Secrets Management | - AWS Secrets Manager: Securely stores and rotates sensitive credentials with Lambda rotation support. |
Protect (PR): - Access Control (PR.AC-1, PR.AC-4) - Data Security (PR.DS-6) - Identity Management and Access Control (PR.AC-7) |
- A.9.2.2: User access provisioning - A.10.1.1: Management of encryption keys and secret information |
| Monitoring and Alarms | - Amazon CloudWatch: Provides real-time metrics, logs, and alarms to monitor performance and health. |
Detect (DE): - Security Continuous Monitoring (DE.CM-3) |
- A.12.4.1: Monitoring activities |
| Resilience and Disaster Recovery | - AWS Resilience Hub: Assesses and improves the architecture’s resilience, recommending strategies for fault tolerance and disaster recovery. |
Recover (RC): - Recovery Planning (RC.RP-1) - Improvements (RC.IM-1) |
- A.17.1.2: Implementing continuity controls - A.17.2.1: Availability of information processing facilities |
| Automation and Maintenance | - AWS Systems Manager (SSM): Automates inventory, patching, and maintenance tasks, with SSM Maintenance Windows and SSM Patch Baselines for streamlined operations. |
Protect (PR): - Maintenance (PR.MA-1, PR.MA-2) - Protective Technology (PR.PT-1) |
- A.12.6.1: Control of technical vulnerabilities - A.12.7.1: Information systems audit considerations |
-
Networking and Security: Amazon VPC creates an isolated network environment with NAT Gateway, NACLs, and VPC Flow Logs. VPC Endpoints provide private access to AWS services (e.g., S3, EC2, SSM), AWS WAF protects against web attacks, AWS IAM secures access control, and AWS KMS encrypts data at rest.
-
Domain and SSL Management: Amazon Route 53 handles DNS and domain registration, while AWS Certificate Manager (ACM) provides SSL/TLS certificates for HTTPS security.
-
Compute Layer: Amazon EC2 instances host the application, providing flexible and scalable compute resources.
-
Load Balancing: The Application Load Balancer (ALB) distributes HTTP/HTTPS traffic across EC2 instances, optimizing for high availability and resilience.
-
Data Storage: Amazon RDS offers a resilient PostgreSQL setup with multi-AZ deployment and custom parameter groups. Amazon S3 securely stores artifacts and logs, with lifecycle policies and KMS-managed encryption keys for compliance.
-
Secrets Management: AWS Secrets Manager securely stores and rotates credentials, such as database passwords, with automated Lambda support for rotation.
-
Monitoring and Alarms: Amazon CloudWatch monitors infrastructure health through metrics, logs, and alarms, enabling proactive management.
-
Resilience and Disaster Recovery: AWS Resilience Hub assesses and recommends enhancements to improve the system's resilience, providing disaster recovery and fault-tolerant strategies.
-
Automation and Maintenance: AWS Systems Manager (SSM) automates inventory, patching, and other maintenance tasks, increasing operational efficiency.
The Citizen Intelligence Agency (CIA) project can be deployed on AWS using the provided CloudFormation stack file. This file is located in the cia-dist-cloudformation/src/main/resources/cia-dist-cloudformation.json folder.
You can find the CloudFormation stack file here.
To launch the CloudFormation stack, follow these steps:
-
Log in to your AWS Management Console.
-
Navigate to the CloudFormation service.
-
Click on Create stack.
-
In the Select Template section, choose Upload a template file and upload the
cia-dist-cloudformation.jsonfile. -
Click Next.
-
Fill out the Stack name and any required parameters. The parameters are described in the CloudFormation stack file. Adjust them according to your requirements.
-
Click Next to configure stack options. You can add tags or configure advanced options as needed.
-
Click Next to review your stack settings. Make sure everything is set up as desired.
-
In the Capabilities section, check the boxes for the following options:
- I acknowledge that AWS CloudFormation might create IAM resources.
- I acknowledge that AWS CloudFormation might create IAM resources with custom names.
-
Click Create stack to start the stack creation process. AWS CloudFormation will create the required resources and deploy the CIA project.
-
Once the stack creation process is complete, you can access the application by navigating to the output URL provided in the CloudFormation stack Outputs tab.
This guide will walk you through installing the CIA project on Debian and Ubuntu 24.4+ systems.
- Install OpenJDK and PostgreSQL:
$ sudo apt-get install openjdk-21-jdk postgresql-16- Install PostgreSQL on Ubuntu:
$ sudo apt-get install postgresql-16 postgresql-contrib postgresql-16-pgaudit- Create an empty database:
Below instructions set the default username/password and database name used for development. We recommend using custom credentials and updating the configuration at /opt/cia/webapps/cia/WEB-INF/database.properties to define your own username/password and database name.
$ sudo su - postgres
$ psql
postgres=# CREATE USER eris WITH password 'discord';
postgres=# CREATE DATABASE cia_dev;
postgres=# GRANT ALL PRIVILEGES ON DATABASE cia_dev to eris;A step-by-step guide to configure PostgreSQL 16 with SSL, prepared transactions, and required extensions.
-
Edit
/etc/postgresql/16/main/postgresql.confand add or update the following lines:max_prepared_transactions = 100 shared_preload_libraries = 'pg_stat_statements, pgaudit, pgcrypto' pgaudit.log = ddl pg_stat_statements.track = all pg_stat_statements.max = 10000
- Save and close the file.
-
Edit
/etc/postgresql/16/main/pg_hba.confand add the following line:host all all ::1/128 md5
- Save and close the file.
-
Generate a secure random passphrase:
openssl rand -base64 48 > passphrase.txt -
Create a passphrase-protected private key:
openssl genrsa -des3 -passout file:passphrase.txt -out server.pass.key 2048
-
Remove the passphrase protection from the private key:
openssl rsa -passin file:passphrase.txt -in server.pass.key -out server.key rm server.pass.key
-
Create a Certificate Signing Request (CSR):
openssl req -new -key server.key -out server.csr \ -subj "/C=UK/ST=Postgresqll/L=Docker/O=Hack23/OU=demo/CN=127.0.0.1" -
Self-sign the certificate (valid for 10 years / 3650 days):
openssl x509 -req -days 3650 -in server.csr -signkey server.key -out server.crt
-
Clean up temporary files:
rm passphrase.txt rm server.csr
-
Copy the new certificate and key into the PostgreSQL data directory:
cp server.crt /var/lib/postgresql/16/main/server.crt cp server.key /var/lib/postgresql/16/main/server.key rm server.key
-
Secure the certificate and key:
chmod 700 /var/lib/postgresql/16/main/server.key chmod 700 /var/lib/postgresql/16/main/server.crt chown -R postgres:postgres /var/lib/postgresql/16/main/
-
Enable SSL in PostgreSQL by adding the following lines to
/etc/postgresql/16/main/postgresql.conf:echo "ssl_cert_file = '/var/lib/postgresql/16/main/server.crt'" \ >> /etc/postgresql/16/main/postgresql.conf echo "ssl_key_file = '/var/lib/postgresql/16/main/server.key'" \ >> /etc/postgresql/16/main/postgresql.conf
-
Create a
.postgresqldirectory for theciauser:mkdir -p /opt/cia/.postgresql
-
Copy the server certificate into this directory:
cp server.crt /opt/cia/.postgresql/root.crt chmod 700 /opt/cia/.postgresql/root.crt chown -R cia:cia /opt/cia/.postgresql/root.crt
-
Remove the server certificate from the current directory (if desired):
rm server.crt
-
Restart PostgreSQL to apply all changes:
systemctl restart postgresql
-
Verify that PostgreSQL is running with SSL by checking the logs or using an SSL-enabled client.
-
Confirm that prepared transactions and required extensions are enabled:
SHOW max_prepared_transactions; \dx
-
Confirm the new IPv6 entry in
pg_hba.confis functioning as expected by connecting viapsqlover::1.
You have now successfully configured:
- Prepared transactions
-
Required PostgreSQL extensions (
pg_stat_statements,pgaudit,pgcrypto) - SSL for secure connections
-
IPv6 loopback access in
pg_hba.conf
- Download the CIA Debian package:
$ wget https://github.com/Hack23/cia/releases/download/2025.1.2/cia-dist-deb-2025.1.2.all.deb```
8. Install the Debian package:
```bash
$ sudo dpkg -i cia-dist-deb-2025.1.2.all.deb- Access the server at https://localhost:28443/cia/.
Welcome to the Citizen Intelligence Agency - Sweden Political Activity Dashboard repository. This project provides a comprehensive and neutral dashboard focusing on political activity in Sweden. By monitoring key political figures and institutions, we offer valuable insights into financial performance, risk metrics, and political trends.
Our dashboard offers a detailed overview of political figures, comprehensive data on various ministries, performance metrics, and visual data representation. Check out our dashboard.md for more details.
Medborgarunderrättelsebyrån tillhandahåller en neutral och omfattande dashboard som fokuserar på politisk aktivitet i Sverige. Genom att övervaka nyckelpolitiska figurer och institutioner erbjuder plattformen värdefulla insikter i finansiell prestanda, riskmetriker och politiska trender. Detta projekt drivs av avancerade AI-teknologier som OpenAI och datavisualiseringsverktyg som Daigram.
Vår dashboard erbjuder en detaljerad översikt över politiska figurer, omfattande data om olika departement, prestandametriker och visuell datarepresentation. Kolla in vår dashboard_sv.md för mer information.
This project is powered by advanced AI technologies like OpenAI for data processing and analysis. We use AI to process and analyze a large amount of data from various open sources such as the Swedish Parliament, Swedish Election Authority, World Bank, and the Swedish National Financial Management Authority. The analyzed data is then represented visually through data visualization tools like Daigram.
For Tasks:
Click tags to check more tools for each tasksFor Jobs:
Alternative AI tools for cia
Similar Open Source Tools
cia
CIA is a powerful open-source tool designed for data analysis and visualization. It provides a user-friendly interface for processing large datasets and generating insightful reports. With CIA, users can easily explore data, perform statistical analysis, and create interactive visualizations to communicate findings effectively. Whether you are a data scientist, analyst, or researcher, CIA offers a comprehensive set of features to streamline your data analysis workflow and uncover valuable insights.
db2rest
DB2Rest is a modern low-code REST DATA API platform that simplifies the development of intelligent applications. It seamlessly integrates existing and new databases with language models (LMs/LLMs) and vector stores, enabling the rapid delivery of context-aware, reasoning applications without vendor lock-in.
agentica
Agentica is a specialized Agentic AI library focused on LLM Function Calling. Users can provide Swagger/OpenAPI documents or TypeScript class types to Agentica for seamless functionality. The library simplifies AI development by handling various tasks effortlessly.
airgeddon
Airgeddon is a versatile bash script designed for Linux systems to conduct wireless network audits. It provides a comprehensive set of features and tools for auditing and securing wireless networks. The script is user-friendly and offers functionalities such as scanning, capturing handshakes, deauth attacks, and more. Airgeddon is regularly updated and supported, making it a valuable tool for both security professionals and enthusiasts.
db2rest
DB2Rest is a modern low code REST DATA API platform that enables the rapid development of intelligent applications by combining databases, language models, and vector stores. It facilitates context-aware, reasoning applications without vendor lock-in. The tool accelerates application delivery, fosters faster innovation with AI, serves as a secure database gateway, and simplifies integration. It supports various databases like PostgreSQL, MySQL, MS SQL Server, Oracle, MongoDB, and more, with planned support for additional databases. Users can connect on Discord for support and contact [email protected] for inquiries.
awesome-azure-openai-llm
This repository is a collection of references to Azure OpenAI, Large Language Models (LLM), and related services and libraries. It provides information on various topics such as RAG, Azure OpenAI, LLM applications, agent design patterns, semantic kernel, prompting, finetuning, challenges & abilities, LLM landscape, surveys & references, AI tools & extensions, datasets, and evaluations. The content covers a wide range of topics related to AI, machine learning, and natural language processing, offering insights into the latest advancements in the field.
agentneo
AgentNeo is a Python package that provides functionalities for project, trace, dataset, experiment management. It allows users to authenticate, create projects, trace agents and LangGraph graphs, manage datasets, and run experiments with metrics. The tool aims to streamline AI project management and analysis by offering a comprehensive set of features.
AutoAudit
AutoAudit is an open-source large language model specifically designed for the field of network security. It aims to provide powerful natural language processing capabilities for security auditing and network defense, including analyzing malicious code, detecting network attacks, and predicting security vulnerabilities. By coupling AutoAudit with ClamAV, a security scanning platform has been created for practical security audit applications. The tool is intended to assist security professionals with accurate and fast analysis and predictions to combat evolving network threats.
chatbox
Chatbox is a desktop client for ChatGPT, Claude, and other LLMs, providing a user-friendly interface for AI copilot assistance on Windows, Mac, and Linux. It offers features like local data storage, multiple LLM provider support, image generation with Dall-E-3, enhanced prompting, keyboard shortcuts, and more. Users can collaborate, access the tool on various platforms, and enjoy multilingual support. Chatbox is constantly evolving with new features to enhance the user experience.
chatbox
Chatbox is a desktop client for ChatGPT, Claude, and other LLMs, providing features like local data storage, multiple LLM provider support, image generation, enhanced prompting, keyboard shortcuts, and more. It offers a user-friendly interface with dark theme, team collaboration, cross-platform availability, web version access, iOS & Android apps, multilingual support, and ongoing feature enhancements. Developed for prompt and API debugging, it has gained popularity for daily chatting and professional role-playing with AI assistance.
ClaudeSync
ClaudeSync is a powerful tool designed to seamlessly synchronize local files with Claude.ai projects. It bridges the gap between local development environment and Claude.ai's knowledge base, offering real-time synchronization, CLI for easy management, support for multiple organizations and projects, intelligent file filtering, configurable sync interval, two-way synchronization, and more. It ensures data privacy, open source transparency, and comes with disclaimers for use at own risk. Users can quickly start syncing by installing, logging in, selecting organization and project, and running sync. Advanced features include API, organization, project, file, chat management, configuration, synchronization modes, scheduled sync, providers, custom ignore file, and troubleshooting. Contributions are welcome, and communication channels include GitHub Issues and Discord. Licensed under MIT License.
RisuAI
RisuAI, or Risu for short, is a cross-platform AI chatting software/web application with powerful features such as multiple API support, assets in the chat, regex functions, and much more.
eko
Eko is a lightweight and flexible command-line tool for managing environment variables in your projects. It allows you to easily set, get, and delete environment variables for different environments, making it simple to manage configurations across development, staging, and production environments. With Eko, you can streamline your workflow and ensure consistency in your application settings without the need for complex setup or configuration files.
AgentNeo
AgentNeo is an advanced, open-source Agentic AI Application Observability, Monitoring, and Evaluation Framework designed to provide deep insights into AI agents, Large Language Model (LLM) calls, and tool interactions. It offers robust logging, visualization, and evaluation capabilities to help debug and optimize AI applications with ease. With features like tracing LLM calls, monitoring agents and tools, tracking interactions, detailed metrics collection, flexible data storage, simple instrumentation, interactive dashboard, project management, execution graph visualization, and evaluation tools, AgentNeo empowers users to build efficient, cost-effective, and high-quality AI-driven solutions.
ComfyUI-Copilot
ComfyUI-Copilot is an intelligent assistant built on the Comfy-UI framework that simplifies and enhances the AI algorithm debugging and deployment process through natural language interactions. It offers intuitive node recommendations, workflow building aids, and model querying services to streamline development processes. With features like interactive Q&A bot, natural language node suggestions, smart workflow assistance, and model querying, ComfyUI-Copilot aims to lower the barriers to entry for beginners, boost development efficiency with AI-driven suggestions, and provide real-time assistance for developers.
fast-llm-security-guardrails
ZenGuard AI enables AI developers to integrate production-level, low-code LLM (Large Language Model) guardrails into their generative AI applications effortlessly. With ZenGuard AI, ensure your application operates within trusted boundaries, is protected from prompt injections, and maintains user privacy without compromising on performance.
For similar tasks
aimeos-core
Aimeos is an Open Source e-commerce framework for online shops consisting of the e-commerce library, the administration interface and different front-ends. It offers a modular stack that provides flexibility and speed. Unlike other shop systems, Aimeos allows users to choose from several user front-ends and customize them according to their needs or create their own. It is suitable for medium to large businesses requiring seamless integration into existing systems like content management, customer relationship management, or enterprise resource planning systems. Aimeos also serves as a base for portals or marketplaces.
qrev
QRev is an open-source alternative to Salesforce, offering AI agents to scale sales organizations infinitely. It aims to provide digital workers for various sales roles or a superagent named Qai. The tech stack includes TypeScript for frontend, NodeJS for backend, MongoDB for app server database, ChromaDB for vector database, SQLite for AI server SQL relational database, and Langchain for LLM tooling. The tool allows users to run client app, app server, and AI server components. It requires Node.js and MongoDB to be installed, and provides detailed setup instructions in the README file.
sktime
sktime is a Python library for time series analysis that provides a unified interface for various time series learning tasks such as classification, regression, clustering, annotation, and forecasting. It offers time series algorithms and tools compatible with scikit-learn for building, tuning, and validating time series models. sktime aims to enhance the interoperability and usability of the time series analysis ecosystem by empowering users to apply algorithms across different tasks and providing interfaces to related libraries like scikit-learn, statsmodels, tsfresh, PyOD, and fbprophet.
pandas-ai
PandaAI is a Python platform that enables users to interact with their data in natural language, catering to both non-technical and technical users. It simplifies data querying and analysis, offering conversational data analytics capabilities with minimal code. Users can ask questions, visualize charts, and compare dataframes effortlessly. The tool aims to streamline data exploration and decision-making processes by providing a user-friendly interface for data manipulation and analysis.
cia
CIA is a powerful open-source tool designed for data analysis and visualization. It provides a user-friendly interface for processing large datasets and generating insightful reports. With CIA, users can easily explore data, perform statistical analysis, and create interactive visualizations to communicate findings effectively. Whether you are a data scientist, analyst, or researcher, CIA offers a comprehensive set of features to streamline your data analysis workflow and uncover valuable insights.
aimeos-headless
Aimeos headless distribution is an ultra-fast, cloud-native, and API-first headless ecommerce solution for Laravel. It offers a full-featured e-commerce package with features like JSON REST API, GraphQL API, multi-vendor support, subscriptions, block/tier pricing, admin backend, and more. The distribution is highly customizable, extensible, and suitable for multi-tenant e-commerce SaaS solutions. It supports multiple languages, AI-based text translation, and provides secure and high-quality source code. Aimeos is designed for AWS, Google, Azure, and Kubernetes based clouds, and can handle a wide range of products efficiently.
For similar jobs
databerry
Chaindesk is a no-code platform that allows users to easily set up a semantic search system for personal data without technical knowledge. It supports loading data from various sources such as raw text, web pages, files (Word, Excel, PowerPoint, PDF, Markdown, Plain Text), and upcoming support for web sites, Notion, and Airtable. The platform offers a user-friendly interface for managing datastores, querying data via a secure API endpoint, and auto-generating ChatGPT Plugins for each datastore. Chaindesk utilizes a Vector Database (Qdrant), Openai's text-embedding-ada-002 for embeddings, and has a chunk size of 1024 tokens. The technology stack includes Next.js, Joy UI, LangchainJS, PostgreSQL, Prisma, and Qdrant, inspired by the ChatGPT Retrieval Plugin.
OAD
OAD is a powerful open-source tool for analyzing and visualizing data. It provides a user-friendly interface for exploring datasets, generating insights, and creating interactive visualizations. With OAD, users can easily import data from various sources, clean and preprocess data, perform statistical analysis, and create customizable visualizations to communicate findings effectively. Whether you are a data scientist, analyst, or researcher, OAD can help you streamline your data analysis workflow and uncover valuable insights from your data.
sqlcoder
Defog's SQLCoder is a family of state-of-the-art large language models (LLMs) designed for converting natural language questions into SQL queries. It outperforms popular open-source models like gpt-4 and gpt-4-turbo on SQL generation tasks. SQLCoder has been trained on more than 20,000 human-curated questions based on 10 different schemas, and the model weights are licensed under CC BY-SA 4.0. Users can interact with SQLCoder through the 'transformers' library and run queries using the 'sqlcoder launch' command in the terminal. The tool has been tested on NVIDIA GPUs with more than 16GB VRAM and Apple Silicon devices with some limitations. SQLCoder offers a demo on their website and supports quantized versions of the model for consumer GPUs with sufficient memory.
TableLLM
TableLLM is a large language model designed for efficient tabular data manipulation tasks in real office scenarios. It can generate code solutions or direct text answers for tasks like insert, delete, update, query, merge, and chart operations on tables embedded in spreadsheets or documents. The model has been fine-tuned based on CodeLlama-7B and 13B, offering two scales: TableLLM-7B and TableLLM-13B. Evaluation results show its performance on benchmarks like WikiSQL, Spider, and self-created table operation benchmark. Users can use TableLLM for code and text generation tasks on tabular data.
mlcraft
Synmetrix (prev. MLCraft) is an open source data engineering platform and semantic layer for centralized metrics management. It provides a complete framework for modeling, integrating, transforming, aggregating, and distributing metrics data at scale. Key features include data modeling and transformations, semantic layer for unified data model, scheduled reports and alerts, versioning, role-based access control, data exploration, caching, and collaboration on metrics modeling. Synmetrix leverages Cube (Cube.js) for flexible data models that consolidate metrics from various sources, enabling downstream distribution via a SQL API for integration into BI tools, reporting, dashboards, and data science. Use cases include data democratization, business intelligence, embedded analytics, and enhancing accuracy in data handling and queries. The tool speeds up data-driven workflows from metrics definition to consumption by combining data engineering best practices with self-service analytics capabilities.
data-scientist-roadmap2024
The Data Scientist Roadmap2024 provides a comprehensive guide to mastering essential tools for data science success. It includes programming languages, machine learning libraries, cloud platforms, and concepts categorized by difficulty. The roadmap covers a wide range of topics from programming languages to machine learning techniques, data visualization tools, and DevOps/MLOps tools. It also includes web development frameworks and specific concepts like supervised and unsupervised learning, NLP, deep learning, reinforcement learning, and statistics. Additionally, it delves into DevOps tools like Airflow and MLFlow, data visualization tools like Tableau and Matplotlib, and other topics such as ETL processes, optimization algorithms, and financial modeling.
VMind
VMind is an open-source solution for intelligent visualization, providing an intelligent chart component based on LLM by VisActor. It allows users to create chart narrative works with natural language interaction, edit charts through dialogue, and export narratives as videos or GIFs. The tool is easy to use, scalable, supports various chart types, and offers one-click export functionality. Users can customize chart styles, specify themes, and aggregate data using LLM models. VMind aims to enhance efficiency in creating data visualization works through dialogue-based editing and natural language interaction.
quadratic
Quadratic is a modern multiplayer spreadsheet application that integrates Python, AI, and SQL functionalities. It aims to streamline team collaboration and data analysis by enabling users to pull data from various sources and utilize popular data science tools. The application supports building dashboards, creating internal tools, mixing data from different sources, exploring data for insights, visualizing Python workflows, and facilitating collaboration between technical and non-technical team members. Quadratic is built with Rust + WASM + WebGL to ensure seamless performance in the browser, and it offers features like WebGL Grid, local file management, Python and Pandas support, Excel formula support, multiplayer capabilities, charts and graphs, and team support. The tool is currently in Beta with ongoing development for additional features like JS support, SQL database support, and AI auto-complete.