cia
Comprehensive open-source intelligence platform analyzing Swedish political activities using AI and data visualization. Tracks politicians, government institutions, and parliamentary data, offering detailed insights, performance metrics, and advanced analytics.
Stars: 162
CIA is a powerful open-source tool designed for data analysis and visualization. It provides a user-friendly interface for processing large datasets and generating insightful reports. With CIA, users can easily explore data, perform statistical analysis, and create interactive visualizations to communicate findings effectively. Whether you are a data scientist, analyst, or researcher, CIA offers a comprehensive set of features to streamline your data analysis workflow and uncover valuable insights.
README:
An independent, volunteer-driven OSINT platform monitoring Swedish political activity
The Citizen Intelligence Agency is a volunteer-driven, open-source intelligence (OSINT) project that provides comprehensive analysis of political activities in Sweden. Through advanced monitoring of key political figures and institutions, we deliver:
- 📊 Financial performance metrics
⚠️ Risk assessment analytics- 📈 Political trend analysis
- 🏆 Politician ranking system
- 📉 Performance comparisons
- 🔍 Transparency insights
Our initiative remains strictly independent and non-partisan, focused on fostering informed decision-making and enhancing democratic engagement.
Explore our comprehensive feature set including:
- 📊 Interactive dashboards
- 🏆 Political scoreboard systems
- 📈 Critical analytics tools
- 🔍 Transparency metrics
- ⚖️ Accountability measures
- 📱 Data-driven insights
- 🌐 Website: www.hack23.com
- 💼 LinkedIn: James Sörling
Our analysis is powered by authoritative Swedish government and international data sources:
| Source | Description |
|---|---|
| 🏛️ Swedish Parliament Open Data | Parliamentary members, committees, and official documents |
| 🗳️ Swedish Election Authority | Election data, political parties, and voting results |
| 🌍 World Bank Open Data | Global economic indicators and demographic data |
| 💹 Swedish Financial Management Authority | Government finances and economic trends |
[
- Project Documentation
- Features of the CIA Project This document showcases the extensive features, providing detailed screenshots of dashboards, scoreboards, and critical analytics. The focus is on transparency, accountability, and data-driven decision-making in Sweden’s political ecosystem.
- Project Architecture - Delve into the architecture of the Citizen Intelligence Agency. This overview provides a look at the enterprise context, system context, system containers, web application components, deployment strategy, and AWS account structure of the project.
- Entity Model - Explore our Entity Model which provides a detailed look at the entities in our system and their relationships. This page is particularly useful for understanding the data structure of our project.
- Api docs - Access the API documentation for the Citizen Intelligence Agency project. This documentation provides a detailed view of the various packages within the system, helping developers understand and work with the project's API.
- FinancialSecurityPlan.md - This financial plan provides a structured and cost-efficient deployment for your application infrastructure in the AWS eu-west-1 (Ireland) region. It integrates key components of scalability, security, and resilience to support critical workloads while maintaining budgetary control
- End-of-Life-Strategy.md Project End-of-Life (EOL) Strategy
Please follow the instructions in our SECURITY.md file for reporting security issues.
This document provides a high-level overview of the key technologies used within the Citizen Intelligence Agency (CIA) project. Each technology plays a vital role in supporting CIA’s goals for data analysis, security, and scalability within the political intelligence domain.
| Category | Technologies |
|---|---|
| Core Framework | Spring Framework |
| Security | Spring Security, Bouncy Castle |
| Data Access | Hibernate, JPA, PostgreSQL, JDBC |
| Transaction Management |
Narayana (Integrated with Spring JpaTransactionManager) |
| Data Auditing | Javers |
| Business Rules Engine | Drools |
| Messaging | ActiveMQ Artemis, Spring JMS |
| Web/UI Layer | Vaadin, Vaadin Sass Compiler, Vaadin Themes |
| Monitoring | JavaMelody, AWS SDK for CloudWatch |
| Testing | JUnit, Mockito, Spring Test, Selenium WebDriver |
| Utilities | Apache Commons, Google Guava, SLF4J, Logback, Jackson |
| Build & Dependency Management | Maven |
This stack comprises:
- Core Framework: The project uses Spring Framework to provide a foundation for dependency injection, component management, and service configuration across modules.
- Security: Spring Security manages authentication and authorization, complemented by Bouncy Castle for cryptographic operations.
- Data Access: A combination of Hibernate, JPA, and PostgreSQL supports robust ORM-based data persistence, with JDBC facilitating additional database connectivity needs.
- Transaction Management: The project uses Narayana as the transaction manager implementation, integrated with Spring’s JpaTransactionManager for distributed transaction support and ensuring transactional integrity.
- Data Auditing: Javers provides auditing and historical versioning, allowing for tracking and comparing changes to data over time.
- Business Rules Engine : Drools is integrated into the CIA project to enable a robust business rules engine.
- Messaging: ActiveMQ Artemis and Spring JMS enable asynchronous communication between application components, supporting distributed and event-driven designs.
- Web/UI Layer: Vaadin powers the UI with a server-driven architecture, providing components like Vaadin Themes and Sass Compiler for a rich, interactive frontend experience directly in Java.
- Monitoring: JavaMelody and AWS SDK for CloudWatch provide real-time application monitoring and logging capabilities, supporting both local and cloud environments.
- Testing: JUnit, Mockito, Spring Test and Selenium WebDriver are used extensively for unit, integration, system, browser and mock testing to ensure application reliability and robustness.
- Utilities: Apache Commons, Google Guava, SLF4J, and Logback offer utility functions and structured logging, enhancing application maintainability and monitoring.
- Build & Dependency Management: Maven handles project builds, dependency management, and plugin configurations, enabling smooth project management and modular builds.
This document provides a comprehensive summary of the AWS services utilized in the Citizen Intelligence Agency (CIA) project infrastructure, as defined by its CloudFormation template. These services work together to ensure a secure, resilient, and scalable deployment environment.
| Category | AWS Services | NIST CSF Function, Category & Subcategory | ISO 27001:2022 Control & Link |
|---|---|---|---|
| Networking and Security | - Amazon VPC: Configures a custom network environment with public/private subnets, route tables, NAT Gateway, Network ACLs (NACLs) for traffic control, and VPC Flow Logs. - VPC Endpoints: Enables private access to AWS services (e.g., S3, EC2, SSM, CloudWatch Logs). - AWS WAF: Protects against web attacks at the ALB layer. - AWS IAM: Manages role-based access control. - AWS KMS: Manages encryption for data at rest. |
Identify (ID): - Asset Management (ID.AM-2) Protect (PR): - Access Control (PR.AC-1, PR.AC-3, PR.AC-5) - Data Security (PR.DS-1, PR.DS-2) - Protective Technology (PR.PT-3) Detect (DE): - Security Continuous Monitoring (DE.CM-3) |
- A.8.1: Asset management - A.9.4.1: Access control policy - A.13.1.1: Network controls - A.13.1.3: Segregation in networks - A.18.1.5: Regulation and compliance (see ISO 27001) |
| Domain and SSL Management | - Amazon Route 53: Manages domain registration and DNS routing. - AWS Certificate Manager (ACM): Issues and manages SSL/TLS certificates. |
Protect (PR): - Data Security (PR.DS-5) Detect (DE): - Anomalies and Events (DE.AE-3) |
- A.10.1.1: Cryptographic controls for data protection - A.12.4.3: Security of network services |
| Compute | - Amazon EC2: Provides scalable compute instances. |
Protect (PR): - Protective Technology (PR.PT-1) Respond (RS): - Analysis (RS.AN-1), Mitigation (RS.MI-2) |
- A.12.1.3: Capacity management for IT infrastructure and services |
| Load Balancing | - Application Load Balancer (ALB): Distributes HTTP/HTTPS traffic across EC2 instances. |
Protect (PR): - Protective Technology (PR.PT-3) Respond (RS): - Communications (RS.CO-2) |
- A.13.1.1: Network controls - A.13.2.1: Information transfer policies |
| Data Storage | - Amazon S3: Stores application artifacts and logs with encryption, access control, and lifecycle policies. - Amazon RDS: PostgreSQL database with multi-AZ deployment. |
Protect (PR): - Data Security (PR.DS-1, PR.DS-5) - Information Protection Processes and Procedures (PR.IP-3, PR.IP-4) - Maintenance (PR.MA-1) Recover (RC): - Recovery Planning (RC.RP-1), Communications (RC.CO-2) |
- A.8.2.3: Information backup - A.10.1.1: Use of cryptographic controls |
| Secrets Management | - AWS Secrets Manager: Securely stores and rotates sensitive credentials with Lambda rotation support. |
Protect (PR): - Access Control (PR.AC-1, PR.AC-4) - Data Security (PR.DS-6) - Identity Management and Access Control (PR.AC-7) |
- A.9.2.2: User access provisioning - A.10.1.1: Management of encryption keys and secret information |
| Monitoring and Alarms | - Amazon CloudWatch: Provides real-time metrics, logs, and alarms to monitor performance and health. |
Detect (DE): - Security Continuous Monitoring (DE.CM-3) |
- A.12.4.1: Monitoring activities |
| Resilience and Disaster Recovery | - AWS Resilience Hub: Assesses and improves the architecture’s resilience, recommending strategies for fault tolerance and disaster recovery. |
Recover (RC): - Recovery Planning (RC.RP-1) - Improvements (RC.IM-1) |
- A.17.1.2: Implementing continuity controls - A.17.2.1: Availability of information processing facilities |
| Automation and Maintenance | - AWS Systems Manager (SSM): Automates inventory, patching, and maintenance tasks, with SSM Maintenance Windows and SSM Patch Baselines for streamlined operations. |
Protect (PR): - Maintenance (PR.MA-1, PR.MA-2) - Protective Technology (PR.PT-1) |
- A.12.6.1: Control of technical vulnerabilities - A.12.7.1: Information systems audit considerations |
-
Networking and Security: Amazon VPC creates an isolated network environment with NAT Gateway, NACLs, and VPC Flow Logs. VPC Endpoints provide private access to AWS services (e.g., S3, EC2, SSM), AWS WAF protects against web attacks, AWS IAM secures access control, and AWS KMS encrypts data at rest.
-
Domain and SSL Management: Amazon Route 53 handles DNS and domain registration, while AWS Certificate Manager (ACM) provides SSL/TLS certificates for HTTPS security.
-
Compute Layer: Amazon EC2 instances host the application, providing flexible and scalable compute resources.
-
Load Balancing: The Application Load Balancer (ALB) distributes HTTP/HTTPS traffic across EC2 instances, optimizing for high availability and resilience.
-
Data Storage: Amazon RDS offers a resilient PostgreSQL setup with multi-AZ deployment and custom parameter groups. Amazon S3 securely stores artifacts and logs, with lifecycle policies and KMS-managed encryption keys for compliance.
-
Secrets Management: AWS Secrets Manager securely stores and rotates credentials, such as database passwords, with automated Lambda support for rotation.
-
Monitoring and Alarms: Amazon CloudWatch monitors infrastructure health through metrics, logs, and alarms, enabling proactive management.
-
Resilience and Disaster Recovery: AWS Resilience Hub assesses and recommends enhancements to improve the system's resilience, providing disaster recovery and fault-tolerant strategies.
-
Automation and Maintenance: AWS Systems Manager (SSM) automates inventory, patching, and other maintenance tasks, increasing operational efficiency.
The Citizen Intelligence Agency (CIA) project can be deployed on AWS using the provided CloudFormation stack file. This file is located in the cia-dist-cloudformation/src/main/resources/cia-dist-cloudformation.json folder.
You can find the CloudFormation stack file here.
To launch the CloudFormation stack, follow these steps:
-
Log in to your AWS Management Console.
-
Navigate to the CloudFormation service.
-
Click on Create stack.
-
In the Select Template section, choose Upload a template file and upload the
cia-dist-cloudformation.jsonfile. -
Click Next.
-
Fill out the Stack name and any required parameters. The parameters are described in the CloudFormation stack file. Adjust them according to your requirements.
-
Click Next to configure stack options. You can add tags or configure advanced options as needed.
-
Click Next to review your stack settings. Make sure everything is set up as desired.
-
In the Capabilities section, check the boxes for the following options:
- I acknowledge that AWS CloudFormation might create IAM resources.
- I acknowledge that AWS CloudFormation might create IAM resources with custom names.
-
Click Create stack to start the stack creation process. AWS CloudFormation will create the required resources and deploy the CIA project.
-
Once the stack creation process is complete, you can access the application by navigating to the output URL provided in the CloudFormation stack Outputs tab.
This guide will walk you through installing the CIA project on Debian and Ubuntu 24.4+ systems.
- Install OpenJDK and PostgreSQL:
$ sudo apt-get install openjdk-21-jdk postgresql-16- Install PostgreSQL on Ubuntu:
$ sudo apt-get install postgresql-16 postgresql-contrib postgresql-16-pgaudit- Create an empty database:
Below instructions set the default username/password and database name used for development. We recommend using custom credentials and updating the configuration at /opt/cia/webapps/cia/WEB-INF/database.properties to define your own username/password and database name.
$ sudo su - postgres
$ psql
postgres=# CREATE USER eris WITH password 'discord';
postgres=# CREATE DATABASE cia_dev;
postgres=# GRANT ALL PRIVILEGES ON DATABASE cia_dev to eris;A step-by-step guide to configure PostgreSQL 16 with SSL, prepared transactions, and required extensions.
-
Edit
/etc/postgresql/16/main/postgresql.confand add or update the following lines:max_prepared_transactions = 100 shared_preload_libraries = 'pg_stat_statements, pgaudit, pgcrypto' pgaudit.log = ddl pg_stat_statements.track = all pg_stat_statements.max = 10000
- Save and close the file.
-
Edit
/etc/postgresql/16/main/pg_hba.confand add the following line:host all all ::1/128 md5
- Save and close the file.
-
Generate a secure random passphrase:
openssl rand -base64 48 > passphrase.txt -
Create a passphrase-protected private key:
openssl genrsa -des3 -passout file:passphrase.txt -out server.pass.key 2048
-
Remove the passphrase protection from the private key:
openssl rsa -passin file:passphrase.txt -in server.pass.key -out server.key rm server.pass.key
-
Create a Certificate Signing Request (CSR):
openssl req -new -key server.key -out server.csr \ -subj "/C=UK/ST=Postgresqll/L=Docker/O=Hack23/OU=demo/CN=127.0.0.1" -
Self-sign the certificate (valid for 10 years / 3650 days):
openssl x509 -req -days 3650 -in server.csr -signkey server.key -out server.crt
-
Clean up temporary files:
rm passphrase.txt rm server.csr
-
Copy the new certificate and key into the PostgreSQL data directory:
cp server.crt /var/lib/postgresql/16/main/server.crt cp server.key /var/lib/postgresql/16/main/server.key rm server.key
-
Secure the certificate and key:
chmod 700 /var/lib/postgresql/16/main/server.key chmod 700 /var/lib/postgresql/16/main/server.crt chown -R postgres:postgres /var/lib/postgresql/16/main/
-
Enable SSL in PostgreSQL by adding the following lines to
/etc/postgresql/16/main/postgresql.conf:echo "ssl_cert_file = '/var/lib/postgresql/16/main/server.crt'" \ >> /etc/postgresql/16/main/postgresql.conf echo "ssl_key_file = '/var/lib/postgresql/16/main/server.key'" \ >> /etc/postgresql/16/main/postgresql.conf
-
Create a
.postgresqldirectory for theciauser:mkdir -p /opt/cia/.postgresql
-
Copy the server certificate into this directory:
cp server.crt /opt/cia/.postgresql/root.crt chmod 700 /opt/cia/.postgresql/root.crt chown -R cia:cia /opt/cia/.postgresql/root.crt
-
Remove the server certificate from the current directory (if desired):
rm server.crt
-
Restart PostgreSQL to apply all changes:
systemctl restart postgresql
-
Verify that PostgreSQL is running with SSL by checking the logs or using an SSL-enabled client.
-
Confirm that prepared transactions and required extensions are enabled:
SHOW max_prepared_transactions; \dx
-
Confirm the new IPv6 entry in
pg_hba.confis functioning as expected by connecting viapsqlover::1.
You have now successfully configured:
- Prepared transactions
-
Required PostgreSQL extensions (
pg_stat_statements,pgaudit,pgcrypto) - SSL for secure connections
-
IPv6 loopback access in
pg_hba.conf
- Download the CIA Debian package:
$ wget https://github.com/Hack23/cia/releases/download/2025.1.2/cia-dist-deb-2025.1.2.all.deb```
8. Install the Debian package:
```bash
$ sudo dpkg -i cia-dist-deb-2025.1.2.all.deb- Access the server at https://localhost:28443/cia/.
Welcome to the Citizen Intelligence Agency - Sweden Political Activity Dashboard repository. This project provides a comprehensive and neutral dashboard focusing on political activity in Sweden. By monitoring key political figures and institutions, we offer valuable insights into financial performance, risk metrics, and political trends.
Our dashboard offers a detailed overview of political figures, comprehensive data on various ministries, performance metrics, and visual data representation. Check out our dashboard.md for more details.
Medborgarunderrättelsebyrån tillhandahåller en neutral och omfattande dashboard som fokuserar på politisk aktivitet i Sverige. Genom att övervaka nyckelpolitiska figurer och institutioner erbjuder plattformen värdefulla insikter i finansiell prestanda, riskmetriker och politiska trender. Detta projekt drivs av avancerade AI-teknologier som OpenAI och datavisualiseringsverktyg som Daigram.
Vår dashboard erbjuder en detaljerad översikt över politiska figurer, omfattande data om olika departement, prestandametriker och visuell datarepresentation. Kolla in vår dashboard_sv.md för mer information.
This project is powered by advanced AI technologies like OpenAI for data processing and analysis. We use AI to process and analyze a large amount of data from various open sources such as the Swedish Parliament, Swedish Election Authority, World Bank, and the Swedish National Financial Management Authority. The analyzed data is then represented visually through data visualization tools like Daigram.
For Tasks:
Click tags to check more tools for each tasksFor Jobs:
Alternative AI tools for cia
Similar Open Source Tools
cia
CIA is a powerful open-source tool designed for data analysis and visualization. It provides a user-friendly interface for processing large datasets and generating insightful reports. With CIA, users can easily explore data, perform statistical analysis, and create interactive visualizations to communicate findings effectively. Whether you are a data scientist, analyst, or researcher, CIA offers a comprehensive set of features to streamline your data analysis workflow and uncover valuable insights.
xpert
Xpert is a powerful tool for data analysis and visualization. It provides a user-friendly interface to explore and manipulate datasets, perform statistical analysis, and create insightful visualizations. With Xpert, users can easily import data from various sources, clean and preprocess data, analyze trends and patterns, and generate interactive charts and graphs. Whether you are a data scientist, analyst, researcher, or student, Xpert simplifies the process of data analysis and visualization, making it accessible to users with varying levels of expertise.
OAD
OAD is a powerful open-source tool for analyzing and visualizing data. It provides a user-friendly interface for exploring datasets, generating insights, and creating interactive visualizations. With OAD, users can easily import data from various sources, clean and preprocess data, perform statistical analysis, and create customizable visualizations to communicate findings effectively. Whether you are a data scientist, analyst, or researcher, OAD can help you streamline your data analysis workflow and uncover valuable insights from your data.
xorq
Xorq (formerly LETSQL) is a data processing library built on top of Ibis and DataFusion to write multi-engine data workflows. It provides a flexible and powerful tool for processing and analyzing data from various sources, enabling users to create complex data pipelines and perform advanced data transformations.
AI_Spectrum
AI_Spectrum is a versatile machine learning library that provides a wide range of tools and algorithms for building and deploying AI models. It offers a user-friendly interface for data preprocessing, model training, and evaluation. With AI_Spectrum, users can easily experiment with different machine learning techniques and optimize their models for various tasks. The library is designed to be flexible and scalable, making it suitable for both beginners and experienced data scientists.
PythonDataScienceFullThrottle
PythonDataScienceFullThrottle is a comprehensive repository containing various Python scripts, libraries, and tools for data science enthusiasts. It includes a wide range of functionalities such as data preprocessing, visualization, machine learning algorithms, and statistical analysis. The repository aims to provide a one-stop solution for individuals looking to dive deep into the world of data science using Python.
atomic-agents
The Atomic Agents framework is a modular and extensible tool designed for creating powerful applications. It leverages Pydantic for data validation and serialization. The framework follows the principles of Atomic Design, providing small and single-purpose components that can be combined. It integrates with Instructor for AI agent architecture and supports various APIs like Cohere, Anthropic, and Gemini. The tool includes documentation, examples, and testing features to ensure smooth development and usage.
God-Level-AI
A drill of scientific methods, processes, algorithms, and systems to build stories & models. An in-depth learning resource for humans. This repository is designed for individuals aiming to excel in the field of Data and AI, providing video sessions and text content for learning. It caters to those in leadership positions, professionals, and students, emphasizing the need for dedicated effort to achieve excellence in the tech field. The content covers various topics with a focus on practical application.
trubrics-sdk
Trubrics-sdk is a software development kit designed to facilitate the integration of analytics features into applications. It provides a set of tools and functionalities that enable developers to easily incorporate analytics capabilities, such as data collection, analysis, and reporting, into their software products. The SDK streamlines the process of implementing analytics solutions, allowing developers to focus on building and enhancing their applications' functionality and user experience. By leveraging trubrics-sdk, developers can quickly and efficiently integrate robust analytics features, gaining valuable insights into user behavior and application performance.
5ire
5ire is a cross-platform desktop client that integrates a local knowledge base for multilingual vectorization, supports parsing and vectorization of various document formats, offers usage analytics to track API spending, provides a prompts library for creating and organizing prompts with variable support, allows bookmarking of conversations, and enables quick keyword searches across conversations. It is licensed under the GNU General Public License version 3.
GrowthHacking-Notes
GrowthHacking-Notes is a repository containing detailed notes, strategies, and resources related to growth hacking. It provides valuable insights and tips for individuals and businesses looking to accelerate their growth through innovative marketing techniques and data-driven strategies. The repository covers various topics such as user acquisition, retention, conversion optimization, and more, making it a comprehensive resource for anyone interested in growth hacking.
multimodal_cognitive_ai
The multimodal cognitive AI repository focuses on research work related to multimodal cognitive artificial intelligence. It explores the integration of multiple modes of data such as text, images, and audio to enhance AI systems' cognitive capabilities. The repository likely contains code, datasets, and research papers related to multimodal AI applications, including natural language processing, computer vision, and audio processing. Researchers and developers interested in advancing AI systems' understanding of multimodal data can find valuable resources and insights in this repository.
ai-workshop-code
The ai-workshop-code repository contains code examples and tutorials for various artificial intelligence concepts and algorithms. It serves as a practical resource for individuals looking to learn and implement AI techniques in their projects. The repository covers a wide range of topics, including machine learning, deep learning, natural language processing, computer vision, and reinforcement learning. By exploring the code and following the tutorials, users can gain hands-on experience with AI technologies and enhance their understanding of how these algorithms work in practice.
Cyberion-Spark-X
Cyberion-Spark-X is a powerful open-source tool designed for cybersecurity professionals and data analysts. It provides advanced capabilities for analyzing and visualizing large datasets to detect security threats and anomalies. The tool integrates with popular data sources and supports various machine learning algorithms for predictive analytics and anomaly detection. Cyberion-Spark-X is user-friendly and highly customizable, making it suitable for both beginners and experienced professionals in the field of cybersecurity and data analysis.
mindsdb
MindsDB is a platform for customizing AI from enterprise data. You can create, serve, and fine-tune models in real-time from your database, vector store, and application data. MindsDB "enhances" SQL syntax with AI capabilities to make it accessible for developers worldwide. With MindsDB’s nearly 200 integrations, any developer can create AI customized for their purpose, faster and more securely. Their AI systems will constantly improve themselves — using companies’ own data, in real-time.
zillionare
This repository contains a collection of articles and tutorials on quantitative finance, including topics such as machine learning, statistical arbitrage, and risk management. The articles are written in a clear and concise style, and they are suitable for both beginners and experienced practitioners. The repository also includes a number of Jupyter notebooks that demonstrate how to use Python for quantitative finance.
For similar tasks
aimeos-core
Aimeos is an Open Source e-commerce framework for online shops consisting of the e-commerce library, the administration interface and different front-ends. It offers a modular stack that provides flexibility and speed. Unlike other shop systems, Aimeos allows users to choose from several user front-ends and customize them according to their needs or create their own. It is suitable for medium to large businesses requiring seamless integration into existing systems like content management, customer relationship management, or enterprise resource planning systems. Aimeos also serves as a base for portals or marketplaces.
qrev
QRev is an open-source alternative to Salesforce, offering AI agents to scale sales organizations infinitely. It aims to provide digital workers for various sales roles or a superagent named Qai. The tech stack includes TypeScript for frontend, NodeJS for backend, MongoDB for app server database, ChromaDB for vector database, SQLite for AI server SQL relational database, and Langchain for LLM tooling. The tool allows users to run client app, app server, and AI server components. It requires Node.js and MongoDB to be installed, and provides detailed setup instructions in the README file.
sktime
sktime is a Python library for time series analysis that provides a unified interface for various time series learning tasks such as classification, regression, clustering, annotation, and forecasting. It offers time series algorithms and tools compatible with scikit-learn for building, tuning, and validating time series models. sktime aims to enhance the interoperability and usability of the time series analysis ecosystem by empowering users to apply algorithms across different tasks and providing interfaces to related libraries like scikit-learn, statsmodels, tsfresh, PyOD, and fbprophet.
pandas-ai
PandaAI is a Python platform that enables users to interact with their data in natural language, catering to both non-technical and technical users. It simplifies data querying and analysis, offering conversational data analytics capabilities with minimal code. Users can ask questions, visualize charts, and compare dataframes effortlessly. The tool aims to streamline data exploration and decision-making processes by providing a user-friendly interface for data manipulation and analysis.
cia
CIA is a powerful open-source tool designed for data analysis and visualization. It provides a user-friendly interface for processing large datasets and generating insightful reports. With CIA, users can easily explore data, perform statistical analysis, and create interactive visualizations to communicate findings effectively. Whether you are a data scientist, analyst, or researcher, CIA offers a comprehensive set of features to streamline your data analysis workflow and uncover valuable insights.
For similar jobs
databerry
Chaindesk is a no-code platform that allows users to easily set up a semantic search system for personal data without technical knowledge. It supports loading data from various sources such as raw text, web pages, files (Word, Excel, PowerPoint, PDF, Markdown, Plain Text), and upcoming support for web sites, Notion, and Airtable. The platform offers a user-friendly interface for managing datastores, querying data via a secure API endpoint, and auto-generating ChatGPT Plugins for each datastore. Chaindesk utilizes a Vector Database (Qdrant), Openai's text-embedding-ada-002 for embeddings, and has a chunk size of 1024 tokens. The technology stack includes Next.js, Joy UI, LangchainJS, PostgreSQL, Prisma, and Qdrant, inspired by the ChatGPT Retrieval Plugin.
OAD
OAD is a powerful open-source tool for analyzing and visualizing data. It provides a user-friendly interface for exploring datasets, generating insights, and creating interactive visualizations. With OAD, users can easily import data from various sources, clean and preprocess data, perform statistical analysis, and create customizable visualizations to communicate findings effectively. Whether you are a data scientist, analyst, or researcher, OAD can help you streamline your data analysis workflow and uncover valuable insights from your data.
sqlcoder
Defog's SQLCoder is a family of state-of-the-art large language models (LLMs) designed for converting natural language questions into SQL queries. It outperforms popular open-source models like gpt-4 and gpt-4-turbo on SQL generation tasks. SQLCoder has been trained on more than 20,000 human-curated questions based on 10 different schemas, and the model weights are licensed under CC BY-SA 4.0. Users can interact with SQLCoder through the 'transformers' library and run queries using the 'sqlcoder launch' command in the terminal. The tool has been tested on NVIDIA GPUs with more than 16GB VRAM and Apple Silicon devices with some limitations. SQLCoder offers a demo on their website and supports quantized versions of the model for consumer GPUs with sufficient memory.
TableLLM
TableLLM is a large language model designed for efficient tabular data manipulation tasks in real office scenarios. It can generate code solutions or direct text answers for tasks like insert, delete, update, query, merge, and chart operations on tables embedded in spreadsheets or documents. The model has been fine-tuned based on CodeLlama-7B and 13B, offering two scales: TableLLM-7B and TableLLM-13B. Evaluation results show its performance on benchmarks like WikiSQL, Spider, and self-created table operation benchmark. Users can use TableLLM for code and text generation tasks on tabular data.
mlcraft
Synmetrix (prev. MLCraft) is an open source data engineering platform and semantic layer for centralized metrics management. It provides a complete framework for modeling, integrating, transforming, aggregating, and distributing metrics data at scale. Key features include data modeling and transformations, semantic layer for unified data model, scheduled reports and alerts, versioning, role-based access control, data exploration, caching, and collaboration on metrics modeling. Synmetrix leverages Cube (Cube.js) for flexible data models that consolidate metrics from various sources, enabling downstream distribution via a SQL API for integration into BI tools, reporting, dashboards, and data science. Use cases include data democratization, business intelligence, embedded analytics, and enhancing accuracy in data handling and queries. The tool speeds up data-driven workflows from metrics definition to consumption by combining data engineering best practices with self-service analytics capabilities.
data-scientist-roadmap2024
The Data Scientist Roadmap2024 provides a comprehensive guide to mastering essential tools for data science success. It includes programming languages, machine learning libraries, cloud platforms, and concepts categorized by difficulty. The roadmap covers a wide range of topics from programming languages to machine learning techniques, data visualization tools, and DevOps/MLOps tools. It also includes web development frameworks and specific concepts like supervised and unsupervised learning, NLP, deep learning, reinforcement learning, and statistics. Additionally, it delves into DevOps tools like Airflow and MLFlow, data visualization tools like Tableau and Matplotlib, and other topics such as ETL processes, optimization algorithms, and financial modeling.
VMind
VMind is an open-source solution for intelligent visualization, providing an intelligent chart component based on LLM by VisActor. It allows users to create chart narrative works with natural language interaction, edit charts through dialogue, and export narratives as videos or GIFs. The tool is easy to use, scalable, supports various chart types, and offers one-click export functionality. Users can customize chart styles, specify themes, and aggregate data using LLM models. VMind aims to enhance efficiency in creating data visualization works through dialogue-based editing and natural language interaction.
quadratic
Quadratic is a modern multiplayer spreadsheet application that integrates Python, AI, and SQL functionalities. It aims to streamline team collaboration and data analysis by enabling users to pull data from various sources and utilize popular data science tools. The application supports building dashboards, creating internal tools, mixing data from different sources, exploring data for insights, visualizing Python workflows, and facilitating collaboration between technical and non-technical team members. Quadratic is built with Rust + WASM + WebGL to ensure seamless performance in the browser, and it offers features like WebGL Grid, local file management, Python and Pandas support, Excel formula support, multiplayer capabilities, charts and graphs, and team support. The tool is currently in Beta with ongoing development for additional features like JS support, SQL database support, and AI auto-complete.