cia
Citizen Intelligence Agency. Open-source intelligence platform analyzing Swedish political activities using AI and data visualization. Tracks politicians, government institutions, and parliamentary data, offering detailed insights, performance metrics, and advanced analytics.
Stars: 205
CIA is a powerful open-source tool designed for data analysis and visualization. It provides a user-friendly interface for processing large datasets and generating insightful reports. With CIA, users can easily explore data, perform statistical analysis, and create interactive visualizations to communicate findings effectively. Whether you are a data scientist, analyst, or researcher, CIA offers a comprehensive set of features to streamline your data analysis workflow and uncover valuable insights.
README:
An independent, volunteer-driven OSINT platform monitoring Swedish political activity
The Citizen Intelligence Agency is a volunteer-driven, open-source intelligence (OSINT) project that provides comprehensive analysis of political activities in Sweden. Through advanced monitoring of key political figures and institutions, we deliver:
- π Financial performance metrics
β οΈ Risk assessment analytics- π Political trend analysis
- π Politician ranking system
- π Performance comparisons
- π Transparency insights
Our initiative remains strictly independent and non-partisan, focused on fostering informed decision-making and enhancing democratic engagement.
Coverage Policy: Per Secure Development Policy, we maintain minimum 80% line coverage and 70% branch coverage across all modules.
Explore our comprehensive feature set including:
- π Interactive dashboards
- π Political scoreboard systems
- π Critical analytics tools
- π Transparency metrics
- βοΈ Accountability measures
- π± Data-driven insights
For a conceptual view of our system architecture and components, see our Architecture Documentation and System Mindmaps.
Explore in-depth technical analysis and architectural insights about this project through our team's blog posts:
We maintain a collection of custom GitHub Copilot agents specialized for different aspects of the project:
- π Task Agent - Product quality, GitHub issue management, ISMS compliance
- π οΈ Stack Specialist - Java, Spring, Vaadin, PostgreSQL expertise
- π¨ UI Enhancement Specialist - Vaadin, accessibility, data visualization
- π Intelligence Operative - Political analysis, OSINT methodologies
- π° Business Development Specialist - Strategic planning, partnerships
- π’ Marketing Specialist - Digital marketing, content strategy
See the agents README for detailed information on using these specialized profiles.
We maintain a comprehensive skills library with 41 strategic skills following security-by-design principles, organized into 13 categories:
- π‘οΈ Secure Code Review - OWASP Top 10, SAST/DAST patterns
- π― Threat Modeling - STRIDE framework, attack trees
- π Secrets Management - Never commit secrets, vault usage
- β Input Validation - Sanitization, XSS, SQL injection prevention
- π Crypto Best Practices - Encryption, hashing, key management
- π ISO 27001 Controls - Control implementation verification
- π― NIST CSF Mapping - Framework alignment
- π§ CIS Controls - Benchmark compliance
- π GDPR Compliance - Data protection requirements
- π Security Documentation - Required architecture docs
- β Unit Testing Patterns - JUnit 5, Mockito, 80% coverage
- π Integration Testing - Spring test context, database testing
- π E2E Testing - Selenium, Playwright patterns
- π Code Quality Checks - SonarCloud, CodeQL integration
- π Spring Framework Patterns - DI, transaction management, AOP
- ποΈ JPA/Hibernate Optimization - Entity design, N+1 prevention
- π¨ Vaadin Component Design - UI patterns, lifecycle management
- π C4 Architecture Documentation - Required architecture docs
- π GitHub Actions Workflows - CI/CD pipeline patterns
- π¦ Maven Build Management - Multi-module builds
- ποΈ PostgreSQL Operations - Database management
- π₯ Contribution Guidelines - PR process, code review
- π Documentation Standards - Markdown, diagrams, clarity
- π·οΈ Issue Triage Workflow - Labeling, assignment, prioritization
- π Political Science Analysis - Comparative politics, policy analysis
- π΅οΈ OSINT Methodologies - Source evaluation, data integration
- π§ Intelligence Analysis Techniques - ACH, SWOT, Red Team
- πΈπͺ Swedish Political System - Riksdag, party system, coalitions
- π Data Science for Intelligence - ML, NLP, network analysis
- π³οΈ Electoral Analysis - Election forecasting, voter behavior
- π§© Behavioral Analysis - Political psychology, cognitive biases
- π¬ Strategic Communication Analysis - Narrative, media analysis
- π Legislative Monitoring - Voting patterns, bill tracking
β οΈ Risk Assessment Frameworks - Political risk, early warnings
- π Product Management Patterns - User stories, backlog, sprint planning
- π‘ AWS CloudWatch Monitoring - Metrics, alarms, dashboards, log insights
- βΏ Accessibility WCAG Patterns - WCAG 2.1 AA, ARIA, keyboard navigation
- π Data Visualization Principles - Chart selection, color theory, dashboards
- π Playwright UI Testing - Browser automation, visual regression, E2E
- π‘ Business Model Canvas - Value proposition, revenue streams, sustainability
- π SEO Best Practices - On-page SEO, technical SEO, keyword research
Skills Integration: All 6 agents reference relevant skills for their domains. See the skills README for complete catalog and agent-skill mappings.
System Architect Simon Moon provides deep architectural analysis of the CIA platform through the lens of sacred geometry and pattern recognition:
- CIA Architecture: The Five Pentacles - Five container types crystallized from the parliamentary domain. Architecture that mirrors political realityβpower flows documented in code.
- CIA Security: Defense Through Transparency - Security through mathematical proof, not mystical obscurity. Five defensive layers. OpenSSF Scorecard 7.2/10. Zero critical vulnerabilities across 5 years.
- CIA Future Security: The Pentagon of Tomorrow - Post-quantum cryptography before quantum computers threaten. AI-augmented detection before AI attacks dominate.
- CIA Financial Strategy: $24.70/Day Democracy - Democracy costs $24.70/day when architecture channels cosmic financial patterns through AWS optimization.
- CIA Workflows: Five-Stage CI/CD & State Machines - Five GitHub Actions workflows orchestrating DevSecOps automation. Data processing through five state transitions.
- CIA Mindmaps: Conceptual Sacred Geometry - Hierarchical thinking revealing natural organizational patterns: 4 current domains expanding into 5 future dimensions.
Developer George Dorn provides hands-on code analysis based on actual repository inspection:
- CIA Code Analysis - Repository deep-dive examining Maven POMs, 49 modules, 1,372 Java files, verified OpenSSF Scorecard 7.2/10. Based on actual repository inspection.
For the complete collection of 50+ blog posts covering cybersecurity, ISMS policies, and architectural patterns, visit the Hack23 Security Blog.
- π Website: www.hack23.com
- πΌ LinkedIn: James SΓΆrling
At Hack23 AB, we believe that true security comes through transparency and demonstrable practices. Our Information Security Management System (ISMS) is publicly available, showcasing our commitment to security excellence and organizational transparency.
Our approach to cybersecurity consulting is built on a foundation of transparent practices:
- π Open Documentation: Complete ISMS framework available for review
- π Policy Transparency: Detailed security policies and procedures publicly accessible
- π― Demonstrable Expertise: Our own security implementation serves as a live demonstration
- π Continuous Improvement: Public documentation enables community feedback and enhancement
"Our commitment to transparency extends to our security practices - demonstrating that true security comes from robust processes, continuous improvement, and a culture where security considerations are integrated into every business decision."
β James Pether SΓΆrling, CEO/Founder
For a comprehensive view of how ISMS-PUBLIC policies map to CIA platform security controls:
Coverage: 32 ISMS policies β’ 100+ security controls β’ ISO 27001 β’ NIST CSF 2.0 β’ CIS Controls v8.1
Our commitment to security transparency is demonstrated through publicly verifiable evidence:
Evidence:
- OpenSSF Scorecard: Full Supply Chain Analysis
- SLSA Attestations: Build Provenance & SBOM
- CII Best Practices: Open Source Security Maturity
Evidence:
- CodeQL Scanning: Security Code Analysis Results
- SonarCloud Security: Vulnerability Dashboard
- Dependabot: Dependency Vulnerability Alerts
Evidence:
- Quality Gate: SonarCloud Quality Dashboard
- Code Metrics: Technical Debt & Maintainability
Evidence:
- FOSSA Analysis: License Compliance Report
- License: Apache License 2.0
Evidence:
- Threat Model: STRIDE Analysis & Attack Trees
- Security Architecture: Defense-in-Depth Design
- CRA Assessment: EU Cyber Resilience Act Compliance
π Transparency Commitment: All security evidence is publicly accessible for stakeholder verification. We believe security through transparency builds trust and demonstrates our commitment to cybersecurity excellence.
Our analysis is powered by authoritative Swedish government and international data sources:
| Source | Description |
|---|---|
| ποΈ Swedish Parliament Open Data | Parliamentary members, committees, and official documents |
| π³οΈ Swedish Election Authority | Election data, political parties, and voting results |
| π World Bank Open Data | Global economic indicators and demographic data |
| πΉ Swedish Financial Management Authority | Government finances and economic trends |
For comprehensive security and compliance evidence, see Security & Compliance Evidence section above.
| JDK Version | Status | Release Info |
|---|---|---|
| Supported | LTS Release | |
| Compatible | Feature Release | |
| Compatible | Feature Release | |
| Compatible | Feature Release | |
| Supported | Current LTS |
For details on our technology lifecycle management, see the End-of-Life Strategy.
| Document | Focus | Description | Documentation Link |
|---|---|---|---|
| Architecture | ποΈ Architecture | C4 model showing current system structure | View Source |
| Future Architecture | ποΈ Architecture | C4 model showing future system structure | View Source |
| Security Architecture | π Security | Security architecture | View Source |
| Future Security Architecture | π Security | Future Security architecture | View Source |
| Mindmaps | π§ Concept | Current system component relationships | View Source |
| Future Mindmaps | π§ Concept | Future capability evolution | View Source |
| SWOT Analysis | πΌ Business | Current strategic assessment | View Source |
| Future SWOT Analysis | πΌ Business | Future strategic opportunities | View Source |
| Data Model | π Data | Current data structures and relationships | View Source |
| Future Data Model | π Data | Enhanced political data architecture | View Source |
| Flowcharts | π Process | Current data processing workflows | View Source |
| Future Flowcharts | π Process | Enhanced AI-driven workflows | View Source |
| State Diagrams | π Behavior | Current system state transitions | View Source |
| Future State Diagrams | π Behavior | Enhanced adaptive state transitions | View Source |
| CI/CD Workflows | π§ DevOps | Current automation processes | View Source |
| Future Workflows | π§ DevOps | Enhanced CI/CD with ML | View Source |
| End-of-Life Strategy | π Lifecycle | Maintenance and EOL planning | View Source |
| Financial Security Plan | π° Security | Cost and security implementation | View Source |
| CIA Features | π Features | Platform features overview | View on hack23.com |
| Threat Model | π‘οΈ Security | STRIDE / MITRE risk analysis | View Source |
| Unit Test Plan | π§ͺ Testing | Comprehensive testing strategy & coverage | View Source |
| Documentation Naming Convention | π Standards | Naming standards for new documentation | View Source |
The CIA platform provides comprehensive intelligence operations (INTOP) and open-source intelligence (OSINT) capabilities. Our intelligence documentation tracks the evolution of analytical frameworks, risk assessment rules, and database views that power political intelligence products.
Unified tracking of intelligence evolution across all capabilities:
| Document | Focus | Description | Documentation Link |
|---|---|---|---|
| Intelligence Evolution Changelog | π― Unified | Comprehensive tracking of intelligence capabilities, database views, risk rules, and analytical frameworks | View Source |
Historical Changelogs (Archived): Intelligence Analysis, Database Views, Risk Rules
Comprehensive documentation of analytical capabilities and methodologies:
| Document | Focus | Description | Documentation Link |
|---|---|---|---|
| Data Analysis - INTOP OSINT | π― Frameworks | 6 analysis frameworks (Temporal, Comparative, Pattern Recognition, Predictive, Network, Decision) | View Source |
| Risk Rules Documentation | π΄ Risk Rules | 50 behavioral detection rules (24 politician, 10 party, 4 committee, 4 ministry, 5 decision, 3 other) | View Source |
| Database View Intelligence Catalog | ποΈ Views | Complete catalog of 85 database views (57 regular + 28 materialized) | View Source |
| Data Quality Monitoring Dashboard | π Quality | Unified data quality monitoring with OSINT, database health, and view validation metrics | View Source |
| Intelligence Data Flow Map | πΊοΈ Pipeline | Data pipeline mappings and framework-to-view relationships | View Source |
| Liquibase Intelligence Analysis | ποΈ Schema | Database schema evolution from intelligence perspective | View Source |
| Tool | Purpose | Location |
|---|---|---|
| Intelligence Changelog Generator | Automated detection of view changes, risk rule updates, and framework enhancements | Script |
| GitHub Actions Workflow | Automated changelog generation on demand | Workflow |
Usage:
# Generate intelligence changelog from recent changes
.github/scripts/generate-intelligence-changelog.sh
# Compare specific commits
.github/scripts/generate-intelligence-changelog.sh <prev_commit> <current_commit>| Category | Count | Description |
|---|---|---|
| Analysis Frameworks | 6 | Temporal, Comparative, Pattern Recognition, Predictive, Network, Decision Intelligence |
| Risk Rules | 50 | 24 politician + 10 party + 4 committee + 4 ministry + 5 decision + 3 other |
| Database Views | 85 | 57 regular views + 28 materialized views |
| OSINT Data Sources | 4 | Riksdagen API, Election Authority, World Bank, Financial Authority |
| Intelligence Products | 10+ | Scorecards, Coalition Analysis, Risk Assessments, Trend Reports, Decision Tracking |
Navigate intelligence documentation efficiently based on your role:
|
Goal: Find views and analytical capabilities
Key Use Cases:
|
Goal: Understand complete intelligence pipeline
Key Use Cases:
|
|
Goal: Maintain schema and optimize performance
Key Use Cases:
|
Goal: Understand capabilities and product features
Key Use Cases:
|
Quick Links:
- πΊοΈ Intelligence Data Flow Map - Central navigation hub
- π Intelligence Evolution Changelog - Capability tracking over time
- π OSINT Data Sources - External API integrations
- π Intelligence Metrics - Current capability counts
Please follow the instructions in our SECURITY.md file for reporting security issues.
This document provides a high-level overview of the key technologies used within the Citizen Intelligence Agency (CIA) project. Each technology plays a vital role in supporting CIAβs goals for data analysis, security, and scalability within the political intelligence domain.
| Category | Technologies |
|---|---|
| Core Framework | Spring Framework |
| Security | Spring Security, Bouncy Castle |
| Data Access | Hibernate, JPA, PostgreSQL, JDBC |
| Transaction Management |
Narayana (Integrated with Spring JpaTransactionManager) |
| Data Auditing | Javers |
| Business Rules Engine | Drools |
| Messaging | ActiveMQ Artemis, Spring JMS |
| Web/UI Layer | Vaadin, Vaadin Sass Compiler, Vaadin Themes |
| Monitoring | JavaMelody, AWS SDK for CloudWatch |
| Testing | JUnit, Mockito, Spring Test, Selenium WebDriver |
| Utilities | Apache Commons, Google Guava, SLF4J, Logback, Jackson |
| Build & Dependency Management | Maven |
This stack comprises:
- Core Framework: The project uses Spring Framework to provide a foundation for dependency injection, component management, and service configuration across modules.
- Security: Spring Security manages authentication and authorization, complemented by Bouncy Castle for cryptographic operations.
- Data Access: A combination of Hibernate, JPA, and PostgreSQL supports robust ORM-based data persistence, with JDBC facilitating additional database connectivity needs.
- Transaction Management: The project uses Narayana as the transaction manager implementation, integrated with Springβs JpaTransactionManager for distributed transaction support and ensuring transactional integrity.
- Data Auditing: Javers provides auditing and historical versioning, allowing for tracking and comparing changes to data over time.
- Business Rules Engine : Drools is integrated into the CIA project to enable a robust business rules engine.
- Messaging: ActiveMQ Artemis and Spring JMS enable asynchronous communication between application components, supporting distributed and event-driven designs.
- Web/UI Layer: Vaadin powers the UI with a server-driven architecture, providing components like Vaadin Themes and Sass Compiler for a rich, interactive frontend experience directly in Java.
- Monitoring: JavaMelody and AWS SDK for CloudWatch provide real-time application monitoring and logging capabilities, supporting both local and cloud environments.
- Testing: JUnit, Mockito, Spring Test and Selenium WebDriver are used extensively for unit, integration, system, browser and mock testing to ensure application reliability and robustness.
- Utilities: Apache Commons, Google Guava, SLF4J, and Logback offer utility functions and structured logging, enhancing application maintainability and monitoring.
- Build & Dependency Management: Maven handles project builds, dependency management, and plugin configurations, enabling smooth project management and modular builds.
This document provides a comprehensive summary of the AWS services utilized in the Citizen Intelligence Agency (CIA) project infrastructure, as defined by its CloudFormation template. These services work together to ensure a secure, resilient, and scalable deployment environment.
| Category | AWS Services | NIST CSF Function, Category & Subcategory | ISO 27001:2022 Control & Link |
|---|---|---|---|
| Networking and Security | - Amazon VPC: Configures a custom network environment with public/private subnets, route tables, NAT Gateway, Network ACLs (NACLs) for traffic control, and VPC Flow Logs. - VPC Endpoints: Enables private access to AWS services (e.g., S3, EC2, SSM, CloudWatch Logs). - AWS WAF: Protects against web attacks at the ALB layer. - AWS IAM: Manages role-based access control. - AWS KMS: Manages encryption for data at rest. |
Identify (ID): - Asset Management (ID.AM-2) Protect (PR): - Access Control (PR.AC-1, PR.AC-3, PR.AC-5) - Data Security (PR.DS-1, PR.DS-2) - Protective Technology (PR.PT-3) Detect (DE): - Security Continuous Monitoring (DE.CM-3) |
- A.8.1: Asset management - A.9.4.1: Access control policy - A.13.1.1: Network controls - A.13.1.3: Segregation in networks - A.18.1.5: Regulation and compliance (see ISO 27001) |
| Domain and SSL Management | - Amazon Route 53: Manages domain registration and DNS routing. - AWS Certificate Manager (ACM): Issues and manages SSL/TLS certificates. |
Protect (PR): - Data Security (PR.DS-5) Detect (DE): - Anomalies and Events (DE.AE-3) |
- A.10.1.1: Cryptographic controls for data protection - A.12.4.3: Security of network services |
| Compute | - Amazon EC2: Provides scalable compute instances. |
Protect (PR): - Protective Technology (PR.PT-1) Respond (RS): - Analysis (RS.AN-1), Mitigation (RS.MI-2) |
- A.12.1.3: Capacity management for IT infrastructure and services |
| Load Balancing | - Application Load Balancer (ALB): Distributes HTTP/HTTPS traffic across EC2 instances. |
Protect (PR): - Protective Technology (PR.PT-3) Respond (RS): - Communications (RS.CO-2) |
- A.13.1.1: Network controls - A.13.2.1: Information transfer policies |
| Data Storage | - Amazon S3: Stores application artifacts and logs with encryption, access control, and lifecycle policies. - Amazon RDS: PostgreSQL database with multi-AZ deployment. |
Protect (PR): - Data Security (PR.DS-1, PR.DS-5) - Information Protection Processes and Procedures (PR.IP-3, PR.IP-4) - Maintenance (PR.MA-1) Recover (RC): - Recovery Planning (RC.RP-1), Communications (RC.CO-2) |
- A.8.2.3: Information backup - A.10.1.1: Use of cryptographic controls |
| Secrets Management | - AWS Secrets Manager: Securely stores and rotates sensitive credentials with Lambda rotation support. |
Protect (PR): - Access Control (PR.AC-1, PR.AC-4) - Data Security (PR.DS-6) - Identity Management and Access Control (PR.AC-7) |
- A.9.2.2: User access provisioning - A.10.1.1: Management of encryption keys and secret information |
| Monitoring and Alarms | - Amazon CloudWatch: Provides real-time metrics, logs, and alarms to monitor performance and health. |
Detect (DE): - Security Continuous Monitoring (DE.CM-3) |
- A.12.4.1: Monitoring activities |
| Resilience and Disaster Recovery | - AWS Resilience Hub: Assesses and improves the architectureβs resilience, recommending strategies for fault tolerance and disaster recovery. |
Recover (RC): - Recovery Planning (RC.RP-1) - Improvements (RC.IM-1) |
- A.17.1.2: Implementing continuity controls - A.17.2.1: Availability of information processing facilities |
| Automation and Maintenance | - AWS Systems Manager (SSM): Automates inventory, patching, and maintenance tasks, with SSM Maintenance Windows and SSM Patch Baselines for streamlined operations. |
Protect (PR): - Maintenance (PR.MA-1, PR.MA-2) - Protective Technology (PR.PT-1) |
- A.12.6.1: Control of technical vulnerabilities - A.12.7.1: Information systems audit considerations |
-
Networking and Security: Amazon VPC creates an isolated network environment with NAT Gateway, NACLs, and VPC Flow Logs. VPC Endpoints provide private access to AWS services (e.g., S3, EC2, SSM), AWS WAF protects against web attacks, AWS IAM secures access control, and AWS KMS encrypts data at rest.
-
Domain and SSL Management: Amazon Route 53 handles DNS and domain registration, while AWS Certificate Manager (ACM) provides SSL/TLS certificates for HTTPS security.
-
Compute Layer: Amazon EC2 instances host the application, providing flexible and scalable compute resources.
-
Load Balancing: The Application Load Balancer (ALB) distributes HTTP/HTTPS traffic across EC2 instances, optimizing for high availability and resilience.
-
Data Storage: Amazon RDS offers a resilient PostgreSQL setup with multi-AZ deployment and custom parameter groups. Amazon S3 securely stores artifacts and logs, with lifecycle policies and KMS-managed encryption keys for compliance.
-
Secrets Management: AWS Secrets Manager securely stores and rotates credentials, such as database passwords, with automated Lambda support for rotation.
-
Monitoring and Alarms: Amazon CloudWatch monitors infrastructure health through metrics, logs, and alarms, enabling proactive management.
-
Resilience and Disaster Recovery: AWS Resilience Hub assesses and recommends enhancements to improve the system's resilience, providing disaster recovery and fault-tolerant strategies.
-
Automation and Maintenance: AWS Systems Manager (SSM) automates inventory, patching, and other maintenance tasks, increasing operational efficiency.
For detailed security implementation, see the Financial Security Plan.
The Citizen Intelligence Agency can be deployed on AWS using our provided CloudFormation template:
- Download the CloudFormation stack file
- Create a new stack in the AWS CloudFormation console
- Upload the template file and configure parameters
- Acknowledge IAM resource creation and launch the stack
- Access the application via the URL in the stack outputs
For local or self-hosted deployment on Debian/Ubuntu 24.4+:
-
Install prerequisites:
sudo apt-get install openjdk-21-jdk postgresql-16 postgresql-contrib postgresql-16-pgaudit postgresql-16-pgvector
-
Configure PostgreSQL as detailed below.
A step-by-step guide to configure PostgreSQL 16 with SSL, prepared transactions, and required extensions.
-
Edit
/etc/postgresql/16/main/postgresql.confand add or update the following lines:max_prepared_transactions = 100 shared_preload_libraries = 'pg_stat_statements, pgaudit, pgcrypto' pgaudit.log = ddl pg_stat_statements.track = all pg_stat_statements.max = 10000
- Save and close the file.
-
Edit
/etc/postgresql/16/main/pg_hba.confand add the following line:host all all ::1/128 md5
- Save and close the file.
-
Generate a secure random passphrase:
openssl rand -base64 48 > passphrase.txt -
Create a passphrase-protected private key:
openssl genrsa -des3 -passout file:passphrase.txt -out server.pass.key 2048
-
Remove the passphrase protection from the private key:
openssl rsa -passin file:passphrase.txt -in server.pass.key -out server.key rm server.pass.key
-
Create a Certificate Signing Request (CSR):
openssl req -new -key server.key -out server.csr \ -subj "/C=UK/ST=Postgresqll/L=Docker/O=Hack23/OU=demo/CN=127.0.0.1" -
Self-sign the certificate (valid for 10 years / 3650 days):
openssl x509 -req -days 3650 -in server.csr -signkey server.key -out server.crt
-
Clean up temporary files:
rm passphrase.txt rm server.csr
-
Copy the new certificate and key into the PostgreSQL data directory:
cp server.crt /var/lib/postgresql/16/main/server.crt cp server.key /var/lib/postgresql/16/main/server.key rm server.key
-
Secure the certificate and key:
chmod 700 /var/lib/postgresql/16/main/server.key chmod 700 /var/lib/postgresql/16/main/server.crt chown -R postgres:postgres /var/lib/postgresql/16/main/
-
Enable SSL in PostgreSQL by adding the following lines to
/etc/postgresql/16/main/postgresql.conf:echo "ssl_cert_file = '/var/lib/postgresql/16/main/server.crt'" \ >> /etc/postgresql/16/main/postgresql.conf echo "ssl_key_file = '/var/lib/postgresql/16/main/server.key'" \ >> /etc/postgresql/16/main/postgresql.conf
-
Create a
.postgresqldirectory for theciauser:mkdir -p /opt/cia/.postgresql
-
Copy the server certificate into this directory:
cp server.crt /opt/cia/.postgresql/root.crt chmod 700 /opt/cia/.postgresql/root.crt chown -R cia:cia /opt/cia/.postgresql/root.crt
-
Remove the server certificate from the current directory (if desired):
rm server.crt
For optimal performance with the CIA platform's 85+ views and 93 tables, add the following settings to /etc/postgresql/16/main/postgresql.conf. Values should be adjusted based on your server's available RAM.
Configure memory settings proportionally to your system RAM:
| Setting | 4GB RAM | 8GB RAM | 16GB+ RAM (Production) |
|---|---|---|---|
shared_buffers |
1GB | 2GB | 4GB |
effective_cache_size |
3GB | 6GB | 12GB |
maintenance_work_mem |
256MB | 512MB | 1GB |
work_mem |
16MB | 32MB | 50MB |
Apply settings using SQL commands (use values from the table above for your RAM configuration):
-- Example for 8GB RAM server - adjust values from the table above for your configuration
-- shared_buffers: ~25% of RAM
-- effective_cache_size: ~75% of RAM
-- maintenance_work_mem: For VACUUM, CREATE INDEX operations
-- work_mem: Per-operation memory for sorts, joins
ALTER SYSTEM SET shared_buffers = '2GB';
ALTER SYSTEM SET effective_cache_size = '6GB';
ALTER SYSTEM SET maintenance_work_mem = '512MB';
ALTER SYSTEM SET work_mem = '32MB';Configure checkpoint settings for optimal write performance:
ALTER SYSTEM SET checkpoint_completion_target = 0.9;
ALTER SYSTEM SET wal_buffers = '16MB';
ALTER SYSTEM SET max_wal_size = '4GB';
ALTER SYSTEM SET min_wal_size = '1GB';For SSD storage (recommended), optimize query planning:
ALTER SYSTEM SET random_page_cost = 1.1; -- For SSD storage
ALTER SYSTEM SET effective_io_concurrency = 200; -- For SSD storageConfigure connection limits:
ALTER SYSTEM SET max_connections = 200;After making changes, apply them:
# Reload configuration (for settings that don't require restart)
sudo -u postgres psql -c "SELECT pg_reload_conf();"
# For settings requiring restart (shared_buffers, max_connections):
sudo systemctl restart postgresqlConfirm settings are applied:
-- Check current settings
SHOW shared_buffers;
SHOW effective_cache_size;
SHOW work_mem;
SHOW maintenance_work_mem;
SHOW checkpoint_completion_target;
SHOW random_page_cost;
SHOW max_connections;π Note: For detailed performance tuning guidelines, database health monitoring, and advanced configuration options, see service.data.impl/README-SCHEMA-MAINTENANCE.md.
-
Restart PostgreSQL to apply all changes:
systemctl restart postgresql
-
Verify that PostgreSQL is running with SSL by checking the logs or using an SSL-enabled client.
-
Confirm that prepared transactions and required extensions are enabled:
SHOW max_prepared_transactions; \dx
-
Confirm the new IPv6 entry in
pg_hba.confis functioning as expected by connecting viapsqlover::1.
Create an empty database:
Below instructions set the default username/password and database name used for development. We recommend using custom credentials and updating the configuration at /opt/cia/webapps/cia/WEB-INF/database.properties to define your own username/password and database name.
$ sudo su - postgres
$ psql
postgres=# CREATE USER eris WITH password 'discord';
postgres=# CREATE DATABASE cia_dev;
postgres=# GRANT ALL PRIVILEGES ON DATABASE cia_dev to eris;-
Download the CIA Debian package:
wget https://github.com/Hack23/cia/releases/download/2025.1.2/cia-dist-deb-2025.1.2.all.deb
-
Install the Debian package:
sudo dpkg -i cia-dist-deb-2025.1.2.all.deb
-
Access the server at https://localhost:28443/cia/.
-
English: Our dashboard provides comprehensive analytics on Swedish political figures and institutions.
-
Swedish: VΓ₯r dashboard erbjuder en detaljerad ΓΆversikt ΓΆver politiska figurer och olika departement i Sverige.
This project is powered by advanced AI technologies for data processing and analysis. We integrate data from various open sources and visualize findings through modern data visualization tools.
For our future vision incorporating more advanced AI capabilities, see our Future Architecture Vision.
| Impact Category | Financial | Operational | Reputational | Regulatory |
|---|---|---|---|---|
| π Confidentiality | ||||
| β Integrity | ||||
| β±οΈ Availability |
- ποΈ Architecture Documentation - C4 model system architecture
- π§ System Mindmaps - Conceptual overview and component relationships
- π Future Architecture Vision - AI-enhanced capabilities roadmap
- π Data Model - Database schema and entity relationships
- ποΈ Entity Model Documentation - Detailed database entity reference
- π API Documentation - Complete API reference
- π¦ Package Dependencies - Visual code package structure
- π Security Architecture - Defense-in-depth security implementation
- π Future Security Architecture - Advanced security capabilities roadmap
- π― Threat Model - STRIDE/MITRE ATT&CK threat analysis
- π ISMS Compliance Mapping - Complete ISMS-PUBLIC policy integration
- π° Financial Security Plan - AWS security deployment and costs
- π CRA Assessment - EU Cyber Resilience Act compliance
- π Security Policy - Vulnerability disclosure and security reporting
- β‘ Workflows - CI/CD pipelines and DevSecOps automation
- π End-of-Life Strategy - Technology maintenance and lifecycle planning
- π§ͺ Unit Test Plan - Testing strategy and coverage requirements
- π E2E Test Plan - End-to-end testing documentation
- π€ Contributing Guidelines - Development contribution guide
- π Code of Conduct - Community standards and expectations
- β¨ CIA Features Showcase - Comprehensive feature demonstrations
- π Political Dashboard (English) - Swedish political analytics
- π Politisk Dashboard (Svenska) - Svensk politisk analys
- π€ GitHub Copilot Instructions - AI-assisted development guidelines
- π₯ Custom Copilot Agents - Specialized AI agents for different project aspects
These policies from Hack23 AB's public ISMS govern this project's security and development practices:
- π οΈ Secure Development Policy - SDLC security requirements (80% coverage minimum)
- π Information Security Policy - Overall security governance framework
- π·οΈ Classification Framework - Data classification and business impact analysis
- π― Threat Modeling Policy - STRIDE framework and threat analysis methodology
- π Vulnerability Management - Security testing and remediation processes
- π¨ Incident Response Plan - Security incident handling procedures
- π Network Security Policy - Network protection and segmentation
- π Access Control Policy - Identity and access management
- π Cryptography Policy - Encryption standards and key management
- πΎ Backup & Recovery Policy - Data protection and recovery procedures
For complete ISMS framework, visit:
π Project Classification:
π·οΈ Classification:
π Project Type: Data Analytics Platform (OSINT) | βοΈ Process Type: Operations
π― Framework Compliance:
For Tasks:
Click tags to check more tools for each tasksFor Jobs:
Alternative AI tools for cia
Similar Open Source Tools
cia
CIA is a powerful open-source tool designed for data analysis and visualization. It provides a user-friendly interface for processing large datasets and generating insightful reports. With CIA, users can easily explore data, perform statistical analysis, and create interactive visualizations to communicate findings effectively. Whether you are a data scientist, analyst, or researcher, CIA offers a comprehensive set of features to streamline your data analysis workflow and uncover valuable insights.
claude-code-ultimate-guide
The Claude Code Ultimate Guide is an exhaustive documentation resource that takes users from beginner to power user in using Claude Code. It includes production-ready templates, workflow guides, a quiz, and a cheatsheet for daily use. The guide covers educational depth, methodologies, and practical examples to help users understand concepts and workflows. It also provides interactive onboarding, a repository structure overview, and learning paths for different user levels. The guide is regularly updated and offers a unique 257-question quiz for comprehensive assessment. Users can also find information on agent teams coverage, methodologies, annotated templates, resource evaluations, and learning paths for different roles like junior developer, senior developer, power user, and product manager/devops/designer.
awesome-LangGraph
Awesome LangGraph is a curated list of projects, resources, and tools for building stateful, multi-actor applications with LangGraph. It provides valuable resources for developers at all stages of development, from beginners to those building production-ready systems. The repository covers core ecosystem components, LangChain ecosystem, LangGraph platform, official resources, starter templates, pre-built agents, example applications, development tools, community projects, AI assistants, content & media, knowledge & retrieval, finance & business, sustainability, learning resources, companies using LangGraph, contributing guidelines, and acknowledgments.
GraphGen
GraphGen is a framework for synthetic data generation guided by knowledge graphs. It enhances supervised fine-tuning for large language models (LLMs) by generating synthetic data based on a fine-grained knowledge graph. The tool identifies knowledge gaps in LLMs, prioritizes generating QA pairs targeting high-value knowledge, incorporates multi-hop neighborhood sampling, and employs style-controlled generation to diversify QA data. Users can use LLaMA-Factory and xtuner for fine-tuning LLMs after data generation.
monoscope
Monoscope is an open-source monitoring and observability platform that uses artificial intelligence to understand and monitor systems automatically. It allows users to ingest and explore logs, traces, and metrics in S3 buckets, query in natural language via LLMs, and create AI agents to detect anomalies. Key capabilities include universal data ingestion, AI-powered understanding, natural language interface, cost-effective storage, and zero configuration. Monoscope is designed to reduce alert fatigue, catch issues before they impact users, and provide visibility across complex systems.
Awesome-Lists-and-CheatSheets
Awesome-Lists is a curated index of selected resources spanning various fields including programming languages and theories, web and frontend development, server-side development and infrastructure, cloud computing and big data, data science and artificial intelligence, product design, etc. It includes articles, books, courses, examples, open-source projects, and more. The repository categorizes resources according to the knowledge system of different domains, aiming to provide valuable and concise material indexes for readers. Users can explore and learn from a wide range of high-quality resources in a systematic way.
gitmesh
GitMesh is an AI-powered Git collaboration network designed to address contributor dropout in open source projects. It offers real-time branch-level insights, intelligent contributor-task matching, and automated workflows. The platform transforms complex codebases into clear contribution journeys, fostering engagement through gamified rewards and integration with open source support programs. GitMesh's mascot, Meshy/Mesh Wolf, symbolizes agility, resilience, and teamwork, reflecting the platform's ethos of efficiency and power through collaboration.
new-api
New API is a next-generation large model gateway and AI asset management system that provides a wide range of features, including a new UI interface, multi-language support, online recharge function, key query for usage quota, compatibility with the original One API database, model charging by usage count, channel weighted randomization, data dashboard, token grouping and model restrictions, support for various authorization login methods, support for Rerank models, OpenAI Realtime API, Claude Messages format, reasoning effort setting, content reasoning, user-specific model rate limiting, request format conversion, cache billing support, and various model support such as gpts, Midjourney-Proxy, Suno API, custom channels, Rerank models, Claude Messages format, Dify, and more.
prism-insight
PRISM-INSIGHT is a comprehensive stock analysis and trading simulation system based on AI agents. It automatically captures daily surging stocks via Telegram channel, generates expert-level analyst reports, and performs trading simulations. The system utilizes OpenAI GPT-4.1 for in-depth stock analysis and GPT-5 for investment strategy simulation. It also interacts with users via Anthropic Claude for Telegram conversations. The system architecture includes AI analysis agents, stock tracking, PDF conversion, and Telegram bot functionalities. Users can customize criteria for identifying surging stocks, modify AI prompts, and adjust chart styles. The project is open-source under the MIT license, and all investment decisions based on the analysis are the responsibility of the user.
Awesome-Lists
Awesome-Lists is a curated list of awesome lists across various domains of computer science and beyond, including programming languages, web development, data science, and more. It provides a comprehensive index of articles, books, courses, open source projects, and other resources. The lists are organized by topic and subtopic, making it easy to find the information you need. Awesome-Lists is a valuable resource for anyone looking to learn more about a particular topic or to stay up-to-date on the latest developments in the field.
helicone
Helicone is an open-source observability platform designed for Language Learning Models (LLMs). It logs requests to OpenAI in a user-friendly UI, offers caching, rate limits, and retries, tracks costs and latencies, provides a playground for iterating on prompts and chat conversations, supports collaboration, and will soon have APIs for feedback and evaluation. The platform is deployed on Cloudflare and consists of services like Web (NextJs), Worker (Cloudflare Workers), Jawn (Express), Supabase, and ClickHouse. Users can interact with Helicone locally by setting up the required services and environment variables. The platform encourages contributions and provides resources for learning, documentation, and integrations.
matrixone
MatrixOne is the industry's first database to bring Git-style version control to data, combined with MySQL compatibility, AI-native capabilities, and cloud-native architecture. It is a HTAP (Hybrid Transactional/Analytical Processing) database with a hyper-converged HSTAP engine that seamlessly handles transactional, analytical, full-text search, and vector search workloads in a single unified systemβno data movement, no ETL, no compromises. Manage your database like code with features like instant snapshots, time travel, branch & merge, instant rollback, and complete audit trail. Built for the AI era, MatrixOne is MySQL-compatible, AI-native, and cloud-native, offering storage-compute separation, elastic scaling, and Kubernetes-native deployment. It serves as one database for everything, replacing multiple databases and ETL jobs with native OLTP, OLAP, full-text search, and vector search capabilities.
anylabeling
AnyLabeling is a tool for effortless data labeling with AI support from YOLO and Segment Anything. It combines features from LabelImg and Labelme with an improved UI and auto-labeling capabilities. Users can annotate images with polygons, rectangles, circles, lines, and points, as well as perform auto-labeling using YOLOv5 and Segment Anything. The tool also supports text detection, recognition, and Key Information Extraction (KIE) labeling, with multiple language options available such as English, Vietnamese, and Chinese.
intlayer
Intlayer is an open-source, flexible i18n toolkit with AI-powered translation and CMS capabilities. It is a modern i18n solution for web and mobile apps, framework-agnostic, and includes features like per-locale content files, TypeScript autocompletion, tree-shakable dictionaries, and CI/CD integration. With Intlayer, internationalization becomes faster, cleaner, and smarter, offering benefits such as cross-framework support, JavaScript-powered content management, simplified setup, enhanced routing, AI-powered translation, and more.
bitcart
Bitcart is a platform designed for merchants, users, and developers, providing easy setup and usage. It includes various linked repositories for core daemons, admin panel, ready store, Docker packaging, Python library for coins connection, BitCCL scripting language, documentation, and official site. The platform aims to simplify the process for merchants and developers to interact and transact with cryptocurrencies, offering a comprehensive ecosystem for managing transactions and payments.
VideoRefer
VideoRefer Suite is a tool designed to enhance the fine-grained spatial-temporal understanding capabilities of Video Large Language Models (Video LLMs). It consists of three primary components: Model (VideoRefer) for perceiving, reasoning, and retrieval for user-defined regions at any specified timestamps, Dataset (VideoRefer-700K) for high-quality object-level video instruction data, and Benchmark (VideoRefer-Bench) to evaluate object-level video understanding capabilities. The tool can understand any object within a video.
For similar tasks
aimeos-core
Aimeos is an Open Source e-commerce framework for online shops consisting of the e-commerce library, the administration interface and different front-ends. It offers a modular stack that provides flexibility and speed. Unlike other shop systems, Aimeos allows users to choose from several user front-ends and customize them according to their needs or create their own. It is suitable for medium to large businesses requiring seamless integration into existing systems like content management, customer relationship management, or enterprise resource planning systems. Aimeos also serves as a base for portals or marketplaces.
qrev
QRev is an open-source alternative to Salesforce, offering AI agents to scale sales organizations infinitely. It aims to provide digital workers for various sales roles or a superagent named Qai. The tech stack includes TypeScript for frontend, NodeJS for backend, MongoDB for app server database, ChromaDB for vector database, SQLite for AI server SQL relational database, and Langchain for LLM tooling. The tool allows users to run client app, app server, and AI server components. It requires Node.js and MongoDB to be installed, and provides detailed setup instructions in the README file.
sktime
sktime is a Python library for time series analysis that provides a unified interface for various time series learning tasks such as classification, regression, clustering, annotation, and forecasting. It offers time series algorithms and tools compatible with scikit-learn for building, tuning, and validating time series models. sktime aims to enhance the interoperability and usability of the time series analysis ecosystem by empowering users to apply algorithms across different tasks and providing interfaces to related libraries like scikit-learn, statsmodels, tsfresh, PyOD, and fbprophet.
pandas-ai
PandaAI is a Python platform that enables users to interact with their data in natural language, catering to both non-technical and technical users. It simplifies data querying and analysis, offering conversational data analytics capabilities with minimal code. Users can ask questions, visualize charts, and compare dataframes effortlessly. The tool aims to streamline data exploration and decision-making processes by providing a user-friendly interface for data manipulation and analysis.
cia
CIA is a powerful open-source tool designed for data analysis and visualization. It provides a user-friendly interface for processing large datasets and generating insightful reports. With CIA, users can easily explore data, perform statistical analysis, and create interactive visualizations to communicate findings effectively. Whether you are a data scientist, analyst, or researcher, CIA offers a comprehensive set of features to streamline your data analysis workflow and uncover valuable insights.
aimeos-headless
Aimeos headless distribution is an ultra-fast, cloud-native, and API-first headless ecommerce solution for Laravel. It offers a full-featured e-commerce package with features like JSON REST API, GraphQL API, multi-vendor support, subscriptions, block/tier pricing, admin backend, and more. The distribution is highly customizable, extensible, and suitable for multi-tenant e-commerce SaaS solutions. It supports multiple languages, AI-based text translation, and provides secure and high-quality source code. Aimeos is designed for AWS, Google, Azure, and Kubernetes based clouds, and can handle a wide range of products efficiently.
datasets
Datasets is a repository that provides a collection of various datasets for machine learning and data analysis projects. It includes datasets in different formats such as CSV, JSON, and Excel, covering a wide range of topics including finance, healthcare, marketing, and more. The repository aims to help data scientists, researchers, and students access high-quality datasets for training models, conducting experiments, and exploring data analysis techniques.
For similar jobs
databerry
Chaindesk is a no-code platform that allows users to easily set up a semantic search system for personal data without technical knowledge. It supports loading data from various sources such as raw text, web pages, files (Word, Excel, PowerPoint, PDF, Markdown, Plain Text), and upcoming support for web sites, Notion, and Airtable. The platform offers a user-friendly interface for managing datastores, querying data via a secure API endpoint, and auto-generating ChatGPT Plugins for each datastore. Chaindesk utilizes a Vector Database (Qdrant), Openai's text-embedding-ada-002 for embeddings, and has a chunk size of 1024 tokens. The technology stack includes Next.js, Joy UI, LangchainJS, PostgreSQL, Prisma, and Qdrant, inspired by the ChatGPT Retrieval Plugin.
OAD
OAD is a powerful open-source tool for analyzing and visualizing data. It provides a user-friendly interface for exploring datasets, generating insights, and creating interactive visualizations. With OAD, users can easily import data from various sources, clean and preprocess data, perform statistical analysis, and create customizable visualizations to communicate findings effectively. Whether you are a data scientist, analyst, or researcher, OAD can help you streamline your data analysis workflow and uncover valuable insights from your data.
sqlcoder
Defog's SQLCoder is a family of state-of-the-art large language models (LLMs) designed for converting natural language questions into SQL queries. It outperforms popular open-source models like gpt-4 and gpt-4-turbo on SQL generation tasks. SQLCoder has been trained on more than 20,000 human-curated questions based on 10 different schemas, and the model weights are licensed under CC BY-SA 4.0. Users can interact with SQLCoder through the 'transformers' library and run queries using the 'sqlcoder launch' command in the terminal. The tool has been tested on NVIDIA GPUs with more than 16GB VRAM and Apple Silicon devices with some limitations. SQLCoder offers a demo on their website and supports quantized versions of the model for consumer GPUs with sufficient memory.
TableLLM
TableLLM is a large language model designed for efficient tabular data manipulation tasks in real office scenarios. It can generate code solutions or direct text answers for tasks like insert, delete, update, query, merge, and chart operations on tables embedded in spreadsheets or documents. The model has been fine-tuned based on CodeLlama-7B and 13B, offering two scales: TableLLM-7B and TableLLM-13B. Evaluation results show its performance on benchmarks like WikiSQL, Spider, and self-created table operation benchmark. Users can use TableLLM for code and text generation tasks on tabular data.
mlcraft
Synmetrix (prev. MLCraft) is an open source data engineering platform and semantic layer for centralized metrics management. It provides a complete framework for modeling, integrating, transforming, aggregating, and distributing metrics data at scale. Key features include data modeling and transformations, semantic layer for unified data model, scheduled reports and alerts, versioning, role-based access control, data exploration, caching, and collaboration on metrics modeling. Synmetrix leverages Cube (Cube.js) for flexible data models that consolidate metrics from various sources, enabling downstream distribution via a SQL API for integration into BI tools, reporting, dashboards, and data science. Use cases include data democratization, business intelligence, embedded analytics, and enhancing accuracy in data handling and queries. The tool speeds up data-driven workflows from metrics definition to consumption by combining data engineering best practices with self-service analytics capabilities.
data-scientist-roadmap2024
The Data Scientist Roadmap2024 provides a comprehensive guide to mastering essential tools for data science success. It includes programming languages, machine learning libraries, cloud platforms, and concepts categorized by difficulty. The roadmap covers a wide range of topics from programming languages to machine learning techniques, data visualization tools, and DevOps/MLOps tools. It also includes web development frameworks and specific concepts like supervised and unsupervised learning, NLP, deep learning, reinforcement learning, and statistics. Additionally, it delves into DevOps tools like Airflow and MLFlow, data visualization tools like Tableau and Matplotlib, and other topics such as ETL processes, optimization algorithms, and financial modeling.
VMind
VMind is an open-source solution for intelligent visualization, providing an intelligent chart component based on LLM by VisActor. It allows users to create chart narrative works with natural language interaction, edit charts through dialogue, and export narratives as videos or GIFs. The tool is easy to use, scalable, supports various chart types, and offers one-click export functionality. Users can customize chart styles, specify themes, and aggregate data using LLM models. VMind aims to enhance efficiency in creating data visualization works through dialogue-based editing and natural language interaction.
quadratic
Quadratic is a modern multiplayer spreadsheet application that integrates Python, AI, and SQL functionalities. It aims to streamline team collaboration and data analysis by enabling users to pull data from various sources and utilize popular data science tools. The application supports building dashboards, creating internal tools, mixing data from different sources, exploring data for insights, visualizing Python workflows, and facilitating collaboration between technical and non-technical team members. Quadratic is built with Rust + WASM + WebGL to ensure seamless performance in the browser, and it offers features like WebGL Grid, local file management, Python and Pandas support, Excel formula support, multiplayer capabilities, charts and graphs, and team support. The tool is currently in Beta with ongoing development for additional features like JS support, SQL database support, and AI auto-complete.
