ShadowCheck - SIGINT Forensics Platform

🛡️ Production-grade SIGINT forensics and wireless network analysis platform. Real-time threat detection, geospatial correlation via PostGIS, and interactive analysis dashboards.

Features

• Dashboard: Real-time network environment overview with threat indicators
• Geospatial Analysis: Interactive Mapbox visualization with spatial correlation
• Network Analysis: Deep dive into individual network characteristics and behavior patterns
• Threat Detection: ML-powered identification of surveillance devices and anomalies
• Analytics: Advanced charts and graphs for network pattern analysis
• Address Enrichment: Multi-API venue and business identification (4 sources)
• Device Classification: Automatic identification of device types and behavioral profiling
• Trilateration: AP location calculation from multiple observations

Architecture

Backend: Node.js/Express REST API with PostgreSQL + PostGIS
Frontend: Vanilla HTML5 with Tailwind CSS, Chart.js, Mapbox GL JS
Database: PostgreSQL 18 with PostGIS extension (566,400+ location records, 173,326+ unique networks)

Prerequisites

• Node.js 18+
• PostgreSQL 18+ with PostGIS

Quick Start

1. Clone and Install

git clone https://github.com/your-username/shadowcheck-static.git
cd shadowcheck-static
npm install

2. Database Setup

Create PostgreSQL database with PostGIS:

CREATE ROLE shadowcheck WITH LOGIN PASSWORD 'your_password';
CREATE DATABASE shadowcheck OWNER shadowcheck;
\c shadowcheck
CREATE EXTENSION postgis;

3. Environment Configuration

Create .env in project root:

DB_USER=shadowcheck
DB_HOST=localhost
DB_NAME=shadowcheck
DB_PASSWORD=your_password
DB_PORT=5432
PORT=3001

See .env.example for all options.

4. Run Migrations

psql -U shadowcheck -d shadowcheck -f sql/functions/create_scoring_function.sql
psql -U shadowcheck -d shadowcheck -f sql/functions/fix_kismet_functions.sql
psql -U shadowcheck -d shadowcheck -f sql/migrations/migrate_network_tags_v2.sql

5. Start Server

npm start

Server runs on http://localhost:3001

Pages

• Dashboard: /
• Geospatial: /geospatial.html
• Networks: /networks.html
• Analytics: /analytics.html
• Surveillance: /surveillance.html

API Endpoints

• GET /api/networks - All networks
• GET /api/threats/quick - Quick threat detection
• GET /api/analytics/* - Analytics data
• GET /api/networks/observations/:bssid - Network observations

See server.js for full endpoint documentation.

Machine Learning

ShadowCheck includes multi-algorithm threat detection with model training and hyperparameter optimization.

Training Endpoint

POST /api/ml/train

Trains logistic regression model on all tagged networks in database.

Request:

curl -X POST http://localhost:3001/api/ml/train

Response:

{
  "ok": true,
  "model": {
    "type": "logistic_regression",
    "accuracy": 0.92,
    "precision": 0.88,
    "recall": 0.95,
    "f1": 0.91,
    "rocAuc": 0.94
  },
  "trainingData": {
    "totalNetworks": 45,
    "threats": 18,
    "falsePositives": 27
  },
  "message": "Model trained successfully"
}

Errors:

• 400: Fewer than 10 tagged networks (minimum required)
• 503: ML model module unavailable

Status Endpoint

GET /api/ml/status

Check model training status and tag statistics.

Advanced ML Iteration

Test multiple algorithms with grid search and cross-validation:

pip install -r scripts/ml/requirements.txt
python3 scripts/ml/ml-iterate.py

Tests Logistic Regression, Random Forest, and Gradient Boosting with hyperparameter tuning.

Features Used for Training

• Observation count (network detections)
• Unique days seen
• Geographic distribution (location clustering)
• Signal strength (RSSI max)
• Distance range from home location
• Behavioral flags (seen at home vs. away)

Project Structure

shadowcheck-static/
├── server.js              # Express API server
├── public/                # Frontend static files
│   ├── index.html
│   ├── geospatial.html
│   ├── networks.html
│   ├── analytics.html
│   └── surveillance.html
├── scripts/               # Utility scripts
│   ├── enrichment/        # Address enrichment
│   ├── geocoding/         # Reverse geocoding
│   └── ml/                # ML utilities
├── sql/                   # Database
│   ├── migrations/        # Schema migrations
│   └── functions/         # SQL functions
├── docs/                  # Documentation
│   ├── architecture/      # System design docs
│   ├── features/          # Feature documentation
│   ├── guides/            # User guides
│   └── security/          # Security policies
└── data/                  # Data files (not in git)

See docs/DIRECTORY_STRUCTURE.md for complete details.

Development

Run dev server:

npm run dev

Run tests:

npm test

Configuration

Key environment variables (see .env.example):

• DB_* - PostgreSQL connection
• PORT - Server port (default: 3001)
• NODE_ENV - development or production

Security

• Use strong database credentials in production
• Enable HTTPS/TLS at reverse proxy layer
• Restrict API access via rate limiting (already enabled)
• See SECURITY.md for detailed security guidelines

Documentation

Additional documentation is available in the docs directory. See docs/INDEX.md for navigation.

Contributing

See CONTRIBUTING.md for code standards and workflow.

Code of Conduct

See CODE_OF_CONDUCT.md.

License

MIT. See LICENSE for details.